Dropout
Train a network that does not depend too heavily on a few neurons.
Dropout explained visually
Use a basketball team to understand random neuron removal during training.
What you will learn
Recognize overfitting and improve generalization with dropout, regularization, and augmentation.
Change participation
Dropout temporarily turns off randomly selected neuron outputs during each training pass.
Reduce dependence
Changing active pathways prevents a network from relying on the same small group every time.
Train, then predict
Dropout is active during training; normal prediction uses the complete network with appropriate scaling.
Choose the rate carefully
Too little may have limited effect, while too much can make learning difficult.
Quick check: test your understanding
When is dropout applied?
During training.
Why change active neurons?
To reduce dependence on narrow pathways.
What happens during prediction?
The full network participates with the implementation accounting for training-time dropout.
Accessible referenceLesson 19 transcript242 words · Matches the published narration and captionsOpen transcript
Imagine a basketball team that always depends on one star player. The team looks strong until that player is unavailable. A better team trains every player to contribute.
Neural networks can also depend too heavily on a small group of neurons. Dropout reduces that dependence during training.
For each training pass, dropout randomly turns off some neuron outputs. Their incoming and outgoing contributions disappear temporarily. On the next pass, a different group may be inactive. The network repeatedly solves the task with changing combinations of available neurons.
This controlled disruption prevents neurons from relying on the same partners every time. More parts of the network learn useful representations, which can reduce overfitting and improve performance on unseen data.
Dropout applies during training. During normal prediction, the full network participates, with the implementation accounting for the difference in activity.
A dropout setting describes how much activity is retained or removed, depending on the software convention. That detail matters when configuring a model. Too little dropout may provide limited protection. Too much can remove so much information that learning becomes difficult.
Dropout often helps when the network has substantial capacity compared with the available data. It may offer less benefit when the dataset already provides extensive variety.
The main idea resembles the basketball team: changing participation during practice makes the whole system less dependent on one narrow path. Next, we will examine L1 regularization, which can push unnecessary weights all the way to zero.