Balancing Bias and Variance
Use controlled evidence to find a model that learns well and generalizes reliably.
Balancing Bias and Variance explained visually
Compare three lane-detection models to choose a practical balance between bias and variance.
What you will learn
Read training signals and distinguish high bias from high variance.
Avoid both extremes
High bias misses important structure, while high variance captures training details that do not transfer to new conditions.
Use the curves to choose the next step
Poor training and validation results point toward bias. Strong training results with weak validation performance point toward variance.
Run controlled experiments
Change one important factor at a time so the effect of data, capacity, regularization, or training strategy remains visible.
Protect the final test
Use validation data for development decisions, then use the test set for a separate final evaluation without repeatedly redesigning around its results.
Quick check: test your understanding
What indicates a useful balance?
Low training error with validation error that remains close and useful.
Why change one factor at a time?
To identify which decision changed the result.
What is the final role of the test set?
To provide a separate evaluation after model and settings are selected.
Accessible referenceLesson 32 transcript407 words · Matches the published narration and captionsOpen transcript
A useful neural network must avoid two different failures. High bias creates a model that cannot capture the important pattern. High variance creates a model that captures training details that do not transfer to new examples. Balancing them requires evidence. Imagine designing a lane detection system for an autonomous delivery robot.
The first model uses a rigid rule based only on road brightness. It misses lanes in shadows and at night. Both its training and validation errors remain high, this model has excessive bias. The second model uses a very large network trained on a small collection of road images.
It recognizes nearly every training image perfectly. However, its validation performance becomes much worse when the road, lighting, or camera angle changes. This model has excessive variance. A third model uses enough capacity to recognize edges, lane markings, and road boundaries. Its training error becomes low, while its validation error remains close and reaches
a useful level. This model provides a better balance. We find that balance by comparing controlled experiments. Begin with a reasonable baseline model. Record its training and validation curves using the same evaluation metric. If both results remain poor, investigate high bias. More useful features, longer appropriate training, or additional model capacity may help.
If training performance is strong but validation performance is weak, investigate high variance. More varied data, suitable augmentation, regularization, early stopping, or a smaller model may improve generalization. Change one important factor at a time. Otherwise, we cannot identify which decision improved or damaged the result.
The validation set guides these design choices during development. After selecting the final model and settings, the test set provides a separate evaluation. Repeatedly using test results to redesign the model would allow test information to influence development and weaken the fairness of the final evaluation.
No single model size creates the correct balance for every project. The answer depends on the difficulty of the task, the amount and variety of available data, and the conditions the model will encounter after deployment. Learning curves make this process visible. They help us separate insufficient learning from excessive specialization.
They also help us decide whether the next improvement should come from the data, the model, or the training strategy. We now know how to monitor training and diagnose bias and variance. In the next lesson, we will look inside the training process again and examine back propagation, gradients, and how each weight receives its correction.