Bias and Variance
Understand the two forces behind underfitting, overfitting, and model generalization.
Bias and Variance explained visually
Use basketball practice and prediction curves to see how bias and variance shape performance.
What you will learn
Diagnose underfitting, understand bias and variance, and prepare features with normalization.
High bias
An overly rigid model makes consistent mistakes because its rule cannot follow the real pattern. This commonly produces weak training and validation results.
High variance
An overly sensitive model changes dramatically with the training examples. It can fit training data closely but perform poorly on unseen examples.
The trade-off
Increasing complexity can reduce bias while increasing variance. Reducing complexity can improve stability while missing important structure.
Use validation evidence
Validation data helps compare candidate models and locate a practical balance that transfers to new examples.
Quick check: test your understanding
What does high bias suggest?
The model is too rigid to capture the pattern.
What does high variance suggest?
The model is too sensitive to its training examples.
What helps evaluate the balance?
Performance on separate validation data.
Accessible referenceLesson 24 transcript338 words · Matches the published narration and captionsOpen transcript
Underfitting and overfitting may appear to be opposite problems. Bias and variance help us understand why they happen. Imagine three students learning to throw a basketball. The first student repeatedly misses the basket in the same direction. Their throws are consistent, but their technique contains a systematic error.
That resembles high bias. A high-bias model uses assumptions that are too simple for the real problem. It repeatedly misses important patterns and commonly underfits. Imagine using a straight line to describe data that follows a strong curve. No matter how carefully we position that line, it cannot represent the complete relationship.
Now consider a second student whose throws land everywhere. One goes left, another goes right, and another travels too far. That resembles high variance. A high-variance model is extremely sensitive to the particular examples used during training. Small changes in the training data can produce very different predictions.
Complex neural networks trained on limited data can develop high variance. They remember noise, backgrounds, or accidental details instead of learning dependable patterns. A third student consistently places the ball close to the basket. The technique is flexible enough to reach the target but stable enough to avoid wild changes.
That represents the balance we want. A successful model needs sufficiently low bias to capture the real pattern and sufficiently low variance to remain dependable on unseen examples. We call this the bias-variance trade-off. Increasing model complexity can reduce bias, but excessive complexity may increase variance. Stronger regularization may reduce variance, but too much can increase bias.
Validation data helps us evaluate this balance. Poor training and validation performance may indicate high bias. Excellent training performance combined with weak validation performance often indicates high variance. There is no single perfect complexity for every problem. We compare architectures, regularization settings, and validation results to find a practical balance.
Once the model has suitable capacity, the data must also be presented in a form it can learn from efficiently. In Lesson 25, we will see why normalization makes that learning process easier.