Diagnosing High Bias
Recognize when a model is too restricted to learn the important pattern.
Diagnosing High Bias explained visually
Use an overly simple animal rule and a street camera to identify high bias and underfitting.
What you will learn
Read training signals and distinguish high bias from high variance.
A rule that is too restrictive
High bias occurs when a model's assumptions or available information are too limited to describe the real relationship.
Look for weak results on both datasets
Training and validation losses may remain high, close together, and flat at an unsatisfactory level. A small gap is not useful when both results are poor.
Investigate the cause
Insufficient capacity, missing features, early stopping, or excessive regularization can all prevent the model from learning enough.
Change the system deliberately
Appropriate additional training, more useful features, suitable capacity, or reduced excessive regularization may help. Compare the curves after each controlled change.
Quick check: test your understanding
What does high bias mean?
The model cannot learn enough of the useful pattern.
How do its curves often look?
Training and validation results are both poor and may remain close together.
Will more epochs always solve it?
No. More training is unlikely to help after both curves have flattened.
Accessible referenceLesson 30 transcript387 words · Matches the published narration and captionsOpen transcript
In Lesson 29, we learned how training and validation curves reveal a model's progress across epochs. Now imagine a student using an overly simple rule to identify animals. The student says that every animal with four legs is a dog. That rule can recognize some dogs, but it also calls cats, horses, and goats dogs.
The rule cannot describe the real pattern. This resembles high bias. High bias occurs when a model makes assumptions that are too restrictive for the problem. Consider a camera that must distinguish pedestrians from bicycles on a city street. Suppose the model receives only one feature, the height of the detected object.
Some pedestrians and bicycles may have similar heights. That single feature cannot separate them reliably. During training, the model makes many mistakes. Its training loss remains high. Its validation loss also remains high. The two learning curves may sit close together and stop improving at an unsatisfactory level.
The small gap does not mean the model performs well. It means the model performs similarly poorly on both datasets. This pattern suggests underfitting. Several conditions can produce it. The model may have too few useful layers or neurons. The input features may omit important information.
Training may stop too early. Excessive regularization may prevent the weights from adapting enough. We should diagnose the cause before changing the system. If both curves continue improving, more training may help. If they have already flattened, simply adding more epochs will probably accomplish very little.
A more capable model may capture the missing structure. For the street camera, image features describing shape, wheels, motion, and body position could provide stronger evidence than height alone. Reducing excessive regularization may also help when the model has been restricted too strongly. The goal is not to make the network as large as possible.
Additional complexity should address evidence visible in the learning curves and validation results. After every change, we compare the curves again. If training and validation performance improve together, the model is learning a more useful rule. High bias means the model cannot learn enough from the available training information.
But a model can also move too far in the opposite direction. It may learn the training set extremely well and fail on new examples. In the next lesson, we will diagnose that pattern, high variance.