Reading Learning Curves
Use training and validation curves to understand how a neural network changes across epochs.
Reading Learning Curves explained visually
Follow basketball practice and two performance lines to read the evidence inside a learning curve.
What you will learn
Read training signals and distinguish high bias from high variance.
Track progress across training
A learning curve plots a metric such as accuracy or loss against training progress, commonly measured in iterations or epochs.
Compare training and validation
The training curve measures examples used to update the weights. The validation curve uses separate examples and shows whether improvement transfers beyond the training data.
Read the curve patterns
Poor values on both curves can indicate high bias. Excellent training performance with worsening validation performance can indicate high variance.
Treat curves as evidence
Learning curves reveal useful symptoms, but data quality, metrics, implementation errors, and distribution changes must also be investigated.
Quick check: test your understanding
What does the horizontal axis commonly represent?
Training iterations or epochs.
Why use a validation curve?
To check performance on separate examples that do not update the network.
What can a growing gap between curves indicate?
The model may be behaving differently on unseen data.
Accessible referenceLesson 29 transcript383 words · Matches the published narration and captionsOpen transcript
In Lesson 28, we prepared image pixels and noisy signals before giving them to a neural network. Once the data is ready and training begins, we need a way to watch the model's progress. One useful tool is a learning curve. Imagine a teenager practicing basketball free throws every afternoon.
On the first day, only a few shots reach the basket. After more practice, the number of successful shots begins to increase. We could record the result after every practice session and draw a graph. The horizontal axis represents practice time. In neural network training, this axis commonly represents iterations or epochs.
The vertical axis represents performance. It might show accuracy, where a higher value indicates better predictions. It could instead show loss, where a lower value indicates smaller errors. A useful learning curve normally includes two lines. The training line shows performance on examples the network uses to update its weights.
The validation line shows performance on separate examples that do not update the network. Think of training data as the questions used during basketball practice. Validation data acts like a new challenge that checks whether the improvement transfers. At the beginning, both losses may be high because the network has learned very little.
As training continues, the training loss should usually decrease. The validation loss may also decrease when the model learns patterns that work beyond the training examples. The shape of these lines provides useful evidence. If both lines stop improving at poor values, the model may have high bias.
It has not learned enough. If training performance becomes excellent while validation performance becomes worse, the model may have high variance. It learned the training examples too specifically. The distance between the curves also matters. A small gap can indicate that training and validation performance remain similar.
A large gap warns that the model behaves differently on unseen data. Learning curves do not automatically explain the exact cause of every problem. Poor data, unsuitable metrics, implementation errors, and distribution changes can create similar symptoms. Still, these curves give us a practical window into training.
They show when learning improves, when progress stops, and when additional training begins to hurt performance on new examples. In the next lesson, we will examine the first important pattern, learning curves that reveal high bias.