Epochs, Batches, and Convergence
Organize training examples and recognize when learning stabilizes.
Epochs, Batches, and Convergence explained visually
See how a delivery robot studies many road images using epochs, batches, and validation.
What you will learn
Connect classification losses with the complete training loop, batches, epochs, and convergence.
One complete pass
An epoch is one pass through the complete training dataset. Networks commonly train for multiple epochs.
Three update styles
Stochastic training updates after one example, full-batch training waits for the whole dataset, and mini-batches provide a practical balance.
Watching convergence
Training converges when additional updates produce only small useful improvements. Loss should generally decrease rather than jump without control.
Avoid memorization
Validation data checks whether performance transfers to unfamiliar examples. This helps detect overfitting before deployment.
Quick check: test your understanding
What is one epoch?
One pass through the complete training set.
Why use mini-batches?
They balance stable gradients with efficient updates.
What helps reveal overfitting?
Performance on separate validation data.
Accessible referenceLesson 16 transcript346 words · Matches the published narration and captionsOpen transcript
A neural network may train on thousands or millions of examples. To organize that process, we use epochs and batches. Imagine a robot studying one thousand road images. One epoch means the robot has processed every image in the training dataset once. After the first epoch, it begins again with the same training set, usually in a different order. Each pass gives the network another opportunity to improve its weights.
The network can update those weights in several ways. Stochastic gradient descent uses one training example at a time. The network sees one image, calculates the loss, and immediately updates its weights. This creates frequent updates, but the direction can vary considerably from one example to another.
Batch gradient descent takes the opposite approach. The network processes the entire dataset before making one weight update. That produces a stable estimate, but it can require substantial memory and time when the dataset is large. Mini-batch gradient descent provides a practical middle choice. The network processes a small group, such as sixteen or thirty-two images. It calculates the average loss for that group and then updates the weights.
Most modern neural-network training uses mini-batches because they work efficiently with GPUs while still providing frequent updates. As the epochs continue, we monitor the loss. At first, the loss may decrease quickly. Later, the improvement often becomes smaller. Training approaches convergence when additional updates produce little meaningful improvement.
The process can stop after a selected number of epochs. It can also stop when the loss no longer decreases enough or when performance on separate validation data stops improving. More epochs do not automatically create a better model. Excessive training can make the network memorize its training examples instead of learning patterns that transfer to new data.
A successful training process balances learning progress with performance on unseen examples. We now understand how a network measures errors, sends them backward, updates its weights, and organizes training across a dataset. In the next lesson, we will continue from this training foundation and examine the next stage of building an effective neural network.