How a Neural Network Learns
See how feedback helps a network improve its weights.
How a Neural Network Learns explained visually
Use basketball practice to understand prediction, error, correction, and evaluation.
Make a prediction
Forward propagation moves an example through the network from input to output. The result may be right or wrong, but it gives training a prediction to evaluate.
Measure the error
A loss function compares the prediction with the correct answer. A small loss means the prediction was close; a larger loss signals that the network needs a greater correction.
Send correction backward
Backpropagation calculates how each weight and bias contributed to the error. An optimizer then changes those parameters, while the learning rate controls the size of each step.
Check new examples
Training repeats over many examples and epochs. Separate evaluation data tests whether the network learned useful patterns that transfer to unfamiliar examples instead of merely memorizing its training set.
Quick check
What is forward propagation?
Moving information from the input through the network to a prediction.
What does loss measure?
The difference between a prediction and the correct answer.
Why use evaluation data?
To test whether learning works on examples not used for parameter updates.
Lesson 5 transcript
Imagine practicing basketball shots with a coach. You take a shot, observe where the ball lands, and make a small correction. If the ball goes too far left, you adjust your aim. Repeating this process gradually improves your accuracy.
A neural network learns through a similar cycle. First, it receives an example. Next, it produces a prediction through forward propagation, where information moves from the input layer toward the output.
Suppose the network sees a zebra but predicts that it is a dog. Because the training example includes the correct answer, the network can compare its prediction with the truth.
A loss function measures the size of the error. A small loss means the prediction was close to the correct answer. A large loss means the network needs a larger improvement.
The network then determines how its weights and biases contributed to that error. A training method called backpropagation carries this information backward through the network. It calculates how a small change in each parameter would affect the loss.
An optimizer uses that information to update the parameters. The learning rate controls the size of each update. If the steps are too large, the network may jump past a good solution. If they are too small, training can take a long time.
The network repeats this cycle across many examples. Predict, measure the error, send the correction backward, and update the parameters. One pass through the complete training dataset is called an epoch. Training often uses multiple epochs so the network has several opportunities to improve.
We also evaluate the model on data that it did not use to update its parameters. This helps us check whether it learned useful patterns instead of simply memorizing the training examples.
Learning does not mean that the computer understands the world like a person. It means that the training process found weights and biases that produce better predictions for the examples and task.
In the final lesson, we will compare classification, regression, and clustering, three common ways to use learning systems.