Backpropagation Step by Step
Trace one complete forward and backward pass through a small six-weight neural network.
Backpropagation Step by Step explained visually
Calculate hidden values, compare a 0.786 prediction with a target of 5, and send correction through all six weights.
What you will learn
Trace derivatives, gradients, and parameter responsibility through a neural network.
Define the small network
Two inputs connect to two hidden neurons, which connect to one output through six weights.
Complete the forward pass
The hidden values are 0.74 and 0.24, producing an output of 0.786.
Begin with output error
The prediction is below the target of 5, so increasing the output would reduce the error.
Update every connection
Chain-rule gradients reach all six weights before the optimizer produces the next parameter values.
Quick check: test your understanding
What output does the example produce?
0.786.
What is the target?
5.
How does an early weight affect the loss?
Through its hidden neuron and the connected output path.
Accessible referenceLesson 36 transcript410 words · Matches the published narration and captionsOpen transcript
Let us follow a small network with two inputs, two hidden neurons, and one output. This simplified example will show how one error reaches every weight. The first input is one point zero, and the second input is zero point four. Six weights connect the network.
Their starting values are zero point five, zero point six, zero point four, negative zero point four, zero point nine, and zero point five. The target output is five. During the forward pass, the first hidden neuron combines the two inputs using the first two weights.
One point zero multiplied by zero point five, plus zero point four multiplied by zero point six, produces zero point seven four. The second hidden neuron uses the next two weights. One point zero multiplied by zero point four, plus zero point four multiplied by negative zero point four, produces zero point two
four. The output neuron combines those hidden values. Zero point seven four multiplied by zero point nine, plus zero point two four multiplied by zero point five, produces zero point seven eight six. The network predicted zero point seven eight six, while the target was five.
The loss therefore reports a large error. Backpropagation begins at the output. It first determines how the loss changes when the prediction changes. Because the prediction is below the target, increasing the output would move the network closer to the correct answer. Next, the chain rule assigns part of that correction to the
two output weights. The first output weight receives a gradient based on the error and the first hidden value. The second receives a gradient based on the same error and the second hidden value. The correction then continues into the hidden layer. Each earlier weight affects the loss through a path containing a
hidden neuron and an output connection. Backpropagation multiplies the upstream gradient by the local derivatives along that path. If the network includes sigmoid or another activation function, its derivative becomes another link in the chain. This changes the gradient magnitude but not the basic logic.
Finally, the optimizer subtracts the learning-rate-scaled gradients from all six weights. The next forward pass produces a different prediction. If the updates helped, the new prediction moves closer to five and the loss decreases. Real networks contain far more layers and parameters, but automatic differentiation performs the same chain-rule calculations efficiently. Before training can begin, the network needs starting weights. Lesson 37 explains why those initial values matter.