From Prediction to Loss

At the end of Lesson 6, we said that computer vision would be our next major topic, and it still will be. But before a neural network can recognize objects in images or understand video, we need to understand one important part of its training process: how it measures mistakes. In this lesson, we will learn about loss, the number that tells a neural network how far its prediction is from the correct answer.

Imagine a teenager named Maya trying to estimate house prices in her neighborhood. She looks at the size of a house, the number of bedrooms, and its location. Then she predicts a price of three hundred thousand dollars. The real selling price is two hundred eighty thousand dollars. Maya's prediction missed the answer by twenty thousand dollars.

A neural network learns in a similar way. It receives an input, produces a prediction, and compares that prediction with the correct answer. The difference becomes an error. A loss function converts that error into a number the network can use. A small loss means the prediction was close. A large loss means the prediction needs more correction.

The network then sends this information backward through its layers. This process is called backpropagation. Backpropagation tells each connection how much it contributed to the mistake. The network adjusts its weights, makes another prediction, and checks the loss again. Repeating this process is how the network learns.

The loss function and the activation function have different jobs. An activation function controls how a neuron passes information forward. A loss function measures how far the final prediction is from the correct answer. For number predictions, such as house prices, one common choice is Mean Squared Error, or M S E.

First, calculate the difference between every prediction and the real value. Next, square each difference. Finally, add the squared differences and divide by the number of examples. Squaring does two useful things. Negative and positive errors no longer cancel each other. It also gives much more importance to large mistakes.

Suppose our model makes five house-price errors: twenty thousand, thirty thousand, ten thousand, negative thirty thousand, and negative twenty thousand dollars. After squaring those errors, their values are four hundred million, nine hundred million, one hundred million, nine hundred million, and four hundred million. Their total is two point eight billion. Divide that by five, and the Mean Squared Error is five hundred sixty million dollars squared.

The unusual squared unit matters less than the comparison. As training continues, a decreasing M S E usually means the predictions are moving closer to the real prices. However, M S E has a weakness. One extremely bad prediction can dominate the result because the error gets squared.

Mean Absolute Error, or M A E, offers another option. Instead of squaring each error, it uses the absolute distance from the correct answer. An error of negative twenty becomes positive twenty. Every mistake contributes according to its size, so one outlier has less influence than it would with M S E.

Huber loss combines both ideas. It treats small errors like M S E, which supports smooth learning. It treats large errors more like M A E, which reduces the influence of outliers. A setting called delta decides where that change happens. So the choice depends on the problem. M S E strongly punishes large mistakes. M A E handles unusual values more gently. Huber loss provides a balance between them.

Now the learning loop is complete: the network predicts, the loss function measures the mistake, backpropagation carries the correction backward, and the weights update before the next attempt. In the next lesson, we will follow one prediction through this loop and see how a weight actually changes.