Huber Loss
Combine smooth learning for small errors with gentler treatment of outliers.
Huber Loss explained visually
Use careful parking corrections to understand Huber loss and its delta boundary.
What you will learn
Measure numerical prediction errors and compare MSE, MAE, and Huber loss.
Use smooth corrections nearby
Huber loss behaves like MSE for small errors, supporting precise and smooth updates.
Limit extreme influence
For larger errors, the loss grows more like MAE so outliers exert less control.
Set the delta boundary
Delta determines where Huber loss changes from squared behavior to linear behavior.
Choose for the data
Huber loss can help when most examples are reliable but occasional outliers appear.
Quick check: test your understanding
How does Huber loss treat small errors?
Like Mean Squared Error.
How does it treat large errors?
More like Mean Absolute Error.
What does delta control?
The point where the loss changes between those behaviors.
Accessible referenceLesson 11 transcript372 words · Matches the published narration and captionsOpen transcript
Mean Squared Error learns smoothly but can give too much power to extreme mistakes. Mean Absolute Error handles outliers more gently but has a sharp corner near zero. Huber loss combines useful behavior from both. Imagine coaching a new driver to park in the center of a space.
When the car is only a few inches away from the center, small and smooth steering corrections work well. But if the car is several feet away, multiplying the correction too aggressively could cause the driver to overreact. A controlled, steady correction is safer. Huber loss follows a similar idea.
For small prediction errors, it behaves like Mean Squared Error. The errors are squared, creating a smooth curve that helps the model make precise adjustments. For large errors, Huber loss changes its behavior. Instead of allowing the squared value to grow rapidly, it increases more like Mean Absolute Error.
A setting called delta controls where this change happens. Errors smaller than delta use the squared region. Errors larger than delta move into the gentler linear region. Imagine delta as a boundary around the center of our parking space. Inside the boundary, the model makes fine corrections. Outside the boundary, it still responds to the mistake, but it prevents that error from completely dominating training.
A small delta makes Huber loss behave more like Mean Absolute Error. A larger delta allows more errors to receive the squared treatment, making it behave more like Mean Squared Error. Choosing delta depends on the scale and behavior of the data. There is no single perfect value for every problem.
Huber loss is useful when most examples are reliable but occasional outliers appear. It offers smooth learning for normal mistakes while limiting the influence of extreme ones. We have now compared three regression loss functions. Mean Squared Error strongly emphasizes large mistakes. Mean Absolute Error treats errors according to their direct distance.
Huber loss creates a controlled balance between them. The correct loss function depends on what errors mean in the real problem, not simply which formula looks easiest. In the next lesson, we will move from predicting numbers to predicting classes. We will learn how Binary Cross-Entropy measures mistakes when a neural network chooses between two possible classes.