Why Normalization Matters
See why features measured on different scales can make neural-network training difficult.
Why Normalization Matters explained visually
Compare houses, measurements, and optimization paths to understand why consistent feature scales matter.
What you will learn
Diagnose underfitting, understand bias and variance, and prepare features with normalization.
Different units, unfair influence
Area, age, price, and counts can have very different numerical ranges. A large numerical value is not automatically a more important feature.
Comparable feature scales
Normalization transforms measurements into consistent ranges while preserving the useful relationship among examples.
Smoother optimization
Comparable scales can help gradient descent travel toward a useful solution more directly and make training more stable.
Prevent data leakage
Scaling values must be learned from training data and reused unchanged for validation, testing, and production examples.
Quick check: test your understanding
Why can raw feature scales be misleading?
Units can create large numbers without greater importance.
How can normalization help optimization?
It gives features comparable scales and supports more stable updates.
Where should scaling statistics come from?
The training data only.
Accessible referenceLesson 25 transcript328 words · Matches the published narration and captionsOpen transcript
Imagine comparing houses using floor area, number of bedrooms, and age. Floor area might contain values such as two thousand. Bedrooms might range from one to six. Age might range from zero to one hundred. These features describe the same house, but their numerical scales are very different.
A neural network does not automatically understand that a value is large because it uses a different unit. Without preparation, features containing larger numbers can have an unnecessarily large influence on calculations and weight updates. Normalization brings features onto more consistent scales. Think of several runners whose results were recorded using different units. One distance is written in meters, another in kilometers, and another in miles.
Before making a fair comparison, we convert them into a consistent system. Normalization performs a similar job for data. Consistent feature scales can make optimization smoother. Gradient descent can move toward a useful solution without repeatedly making large corrections in one direction and tiny corrections in another.
This often helps training converge faster and more reliably. Normalization can also make weight initialization less sensitive to the original units. The network begins with inputs that occupy more predictable numerical ranges. It may improve generalization as well. The model is encouraged to learn relationships among features instead of depending unnecessarily on their raw scales.
Normalization does not mean every dataset must use exactly the same method. Two widely used approaches are min-max normalization and z-score normalization. Min-max normalization usually transforms values into a selected range, commonly zero to one. Z-score normalization centers values around their mean and describes their distance using standard deviations.
The correct method depends on the data, the model, and how extreme values should be handled. Normalization must also be learned from the training data. Using information from the test set while calculating normalization values can leak knowledge into the training process and produce misleading results. In Lesson 26, we will follow a group of exam scores and transform them using min-max normalization.