Scale Features Before Nearest Neighbors
Mileage dominates an unscaled neighbor distance over a small weather score.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
Mileage dominates an unscaled neighbor distance over a small weather score.
A practical question
Mileage ranges into the hundreds, while the rain flag is either zero or one. In an unscaled numeric distance, mileage can dominate the comparison. Standardizing continuous features can put their differences on a more comparable scale.
The useful idea
Calculate the scaling values from training trips only, then apply the same transformation to validation and future trips. If a future week's data changes the training mean, that would leak information backward into an earlier evaluation. A category flag may need different treatment from a temperature reading.
Work through the example
Compare the nearest trips before and after scaling. Have the neighbors become more relevant to delay, or just different? Evaluate the final warning task instead of assuming that standardized numbers are automatically better. Mileage might be measured in tens of kilometers while another useful input has values near zero or one. Without scaling, mileage can dominate a simple distance calculation merely because of its units. Fit the scaling parameters on the training period and apply them unchanged to later trips. Compare a few neighbors before and after scaling and ask whether they became more relevant to the delay question. Do not learn a fresh mean from the final test period, since that leaks information into evaluation. Use this case to test your understanding: Standardize distance and temperature before comparing nearby trips. Here is the limitation to watch for: Scaling on all records before splitting leaks held-out information. This supports a careful conclusion: Fit scaling parameters on training data, then apply them unchanged to later examples. After scaling, inspect a city trip whose nearest examples changed. If the new neighbors share route conditions and the held-out prediction improves, the transformation has a practical reason. Store the training-period scaler so later trips are treated in the same way.
Limits and next step
Next, we'll decide how many nearby records should vote on a new delivery.
Worked example
Standardize distance and temperature before comparing nearby trips.
Quick check: test your understanding
Try it yourself: Consider this case: Standardize distance and temperature before comparing nearby trips. Explain how you would check the result. In your answer, address this limitation: Scaling on all records before splitting leaks held-out information.
Check the reasoning
Fit scaling parameters on training data, then apply them unchanged to later examples. Scaling on all records before splitting leaks held-out information.
What is the main idea behind scale features before nearest neighbors?
Fit scaling parameters on training data, then apply them unchanged to later examples.
How is it used in the example?
Standardize distance and temperature before comparing nearby trips.
What limitation should be checked?
Scaling on all records before splitting leaks held-out information.
Further reading
Machine Learning — Tom M. Mitchell.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 42 transcript302 words · Narration ready for the upcoming videoOpen transcript
Mileage ranges into the hundreds, while the rain flag is either zero or one. In an unscaled numeric distance, mileage can dominate the comparison. Standardizing continuous features can put their differences on a more comparable scale.
Calculate the scaling values from training trips only, then apply the same transformation to validation and future trips. If a future week's data changes the training mean, that would leak information backward into an earlier evaluation. A category flag may need different treatment from a temperature reading.
Compare the nearest trips before and after scaling. Have the neighbors become more relevant to delay, or just different? Evaluate the final warning task instead of assuming that standardized numbers are automatically better. Mileage might be measured in tens of kilometers while another useful input has values near zero or one. Without scaling, mileage can dominate a simple distance calculation merely because of its units. Fit the scaling parameters on the training period and apply them unchanged to later trips. Compare a few neighbors before and after scaling and ask whether they became more relevant to the delay question. Do not learn a fresh mean from the final test period, since that leaks information into evaluation. Use this case to test your understanding: Standardize distance and temperature before comparing nearby trips. Here is the limitation to watch for: Scaling on all records before splitting leaks held-out information. This supports a careful conclusion: Fit scaling parameters on training data, then apply them unchanged to later examples. After scaling, inspect a city trip whose nearest examples changed. If the new neighbors share route conditions and the held-out prediction improves, the transformation has a practical reason. Store the training-period scaler so later trips are treated in the same way.
Next, we'll decide how many nearby records should vote on a new delivery.