Tune Neighbor Count on Validation Data
One-neighbor predictions fluctuate wildly on isolated deliveries.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
One-neighbor predictions fluctuate wildly on isolated deliveries.
A practical question
With just one nearest delivery, a single mislabeled or unusual trip can flip the prediction. With twenty neighbors, the result may be calmer but miss a rare storm pattern. Neighbor count controls how local the decision is.
The useful idea
Try one, five, and twenty neighbors under the same scaling and validation split. Compare both overall errors and severe-delay recall. If the larger group is dominated by common on-time routes, a local warning can disappear.
Work through the example
Choose the count on validation data rather than watching the final test score until you like one value. Inspect the actual neighbors of a few disputed trips so the result remains understandable. One nearest trip can be an odd exception; many neighbors can wash out a meaningful local pattern. Compare several neighbor counts on the same validation period, checking both overall error and the actual trips each choice gets wrong. Choose a rule before opening the final test period. If trips from one depot dominate the historical data, the nearest group may still ignore a smaller depot even with a well-chosen count. Inspect the examples rather than treating a tuning curve as a full explanation. Now check the idea on a concrete example: Compare one, five, and twenty neighbors on the same validation period. A possible trap is worth naming: Too many neighbors can wash out unusual but real route patterns. The answer follows from that evidence: Neighbor count controls how local or smooth a nearest-neighbor prediction becomes. Look at a rare snowy delivery. One neighbor may be an accident; many neighbors may include irrelevant dry routes. Compare which past trips contribute under each count, then decide from later outcomes whether the extra examples helped or diluted the useful signal.
Limits and next step
Next, we'll see why simply adding dozens of weak features can make these distances less useful.
Worked example
Compare one, five, and twenty neighbors on the same validation period.
Quick check: test your understanding
Try it yourself: Consider this case: Compare one, five, and twenty neighbors on the same validation period. Explain how you would check the result. In your answer, address this limitation: Too many neighbors can wash out unusual but real route patterns.
Check the reasoning
Neighbor count controls how local or smooth a nearest-neighbor prediction becomes. Too many neighbors can wash out unusual but real route patterns.
What is the main idea behind tune neighbor count on validation data?
Neighbor count controls how local or smooth a nearest-neighbor prediction becomes.
How is it used in the example?
Compare one, five, and twenty neighbors on the same validation period.
What limitation should be checked?
Too many neighbors can wash out unusual but real route patterns.
Further reading
Machine Learning — Tom M. Mitchell.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 43 transcript298 words · Narration ready for the upcoming videoOpen transcript
With just one nearest delivery, a single mislabeled or unusual trip can flip the prediction. With twenty neighbors, the result may be calmer but miss a rare storm pattern. Neighbor count controls how local the decision is.
Try one, five, and twenty neighbors under the same scaling and validation split. Compare both overall errors and severe-delay recall. If the larger group is dominated by common on-time routes, a local warning can disappear.
Choose the count on validation data rather than watching the final test score until you like one value. Inspect the actual neighbors of a few disputed trips so the result remains understandable. One nearest trip can be an odd exception; many neighbors can wash out a meaningful local pattern. Compare several neighbor counts on the same validation period, checking both overall error and the actual trips each choice gets wrong. Choose a rule before opening the final test period. If trips from one depot dominate the historical data, the nearest group may still ignore a smaller depot even with a well-chosen count. Inspect the examples rather than treating a tuning curve as a full explanation. Now check the idea on a concrete example: Compare one, five, and twenty neighbors on the same validation period. A possible trap is worth naming: Too many neighbors can wash out unusual but real route patterns. The answer follows from that evidence: Neighbor count controls how local or smooth a nearest-neighbor prediction becomes. Look at a rare snowy delivery. One neighbor may be an accident; many neighbors may include irrelevant dry routes. Compare which past trips contribute under each count, then decide from later outcomes whether the extra examples helped or diluted the useful signal.
Next, we'll see why simply adding dozens of weak features can make these distances less useful.