Recognize High-Dimensional Distance Problems
The dataset adds dozens of weakly relevant delivery features.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
The dataset adds dozens of weakly relevant delivery features.
A practical question
The dispatcher adds thirty fields to every delivery record. Some describe useful traffic conditions; others are weak or duplicate existing information. The nearest trips now look strangely far apart in every direction.
The useful idea
In a high-dimensional representation, data can be sparse and irrelevant differences can overwhelm the few features that matter. Distance rankings may become less informative. More columns do not automatically add more reliable evidence.
Work through the example
Compare held-out results and neighbor lists with a focused feature set and with the expanded one. Remove redundant or unavailable-at-prediction fields, or learn a representation under a valid training protocol. Check whether rare rural routes still find reasonable neighbors. When many weak or redundant inputs are added, two trips may look similarly distant even if one differs on the condition that matters. Remove related or irrelevant features in a controlled comparison and see whether validation neighbors and errors become more useful. Do not use the final test set to decide which inputs to keep. A compact representation helps only if it preserves information available when a real dispatcher asks for a prediction. Record what was dropped and why. Use this case to test your understanding: Compare validation results before and after removing redundant features. Here is the limitation to watch for: Adding dimensions is not guaranteed to improve nearest-neighbor search. This supports a careful conclusion: In high dimensions, distances can become less informative and data can be sparse. With many route indicators, a distant trip can appear close because small differences cancel across numerous fields. Inspect the actual nearest records before and after removing redundant inputs. The useful feature set is the one that supports future decisions, not the longest list.
Limits and next step
Next, we'll combine several unstable trees and see what resampling can and cannot fix.
Worked example
Compare validation results before and after removing redundant features.
Quick check: test your understanding
Try it yourself: Consider this case: Compare validation results before and after removing redundant features. Explain how you would check the result. In your answer, address this limitation: Adding dimensions is not guaranteed to improve nearest-neighbor search.
Check the reasoning
In high dimensions, distances can become less informative and data can be sparse. Adding dimensions is not guaranteed to improve nearest-neighbor search.
What is the main idea behind recognize high-dimensional distance problems?
In high dimensions, distances can become less informative and data can be sparse.
How is it used in the example?
Compare validation results before and after removing redundant features.
What limitation should be checked?
Adding dimensions is not guaranteed to improve nearest-neighbor search.
Further reading
Machine Learning — Tom M. Mitchell.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 44 transcript290 words · Narration ready for the upcoming videoOpen transcript
The dispatcher adds thirty fields to every delivery record. Some describe useful traffic conditions; others are weak or duplicate existing information. The nearest trips now look strangely far apart in every direction.
In a high-dimensional representation, data can be sparse and irrelevant differences can overwhelm the few features that matter. Distance rankings may become less informative. More columns do not automatically add more reliable evidence.
Compare held-out results and neighbor lists with a focused feature set and with the expanded one. Remove redundant or unavailable-at-prediction fields, or learn a representation under a valid training protocol. Check whether rare rural routes still find reasonable neighbors. When many weak or redundant inputs are added, two trips may look similarly distant even if one differs on the condition that matters. Remove related or irrelevant features in a controlled comparison and see whether validation neighbors and errors become more useful. Do not use the final test set to decide which inputs to keep. A compact representation helps only if it preserves information available when a real dispatcher asks for a prediction. Record what was dropped and why. Use this case to test your understanding: Compare validation results before and after removing redundant features. Here is the limitation to watch for: Adding dimensions is not guaranteed to improve nearest-neighbor search. This supports a careful conclusion: In high dimensions, distances can become less informative and data can be sparse. With many route indicators, a distant trip can appear close because small differences cancel across numerous fields. Inspect the actual nearest records before and after removing redundant inputs. The useful feature set is the one that supports future decisions, not the longest list.
Next, we'll combine several unstable trees and see what resampling can and cannot fix.