Measure What Is Known at Prediction Time
A late-arrival field appears highly predictive in historical data.
Measure What Is Known at Prediction Time explained visually
A late-arrival field appears highly predictive in historical data. English captions are optional and start off. The complete transcript follows the written lesson.
What you will learn
A late-arrival field appears highly predictive in historical data.
A practical question
A column called 'latest arrival estimate' predicts delays almost perfectly. But when was it entered? If the estimate came after the driver had already fallen behind, it cannot support a warning before departure.
The useful idea
Fix the prediction time first. Rebuild each past example using only values recorded by that moment. An as-of lookup needs the actual timestamps of data availability, not just a convenient date column. Even a field stored under the trip's start date may contain a later update.
Work through the example
Replay one historical warning from the dispatch screen as it looked before the truck left. Remove anything learned afterward and compare the score. If performance drops, that is a more honest estimate of the proposed service.
Limits and next step
Keep feature availability in the pipeline contract. Next, we'll use a decision tree to split the remaining valid examples.
Worked example
Replay one old prediction using only values recorded before the vehicle left.
Quick check: test your understanding
Try it yourself: In the situation described above, explain the method and one reason its output might be misleading.
Check the reasoning
Every feature must be available at the actual decision time, before departure here. A timestamped table can contain facts that were added later.
What is the main idea behind measure what is known at prediction time?
Every feature must be available at the actual decision time, before departure here.
How is it used in the example?
Replay one old prediction using only values recorded before the vehicle left.
What limitation should be checked?
A timestamped table can contain facts that were added later.
Further reading
Machine Learning — Tom M. Mitchell.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 24 transcript134 words · Video transcript · Matches the published narration and captionsOpen transcript
A column called 'latest arrival estimate' predicts delays almost perfectly. But when was it entered? If the estimate came after the driver had already fallen behind, it cannot support a warning before departure.
Fix the prediction time first. Rebuild each past example using only values recorded by that moment. An as-of lookup needs the actual timestamps of data availability, not just a convenient date column. Even a field stored under the trip's start date may contain a later update.
Replay one historical warning from the dispatch screen as it looked before the truck left. Remove anything learned afterward and compare the score. If performance drops, that is a more honest estimate of the proposed service.
Keep feature availability in the pipeline contract. Next, we'll use a decision tree to split the remaining valid examples.