Frame a Prediction and Baseline
The final project starts with an operational delay question.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
The final project starts with an operational delay question.
A practical question
For the final project, the dispatcher wants to know before departure whether a trip will arrive more than ten minutes late. That sentence defines a decision time and an outcome we can check afterward.
The useful idea
List inputs genuinely available before the truck leaves. Define how lateness is measured, what counts as a canceled trip, and what an alert should trigger. Start with a simple baseline, such as the historical late rate for a route, so every more complex candidate has something honest to beat.
Work through the example
Walk through one late and one on-time trip using the same definitions. If a field only appears after the route starts, remove it from the predeparture task. Write the target as a clear yes-or-no question: was the delivery more than the agreed number of minutes late, measured from a specified schedule? Make a simple route-frequency baseline using only earlier trips. Compare the model against that baseline on a later period. If the label or prediction time changes between examples, a strong score may answer the wrong question. A shared task definition comes before model selection. Now check the idea on a concrete example: Write a yes-or-no ten-minute-late target and a route-frequency baseline. A possible trap is worth naming: An undefined target prevents meaningful evaluation. The answer follows from that evidence: Specify the decision time, label, eligible inputs, baseline, and cost of mistakes. A route-frequency baseline answers the same defined question as the new model with far less complexity. If the new model cannot beat it on later trips that matter to dispatchers, revisit the task and inputs before adopting a more elaborate method. Check the label and prediction time together on a few records to ensure both methods use information that existed before departure. That makes the baseline comparison honest.
Limits and next step
Next, we'll make the split, preprocessing, and training steps reproducible for another person.
Worked example
Write a yes-or-no ten-minute-late target and a route-frequency baseline.
Quick check: test your understanding
Try it yourself: Consider this case: Write a yes-or-no ten-minute-late target and a route-frequency baseline. Explain how you would check the result. In your answer, address this limitation: An undefined target prevents meaningful evaluation.
Check the reasoning
Specify the decision time, label, eligible inputs, baseline, and cost of mistakes. An undefined target prevents meaningful evaluation.
What is the main idea behind frame a prediction and baseline?
Specify the decision time, label, eligible inputs, baseline, and cost of mistakes.
How is it used in the example?
Write a yes-or-no ten-minute-late target and a route-frequency baseline.
What limitation should be checked?
An undefined target prevents meaningful evaluation.
Further reading
Machine Learning — Tom M. Mitchell.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 77 transcript306 words · Narration ready for the upcoming videoOpen transcript
For the final project, the dispatcher wants to know before departure whether a trip will arrive more than ten minutes late. That sentence defines a decision time and an outcome we can check afterward.
List inputs genuinely available before the truck leaves. Define how lateness is measured, what counts as a canceled trip, and what an alert should trigger. Start with a simple baseline, such as the historical late rate for a route, so every more complex candidate has something honest to beat.
Walk through one late and one on-time trip using the same definitions. If a field only appears after the route starts, remove it from the predeparture task. Write the target as a clear yes-or-no question: was the delivery more than the agreed number of minutes late, measured from a specified schedule? Make a simple route-frequency baseline using only earlier trips. Compare the model against that baseline on a later period. If the label or prediction time changes between examples, a strong score may answer the wrong question. A shared task definition comes before model selection. Now check the idea on a concrete example: Write a yes-or-no ten-minute-late target and a route-frequency baseline. A possible trap is worth naming: An undefined target prevents meaningful evaluation. The answer follows from that evidence: Specify the decision time, label, eligible inputs, baseline, and cost of mistakes. A route-frequency baseline answers the same defined question as the new model with far less complexity. If the new model cannot beat it on later trips that matter to dispatchers, revisit the task and inputs before adopting a more elaborate method. Check the label and prediction time together on a few records to ensure both methods use information that existed before departure. That makes the baseline comparison honest.
Next, we'll make the split, preprocessing, and training steps reproducible for another person.