Capstone: Build and Compare a Prediction System
Now let us assemble a complete learning project: predict delivery delay before departure.
Capstone: Build and Compare a Prediction System explained visually
Now let us assemble a complete learning project: predict delivery delay before departure. English captions are optional and start off. The complete transcript follows the written lesson.
What you will learn
Now let us assemble a complete learning project: predict delivery delay before departure.
Define a reproducible experiment
Now let us assemble a complete learning project: predict delivery delay before departure. Define the target in minutes, list features available at that time, and decide how errors will be measured. Keep a simple historical-average baseline. This gives every later model a clear standard to beat.
Compare useful alternatives
Use a split that reflects future deployment, such as earlier dates for training and later dates for validation and testing. Inspect data quality, fit preprocessing within the training process, and compare a small set of models. Linear regression, a tree-based model, and a carefully tuned alternative are enough to begin. Record settings and computational cost instead of trying every possible algorithm without a plan.
Write an honest result
Select the system using validation evidence, then assess it on the reserved test set. Report average errors together with important subgroups and difficult examples. Include the baseline, limitations, prediction-time requirements, and monitoring plan. If the complex model barely improves accuracy but is much slower or harder to maintain, the simpler model may be the better operational choice. A successful project produces an understandable decision system and a reproducible report, not only a high score.
A worked example
Suppose the baseline has a mean absolute error of eight minutes and the selected model has six minutes on the same test set. The absolute improvement is two minutes, and the relative reduction is twenty-five percent. Report the dataset and split context, and inspect whether long-delay cases also improved.
Worked example
Suppose the baseline has a mean absolute error of eight minutes and the selected model has six minutes on the same test set. The absolute improvement is two minutes, and the relative reduction is 25 percent. Report the dataset and split context, and inspect whether long-delay cases also improved.
Quick check: test your understanding
Try it yourself: Prepare a project report with the target, available features, split rule, baseline, chosen pipeline, metrics, difficult cases, and update plan.
Check the reasoning
The report should let another person reproduce the comparison and understand where the system may fail. Keep the final test separate from the choices that selected the model.
Why include a simple baseline in the final report?
It shows whether the additional complexity delivers a meaningful improvement for the actual task.
How would you solve the practice case?
The report should let another person reproduce the comparison and understand where the system may fail. Keep the final test separate from the choices that selected the model.
What does the worked example show?
Suppose the baseline has a mean absolute error of eight minutes and the selected model has six minutes on the same test set. The absolute improvement is two minutes, and the relative reduction is 25 percent. Report the dataset and split context, and inspect whether long-delay cases also improved.
Further reading
Machine Learning — Tom M. Mitchell.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 16 transcript233 words · Video transcript · Matches the published narration and captionsOpen transcript
Now let us assemble a complete learning project: predict delivery delay before departure. Define the target in minutes, list features available at that time, and decide how errors will be measured. Keep a simple historical-average baseline. This gives every later model a clear standard to beat.
Use a split that reflects future deployment, such as earlier dates for training and later dates for validation and testing. Inspect data quality, fit preprocessing within the training process, and compare a small set of models. Linear regression, a tree-based model, and a carefully tuned alternative are enough to begin. Record settings and computational cost instead of trying every possible algorithm without a plan.
Select the system using validation evidence, then assess it on the reserved test set. Report average errors together with important subgroups and difficult examples. Include the baseline, limitations, prediction-time requirements, and monitoring plan. If the complex model barely improves accuracy but is much slower or harder to maintain, the simpler model may be the better operational choice. A successful project produces an understandable decision system and a reproducible report, not only a high score.
Suppose the baseline has a mean absolute error of eight minutes and the selected model has six minutes on the same test set. The absolute improvement is two minutes, and the relative reduction is twenty-five percent. Report the dataset and split context, and inspect whether long-delay cases also improved.