Boosting as Sequential Correction
One model struggles with delayed trips from bad weather.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
One model struggles with delayed trips from bad weather.
A practical question
Our first delivery model underestimates delays in difficult weather. Boosting adds models in stages, with later stages trained to improve the ensemble under a chosen objective.
The useful idea
The update is guided by the current model's loss and predictions, not by a handwritten list of 'bad trips.' Small corrective trees can gradually capture patterns missed by the initial predictor. Too many stages or too much correction can fit noise.
Work through the example
Track validation performance after each stage and inspect errors on held-out routes. If training error keeps falling while validation worsens, stop before the final test period is used as a tuning signal. Boosting builds a sequence: each new learner tries to correct some of the error left by the earlier ones. Watch validation performance as trees are added. At first, later stages may improve difficult trips; eventually they may chase unusual noise in the training records. Keep the final test period closed while deciding when to stop. If a single mislabeled delivery drives repeated corrections, fix the data issue instead of relying on more stages to rescue the model. Now check the idea on a concrete example: Add a small corrective tree, then check validation performance after each stage. A possible trap is worth naming: Too many stages or overly large updates can overfit. The answer follows from that evidence: Boosting trains models in sequence so later stages address errors under a chosen objective. Watch a held-out severe delay while corrective trees are added. Did the warning improve early and then become unstable? Record the stage where validation stops improving and inspect the training examples responsible, rather than choosing the largest possible ensemble by default.
Limits and next step
Keep the loss, learning settings, and chosen stage. Next, we'll examine cases where all our ensemble members agree and are still wrong.
Worked example
Add a small corrective tree, then check validation performance after each stage.
Quick check: test your understanding
Try it yourself: Consider this case: Add a small corrective tree, then check validation performance after each stage. Explain how you would check the result. In your answer, address this limitation: Too many stages or overly large updates can overfit.
Check the reasoning
Boosting trains models in sequence so later stages address errors under a chosen objective. Too many stages or overly large updates can overfit.
What is the main idea behind boosting as sequential correction?
Boosting trains models in sequence so later stages address errors under a chosen objective.
How is it used in the example?
Add a small corrective tree, then check validation performance after each stage.
What limitation should be checked?
Too many stages or overly large updates can overfit.
Further reading
Machine Learning — Tom M. Mitchell.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 47 transcript294 words · Narration ready for the upcoming videoOpen transcript
Our first delivery model underestimates delays in difficult weather. Boosting adds models in stages, with later stages trained to improve the ensemble under a chosen objective.
The update is guided by the current model's loss and predictions, not by a handwritten list of 'bad trips.' Small corrective trees can gradually capture patterns missed by the initial predictor. Too many stages or too much correction can fit noise.
Track validation performance after each stage and inspect errors on held-out routes. If training error keeps falling while validation worsens, stop before the final test period is used as a tuning signal. Boosting builds a sequence: each new learner tries to correct some of the error left by the earlier ones. Watch validation performance as trees are added. At first, later stages may improve difficult trips; eventually they may chase unusual noise in the training records. Keep the final test period closed while deciding when to stop. If a single mislabeled delivery drives repeated corrections, fix the data issue instead of relying on more stages to rescue the model. Now check the idea on a concrete example: Add a small corrective tree, then check validation performance after each stage. A possible trap is worth naming: Too many stages or overly large updates can overfit. The answer follows from that evidence: Boosting trains models in sequence so later stages address errors under a chosen objective. Watch a held-out severe delay while corrective trees are added. Did the warning improve early and then become unstable? Record the stage where validation stops improving and inspect the training examples responsible, rather than choosing the largest possible ensemble by default.
Keep the loss, learning settings, and chosen stage. Next, we'll examine cases where all our ensemble members agree and are still wrong.