Inspect Shared Errors in an Ensemble
All ensemble members miss the same rural route.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
All ensemble members miss the same rural route.
A practical question
Every tree in the ensemble predicts that a rural delivery will be on time. The real trip is late because of a road closure that no input field described. Combining many votes cannot recover missing evidence.
The useful idea
List the cases where all models are wrong, then group them by route and condition. Are the training examples sparse, labels unreliable, or important features absent? Resampling creates variety in fitted models, but it doesn't automatically create new observations.
Work through the example
Compare one shared failure with a case where the trees disagree. The second may benefit from averaging; the first may require better data or a review rule. Show the dispatcher both patterns instead of only an improved average score. An ensemble can have a strong average score while every member misses the same road closure. Collect cases where all candidate models fail, and group them by weather, depot, and route condition. Shared mistakes point toward missing information, mislabeled outcomes, or a deployment shift that voting cannot fix. Compare with a simple baseline on those cases. The value of combining models is measured by decisions improved for future deliveries, not by the number of models in the pipeline. Use this case to test your understanding: Tabulate cases where every model is wrong, not only average ensemble accuracy. Here is the limitation to watch for: More models cannot vote away a missing feature or biased dataset. This supports a careful conclusion: Combining predictions helps most when their mistakes differ. Choose one rural route that every model predicted incorrectly. A voting system cannot outvote a shared omission. Trace which predeparture information was missing, which labels were available, and whether the same failure appears in a later period before collecting more models.
Limits and next step
Next, we'll study a different classification idea: separating classes with a margin around a decision boundary.
Worked example
Tabulate cases where every model is wrong, not only average ensemble accuracy.
Quick check: test your understanding
Try it yourself: Consider this case: Tabulate cases where every model is wrong, not only average ensemble accuracy. Explain how you would check the result. In your answer, address this limitation: More models cannot vote away a missing feature or biased dataset.
Check the reasoning
Combining predictions helps most when their mistakes differ. More models cannot vote away a missing feature or biased dataset.
What is the main idea behind inspect shared errors in an ensemble?
Combining predictions helps most when their mistakes differ.
How is it used in the example?
Tabulate cases where every model is wrong, not only average ensemble accuracy.
What limitation should be checked?
More models cannot vote away a missing feature or biased dataset.
Further reading
Machine Learning — Tom M. Mitchell.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 48 transcript299 words · Narration ready for the upcoming videoOpen transcript
Every tree in the ensemble predicts that a rural delivery will be on time. The real trip is late because of a road closure that no input field described. Combining many votes cannot recover missing evidence.
List the cases where all models are wrong, then group them by route and condition. Are the training examples sparse, labels unreliable, or important features absent? Resampling creates variety in fitted models, but it doesn't automatically create new observations.
Compare one shared failure with a case where the trees disagree. The second may benefit from averaging; the first may require better data or a review rule. Show the dispatcher both patterns instead of only an improved average score. An ensemble can have a strong average score while every member misses the same road closure. Collect cases where all candidate models fail, and group them by weather, depot, and route condition. Shared mistakes point toward missing information, mislabeled outcomes, or a deployment shift that voting cannot fix. Compare with a simple baseline on those cases. The value of combining models is measured by decisions improved for future deliveries, not by the number of models in the pipeline. Use this case to test your understanding: Tabulate cases where every model is wrong, not only average ensemble accuracy. Here is the limitation to watch for: More models cannot vote away a missing feature or biased dataset. This supports a careful conclusion: Combining predictions helps most when their mistakes differ. Choose one rural route that every model predicted incorrectly. A voting system cannot outvote a shared omission. Trace which predeparture information was missing, which labels were available, and whether the same failure appears in a later period before collecting more models.
Next, we'll study a different classification idea: separating classes with a margin around a decision boundary.