Random Forests: Combine Different Trees
One decision tree can change dramatically when a few training examples change.
Random Forests: Combine Different Trees explained visually
One decision tree can change dramatically when a few training examples change. English captions are optional and start off. The complete transcript follows the written lesson.
What you will learn
One decision tree can change dramatically when a few training examples change.
Reduce dependence on one tree
One decision tree can change dramatically when a few training examples change. A random forest combines many trees to reduce dependence on one particular set of splits. Each tree sees a randomized training sample, and candidate features are typically randomized during splitting. The resulting predictions are combined by averaging or voting.
Create useful diversity
The benefit comes from combining predictors whose errors are not perfectly correlated. Building many identical trees would add little. Forests can capture nonlinear relationships and interactions with limited preprocessing, but they still depend on the quality of the features and labels. More trees do not repair leakage, missing coverage, or a target defined incorrectly.
Evaluate the ensemble
Out-of-bag observations can provide an internal performance estimate for bootstrap-based forests, but the evaluation assumptions still matter. Use a deployment-relevant validation design, especially for grouped or time-dependent data. Feature-importance values also need care: correlated features can share or distort importance, and some measures favor certain feature types. Inspect predictions and error groups rather than treating one importance chart as a complete explanation.
A worked example
If three regression trees predict ten, twelve, and fourteen minutes of delay, their simple average is twelve minutes. Averaging can reduce variability, but if all three learned the same wrong relationship, their agreement does not make the result correct.
Worked example
If three regression trees predict 10, 12, and 14 minutes of delay, their simple average is 12 minutes. Averaging can reduce variability, but if all three learned the same wrong relationship, their agreement does not make the result correct.
Quick check: test your understanding
Try it yourself: Why might hundreds of highly similar trees offer less improvement than a diverse collection?
Check the reasoning
Their errors may be strongly correlated, leaving little independent variation for averaging to reduce.
Can a random forest remove leakage introduced before training?
No. It may exploit that leakage very effectively and produce misleading validation scores.
How would you solve the practice case?
Their errors may be strongly correlated, leaving little independent variation for averaging to reduce.
What does the worked example show?
If three regression trees predict 10, 12, and 14 minutes of delay, their simple average is 12 minutes. Averaging can reduce variability, but if all three learned the same wrong relationship, their agreement does not make the result correct.
Further reading
Machine Learning — Tom M. Mitchell.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 9 transcript206 words · Video transcript · Matches the published narration and captionsOpen transcript
One decision tree can change dramatically when a few training examples change. A random forest combines many trees to reduce dependence on one particular set of splits. Each tree sees a randomized training sample, and candidate features are typically randomized during splitting. The resulting predictions are combined by averaging or voting.
The benefit comes from combining predictors whose errors are not perfectly correlated. Building many identical trees would add little. Forests can capture nonlinear relationships and interactions with limited preprocessing, but they still depend on the quality of the features and labels. More trees do not repair leakage, missing coverage, or a target defined incorrectly.
Out-of-bag observations can provide an internal performance estimate for bootstrap-based forests, but the evaluation assumptions still matter. Use a deployment-relevant validation design, especially for grouped or time-dependent data. Feature-importance values also need care: correlated features can share or distort importance, and some measures favor certain feature types. Inspect predictions and error groups rather than treating one importance chart as a complete explanation.
If three regression trees predict ten, twelve, and fourteen minutes of delay, their simple average is twelve minutes. Averaging can reduce variability, but if all three learned the same wrong relationship, their agreement does not make the result correct.