Random Forest Feature Choices
Every tree keeps choosing mileage first.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
Every tree keeps choosing mileage first.
A practical question
Every bagged tree keeps choosing mileage first. They make similar mistakes because they often build similar early splits. A random forest also samples candidate features at splits, giving other useful signals a chance.
The useful idea
The chosen feature subset and bootstrap records can make trees less correlated. That diversity may improve the ensemble, but feature sampling does not guarantee a better model. A crucial predictor unavailable to too many splits might cause other trade-offs.
Work through the example
Compare tree agreement and held-out errors with different feature choices. Inspect whether rainy rural trips benefit or whether all trees still miss them. Report the effect on the actual warning task, not just the number of trees. A random forest also limits which inputs each tree can consider at a split. That encourages different trees to notice different route patterns instead of all choosing the strongest feature first. Try a few settings using the training and validation periods, then inspect whether their predictions differ on difficult deliveries. Tree diversity helps only when the combined decision improves honest evaluation. Save the feature encoding and forest settings, since changing either can make the same named model behave differently. Use this case to test your understanding: Compare tree diversity when each split sees only a subset of inputs. Here is the limitation to watch for: Feature randomness is a design choice, not a guarantee of independent errors. This supports a careful conclusion: A random forest samples candidate features at splits as well as bootstrap records. A forest can be more diverse without being more accurate on the important trips. Check a later weather period and note which trees miss the same routes. Change the feature-subset setting only through validation, then preserve the final test for the actual comparison.
Limits and next step
Next, we'll build an ensemble in sequence rather than fitting all trees independently.
Worked example
Compare tree diversity when each split sees only a subset of inputs.
Quick check: test your understanding
Try it yourself: Consider this case: Compare tree diversity when each split sees only a subset of inputs. Explain how you would check the result. In your answer, address this limitation: Feature randomness is a design choice, not a guarantee of independent errors.
Check the reasoning
A random forest samples candidate features at splits as well as bootstrap records. Feature randomness is a design choice, not a guarantee of independent errors.
What is the main idea behind random forest feature choices?
A random forest samples candidate features at splits as well as bootstrap records.
How is it used in the example?
Compare tree diversity when each split sees only a subset of inputs.
What limitation should be checked?
Feature randomness is a design choice, not a guarantee of independent errors.
Further reading
Machine Learning — Tom M. Mitchell.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 46 transcript300 words · Narration ready for the upcoming videoOpen transcript
Every bagged tree keeps choosing mileage first. They make similar mistakes because they often build similar early splits. A random forest also samples candidate features at splits, giving other useful signals a chance.
The chosen feature subset and bootstrap records can make trees less correlated. That diversity may improve the ensemble, but feature sampling does not guarantee a better model. A crucial predictor unavailable to too many splits might cause other trade-offs.
Compare tree agreement and held-out errors with different feature choices. Inspect whether rainy rural trips benefit or whether all trees still miss them. Report the effect on the actual warning task, not just the number of trees. A random forest also limits which inputs each tree can consider at a split. That encourages different trees to notice different route patterns instead of all choosing the strongest feature first. Try a few settings using the training and validation periods, then inspect whether their predictions differ on difficult deliveries. Tree diversity helps only when the combined decision improves honest evaluation. Save the feature encoding and forest settings, since changing either can make the same named model behave differently. Use this case to test your understanding: Compare tree diversity when each split sees only a subset of inputs. Here is the limitation to watch for: Feature randomness is a design choice, not a guarantee of independent errors. This supports a careful conclusion: A random forest samples candidate features at splits as well as bootstrap records. A forest can be more diverse without being more accurate on the important trips. Check a later weather period and note which trees miss the same routes. Change the feature-subset setting only through validation, then preserve the final test for the actual comparison.
Next, we'll build an ensemble in sequence rather than fitting all trees independently.