Choose a Feature with Information Gain
Distance or weather could be the tree's first question.
Choose a Feature with Information Gain explained visually
Distance or weather could be the tree's first question. English captions are optional and start off. The complete transcript follows the written lesson.
What you will learn
Distance or weather could be the tree's first question.
A practical question
Should the first tree question be about distance or weather? For each candidate split, compare the parent node's impurity with the average impurity of its children, weighted by how many deliveries each child receives.
The useful idea
The reduction is information gain under the chosen impurity measure. A tiny pure child with two trips may contribute little if the much larger child remains mixed. The tree selects a useful local split; it doesn't prove that the chosen feature caused delays or that this is the best possible whole tree.
Work through the example
Calculate both candidates from training data and show the counts beside the scores. Then inspect performance on separate deliveries, especially under conditions missing from the original sample.
Limits and next step
A tree can keep making splits until it memorizes exceptions. Next, we'll limit that growth and validate the decision.
Worked example
Compute each candidate split from training labels and prefer the useful reduction.
Quick check: test your understanding
Try it yourself: In the situation described above, explain the method and one reason its output might be misleading.
Check the reasoning
Information gain compares parent impurity with the weighted impurity after a split. Greedy local improvement does not guarantee the globally best tree.
What is the main idea behind choose a feature with information gain?
Information gain compares parent impurity with the weighted impurity after a split.
How is it used in the example?
Compute each candidate split from training labels and prefer the useful reduction.
What limitation should be checked?
Greedy local improvement does not guarantee the globally best tree.
Further reading
Machine Learning — Tom M. Mitchell.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 26 transcript132 words · Video transcript · Matches the published narration and captionsOpen transcript
Should the first tree question be about distance or weather? For each candidate split, compare the parent node's impurity with the average impurity of its children, weighted by how many deliveries each child receives.
The reduction is information gain under the chosen impurity measure. A tiny pure child with two trips may contribute little if the much larger child remains mixed. The tree selects a useful local split; it doesn't prove that the chosen feature caused delays or that this is the best possible whole tree.
Calculate both candidates from training data and show the counts beside the scores. Then inspect performance on separate deliveries, especially under conditions missing from the original sample.
A tree can keep making splits until it memorizes exceptions. Next, we'll limit that growth and validate the decision.