Inductive Bias: Why Examples Are Not Enough
The team must decide which routes to compare before looking at all future trips.
Inductive Bias: Why Examples Are Not Enough explained visually
The team must decide which routes to compare before looking at all future trips. English captions are optional and start off. The complete transcript follows the written lesson.
What you will learn
The team must decide which routes to compare before looking at all future trips.
A practical question
We have seen several hundred deliveries, but tomorrow brings a route we've never recorded. Many rules can agree on the old trips while disagreeing on the new one. How does a learner choose? It relies on an inductive bias: a restriction or preference about plausible solutions.
The useful idea
One approach favors short, simple rules. Another memorizes every training record in a lookup table. The table can be perfect on known trips while having no principled answer for a new route. Simplicity can help, but a too-simple rule can miss a real weather interaction.
Work through the example
Compare what both approaches say for a held-out route, then inspect the errors. Neither set of assumptions is a theorem about the future. State the bias so the dispatcher knows what kind of generalization is being attempted.
Limits and next step
Next, we'll check whether the training records themselves are trustworthy before fitting any rule.
Worked example
Compare a learner favoring simple rules with one allowing arbitrary lookup tables.
Quick check: test your understanding
Try it yourself: In the situation described above, explain the method and one reason its output might be misleading.
Check the reasoning
Inductive bias is the preference or restriction that lets a learner generalize beyond observed examples. Examples alone do not uniquely determine the correct prediction for unseen cases.
What is the main idea behind inductive bias: why examples are not enough?
Inductive bias is the preference or restriction that lets a learner generalize beyond observed examples.
How is it used in the example?
Compare a learner favoring simple rules with one allowing arbitrary lookup tables.
What limitation should be checked?
Examples alone do not uniquely determine the correct prediction for unseen cases.
Further reading
Machine Learning — Tom M. Mitchell.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 20 transcript142 words · Video transcript · Matches the published narration and captionsOpen transcript
We have seen several hundred deliveries, but tomorrow brings a route we've never recorded. Many rules can agree on the old trips while disagreeing on the new one. How does a learner choose? It relies on an inductive bias: a restriction or preference about plausible solutions.
One approach favors short, simple rules. Another memorizes every training record in a lookup table. The table can be perfect on known trips while having no principled answer for a new route. Simplicity can help, but a too-simple rule can miss a real weather interaction.
Compare what both approaches say for a held-out route, then inspect the errors. Neither set of assumptions is a theorem about the future. State the bias so the dispatcher knows what kind of generalization is being attempted.
Next, we'll check whether the training records themselves are trustworthy before fitting any rule.