Balance Exploration with Safe Decisions
A new route might be faster but has little evidence.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
A new route might be faster but has little evidence.
A practical question
An unfamiliar route might be faster, but the evidence is thin. Using the current best-known route is exploitation; trying another to learn its performance is exploration. Both have a place, but a live delivery has real stakes.
The useful idea
Start with bounded, monitored trials on trips where a detour is acceptable. Compare outcomes with the established route under similar conditions, and set limits on service delays. A simulator can help design a test, but its assumptions also need checking against reality.
Work through the example
Don't reroute every critical shipment just to gather data. Document who may approve an experiment, when it stops, and how its results affect the policy. New evidence should be collected without hiding the cost of learning. Learning about a new route can require trying it, but experimenting on every critical delivery would be reckless. Reserve a limited, monitored group where a human can intervene, and compare it with the current policy under similar conditions. Record why each action was selected so later analysis can separate a policy effect from easy cases being chosen first. Stop the trial when a predefined safety limit is crossed. Exploration is useful only when its evidence is gathered responsibly. Use this case to test your understanding: Try a limited, monitored experiment rather than rerouting every critical delivery at once. Here is the limitation to watch for: Operational safety constraints may rule out some experiments. This supports a careful conclusion: Exploration gathers information; exploitation chooses the currently preferred action. Set a limit on how many low-risk deliveries enter the trial and how quickly a person can stop it. Compare outcomes with similar current-policy trips. A trial that only selects easy days does not show how the route performs when the choice matters most.
Limits and next step
Next, we'll return to a more immediate deployment risk for the prediction service: data leakage.
Worked example
Try a limited, monitored experiment rather than rerouting every critical delivery at once.
Quick check: test your understanding
Try it yourself: Consider this case: Try a limited, monitored experiment rather than rerouting every critical delivery at once. Explain how you would check the result. In your answer, address this limitation: Operational safety constraints may rule out some experiments.
Check the reasoning
Exploration gathers information; exploitation chooses the currently preferred action. Operational safety constraints may rule out some experiments.
What is the main idea behind balance exploration with safe decisions?
Exploration gathers information; exploitation chooses the currently preferred action.
How is it used in the example?
Try a limited, monitored experiment rather than rerouting every critical delivery at once.
What limitation should be checked?
Operational safety constraints may rule out some experiments.
Further reading
Machine Learning — Tom M. Mitchell.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 72 transcript301 words · Narration ready for the upcoming videoOpen transcript
An unfamiliar route might be faster, but the evidence is thin. Using the current best-known route is exploitation; trying another to learn its performance is exploration. Both have a place, but a live delivery has real stakes.
Start with bounded, monitored trials on trips where a detour is acceptable. Compare outcomes with the established route under similar conditions, and set limits on service delays. A simulator can help design a test, but its assumptions also need checking against reality.
Don't reroute every critical shipment just to gather data. Document who may approve an experiment, when it stops, and how its results affect the policy. New evidence should be collected without hiding the cost of learning. Learning about a new route can require trying it, but experimenting on every critical delivery would be reckless. Reserve a limited, monitored group where a human can intervene, and compare it with the current policy under similar conditions. Record why each action was selected so later analysis can separate a policy effect from easy cases being chosen first. Stop the trial when a predefined safety limit is crossed. Exploration is useful only when its evidence is gathered responsibly. Use this case to test your understanding: Try a limited, monitored experiment rather than rerouting every critical delivery at once. Here is the limitation to watch for: Operational safety constraints may rule out some experiments. This supports a careful conclusion: Exploration gathers information; exploitation chooses the currently preferred action. Set a limit on how many low-risk deliveries enter the trial and how quickly a person can stop it. Compare outcomes with similar current-policy trips. A trial that only selects easy days does not show how the route performs when the choice matters most.
Next, we'll return to a more immediate deployment risk for the prediction service: data leakage.