Actions, States, and Rewards
The dispatcher may warn, reroute, or wait.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
The dispatcher may warn, reroute, or wait.
A practical question
A warning can prompt the dispatcher to reroute a truck. Now we're choosing an action, not merely predicting an outcome. A reinforcement-learning formulation describes a state, available actions, subsequent states, and a reward for the resulting sequence.
The useful idea
The state might include known traffic, the truck's location, and remaining deliveries. Actions could be continue, reroute, or wait. A reward should reflect not only on-time arrival but also detour cost and effects on other deliveries. A policy maps a state to an action.
Work through the example
Sketch one departure situation and follow what happens after a reroute. If the reward credits only this truck's arrival, the policy may neglect the rest of the fleet. Define the real operational goal before optimizing a simulator. A prediction tells a dispatcher what might happen; a decision policy chooses what to do. Describe the information available before departure, the possible reroute action, and a later reward that reflects on-time delivery and cost. Ask who bears the cost of a route change. A policy cannot learn safely from observed outcomes if the historical data omit why dispatchers chose their routes. Start with a small, reviewable decision problem before considering automation. Now check the idea on a concrete example: Define state before departure, a reroute action, and a later on-time reward. A possible trap is worth naming: The reward must reflect real costs, not just an easy proxy. The answer follows from that evidence: A reinforcement-learning problem has states, actions, rewards, and a policy choosing actions. A route action may save one truck while making a later delivery harder. Ask whose outcomes enter the reward and whether historical dispatch choices were recorded. A policy trained on only successful reroutes could mistake selective past decisions for a universally good action.
Limits and next step
Next, we'll consider why an action with a good immediate result can have a poor later consequence.
Worked example
Define state before departure, a reroute action, and a later on-time reward.
Quick check: test your understanding
Try it yourself: Consider this case: Define state before departure, a reroute action, and a later on-time reward. Explain how you would check the result. In your answer, address this limitation: The reward must reflect real costs, not just an easy proxy.
Check the reasoning
A reinforcement-learning problem has states, actions, rewards, and a policy choosing actions. The reward must reflect real costs, not just an easy proxy.
What is the main idea behind actions, states, and rewards?
A reinforcement-learning problem has states, actions, rewards, and a policy choosing actions.
How is it used in the example?
Define state before departure, a reroute action, and a later on-time reward.
What limitation should be checked?
The reward must reflect real costs, not just an easy proxy.
Further reading
Machine Learning — Tom M. Mitchell.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 69 transcript305 words · Narration ready for the upcoming videoOpen transcript
A warning can prompt the dispatcher to reroute a truck. Now we're choosing an action, not merely predicting an outcome. A reinforcement-learning formulation describes a state, available actions, subsequent states, and a reward for the resulting sequence.
The state might include known traffic, the truck's location, and remaining deliveries. Actions could be continue, reroute, or wait. A reward should reflect not only on-time arrival but also detour cost and effects on other deliveries. A policy maps a state to an action.
Sketch one departure situation and follow what happens after a reroute. If the reward credits only this truck's arrival, the policy may neglect the rest of the fleet. Define the real operational goal before optimizing a simulator. A prediction tells a dispatcher what might happen; a decision policy chooses what to do. Describe the information available before departure, the possible reroute action, and a later reward that reflects on-time delivery and cost. Ask who bears the cost of a route change. A policy cannot learn safely from observed outcomes if the historical data omit why dispatchers chose their routes. Start with a small, reviewable decision problem before considering automation. Now check the idea on a concrete example: Define state before departure, a reroute action, and a later on-time reward. A possible trap is worth naming: The reward must reflect real costs, not just an easy proxy. The answer follows from that evidence: A reinforcement-learning problem has states, actions, rewards, and a policy choosing actions. A route action may save one truck while making a later delivery harder. Ask whose outcomes enter the reward and whether historical dispatch choices were recorded. A policy trained on only successful reroutes could mistake selective past decisions for a universally good action.
Next, we'll consider why an action with a good immediate result can have a poor later consequence.