Confusion Matrix for Real Decisions
The owner wants to see missed delays and unnecessary warnings separately.
Confusion Matrix for Real Decisions explained visually
The owner wants to see missed delays and unnecessary warnings separately. English captions are optional and start off. The complete transcript follows the written lesson.
What you will learn
The owner wants to see missed delays and unnecessary warnings separately.
A practical question
The dispatcher asks a practical question: how many late deliveries did we miss, and how many unnecessary warnings did we send? A confusion matrix separates those outcomes at a chosen warning threshold.
The useful idea
Count flagged late trips as true positives, missed late trips as false negatives, warnings on on-time trips as false positives, and unflagged on-time trips as true negatives. Recall divides true positives by all actual late trips. Precision divides true positives by all flagged trips. They answer different questions.
Work through the example
For twenty truly late trips, count how many were flagged before reporting recall. Then count all the warnings before reporting precision. Show the raw counts; a percentage can conceal a tiny test sample.
Limits and next step
The matrix belongs to one threshold and one evaluation population. Next, we'll construct validation folds that keep related deliveries and future weeks out of training.
Worked example
From 20 actual late trips, count how many were flagged and how many were missed.
Quick check: test your understanding
Try it yourself: In the situation described above, explain the method and one reason its output might be misleading.
Check the reasoning
A confusion matrix counts true positives, false positives, true negatives, and false negatives for a chosen threshold. Accuracy alone hides the cost of each kind of mistake.
What is the main idea behind confusion matrix for real decisions?
A confusion matrix counts true positives, false positives, true negatives, and false negatives for a chosen threshold.
How is it used in the example?
From 20 actual late trips, count how many were flagged and how many were missed.
What limitation should be checked?
Accuracy alone hides the cost of each kind of mistake.
Further reading
Machine Learning — Tom M. Mitchell.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 33 transcript138 words · Video transcript · Matches the published narration and captionsOpen transcript
The dispatcher asks a practical question: how many late deliveries did we miss, and how many unnecessary warnings did we send? A confusion matrix separates those outcomes at a chosen warning threshold.
Count flagged late trips as true positives, missed late trips as false negatives, warnings on on-time trips as false positives, and unflagged on-time trips as true negatives. Recall divides true positives by all actual late trips. Precision divides true positives by all flagged trips. They answer different questions.
For twenty truly late trips, count how many were flagged before reporting recall. Then count all the warnings before reporting precision. Show the raw counts; a percentage can conceal a tiny test sample.
The matrix belongs to one threshold and one evaluation population. Next, we'll construct validation folds that keep related deliveries and future weeks out of training.