Read Precision-Recall for a Detector
A threshold misses distant signs while a lower one flags reflections.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
A threshold misses distant signs while a lower one flags reflections.
A practical question
Raise the detector threshold and false alarms on reflections disappear, but some small signs disappear too. Lower it and you find more signs, along with more wrong boxes. Precision and recall summarize this trade-off under a defined match rule.
The useful idea
Precision asks what fraction of reported detections are correct. Recall asks what fraction of annotated targets were found. We need an overlap threshold to decide whether a predicted box matches a real one, and we must handle duplicates consistently.
Work through the example
Sweep the confidence threshold on a held-out set and inspect the precision-recall curve. Open examples near the operating point: a false sign in a window and a missed small sign. A single average hides which failure affects the use case. Small signs far down the road often disappear when the confidence threshold is raised. At a lower threshold, reflections can become false signs. Sweep several settings on held-out street scenes and inspect the specific errors near the chosen point. The operator may prefer a few reviewable false alarms over missing an important sign, but that is a task decision. Report precision and recall together with the box-matching rule and conditions. Use this case to test your understanding: Measure precision and recall for parking signs on held-out street scenes. Here is the limitation to watch for: One high precision value is incomplete without its recall and test conditions. The answer follows from that evidence: A precision-recall curve shows the trade-off as the decision threshold changes. A missed small sign may matter more to a driver than several easy close signs. Review the false alarms and missed signs at the operating threshold with the person who uses the result. The appropriate trade-off comes from the task, not the curve alone.
Limits and next step
Choose the operating point for the actual review workflow. Next, we'll focus on why small and partly hidden objects are particularly difficult.
Worked example
Measure precision and recall for parking signs on held-out street scenes.
Quick check: test your understanding
Try it yourself: Consider this case: Measure precision and recall for parking signs on held-out street scenes. Explain how you would check the result. In your answer, address this limitation: One high precision value is incomplete without its recall and test conditions.
Check the reasoning
A precision-recall curve shows the trade-off as the decision threshold changes. One high precision value is incomplete without its recall and test conditions.
What is the main idea behind read precision-recall for a detector?
A precision-recall curve shows the trade-off as the decision threshold changes.
How is it used in the example?
Measure precision and recall for parking signs on held-out street scenes.
What limitation should be checked?
One high precision value is incomplete without its recall and test conditions.
Further reading
Programming Computer Vision with Python — Jan Erik Solem.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 55 transcript308 words · Narration ready for the upcoming videoOpen transcript
Raise the detector threshold and false alarms on reflections disappear, but some small signs disappear too. Lower it and you find more signs, along with more wrong boxes. Precision and recall summarize this trade-off under a defined match rule.
Precision asks what fraction of reported detections are correct. Recall asks what fraction of annotated targets were found. We need an overlap threshold to decide whether a predicted box matches a real one, and we must handle duplicates consistently.
Sweep the confidence threshold on a held-out set and inspect the precision-recall curve. Open examples near the operating point: a false sign in a window and a missed small sign. A single average hides which failure affects the use case. Small signs far down the road often disappear when the confidence threshold is raised. At a lower threshold, reflections can become false signs. Sweep several settings on held-out street scenes and inspect the specific errors near the chosen point. The operator may prefer a few reviewable false alarms over missing an important sign, but that is a task decision. Report precision and recall together with the box-matching rule and conditions. Use this case to test your understanding: Measure precision and recall for parking signs on held-out street scenes. Here is the limitation to watch for: One high precision value is incomplete without its recall and test conditions. The answer follows from that evidence: A precision-recall curve shows the trade-off as the decision threshold changes. A missed small sign may matter more to a driver than several easy close signs. Review the false alarms and missed signs at the operating threshold with the person who uses the result. The appropriate trade-off comes from the task, not the curve alone.
Choose the operating point for the actual review workflow. Next, we'll focus on why small and partly hidden objects are particularly difficult.