Review Ambiguous Scene Predictions
A scene label is uncertain because a sign is partly hidden.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
A scene label is uncertain because a sign is partly hidden.
A practical question
The system says a parking space is occupied, but the image shows a deep shadow and perhaps a bicycle. What should an operator see? The image, the marked space, the predicted state, and the evidence that led to uncertainty.
The useful idea
Set a review rule for ambiguous or consequential cases. A score alone may be poorly calibrated under a new camera or at night. Show the raw crop and recent frames when available, and let a person correct the state with a recorded reason. That correction can later inform evaluation, but it should not silently rewrite a test set.
Work through the example
Try cases with glare, partial vehicles, and empty spaces under strong shadow. Can a reviewer decide from the evidence, or is a new capture needed? Track how often review changes the decision. A human reviewer should see the uncertain camera frame, the proposed label, and why the system asked for help. If correcting an occupancy count takes more work than redoing it, the review path is poorly designed. Save the correction with enough context to learn from it later, but do not assume every uncertain example is a model failure. Some scenes simply hide the evidence needed for a reliable answer. Now check the idea on a concrete example: Send ambiguous cases to a person with a clear correction path. A possible trap is worth naming: A system must not present a guess as a verified observation. This supports a careful conclusion: Review workflows should expose the image region, confidence, and competing interpretations. Give the reviewer the uncertain space image and the system's proposed answer. If a shadow hides the car, allow an unknown state. Record corrections so future evaluation can distinguish an avoidable model error from missing visual evidence.
Limits and next step
Next, we'll write annotation instructions so different people label these situations consistently.
Worked example
Send ambiguous cases to a person with a clear correction path.
Quick check: test your understanding
Try it yourself: Consider this case: Send ambiguous cases to a person with a clear correction path. Explain how you would check the result. In your answer, address this limitation: A system must not present a guess as a verified observation.
Check the reasoning
Review workflows should expose the image region, confidence, and competing interpretations. A system must not present a guess as a verified observation.
What is the main idea behind review ambiguous scene predictions?
Review workflows should expose the image region, confidence, and competing interpretations.
How is it used in the example?
Send ambiguous cases to a person with a clear correction path.
What limitation should be checked?
A system must not present a guess as a verified observation.
Further reading
Programming Computer Vision with Python — Jan Erik Solem.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 68 transcript299 words · Narration ready for the upcoming videoOpen transcript
The system says a parking space is occupied, but the image shows a deep shadow and perhaps a bicycle. What should an operator see? The image, the marked space, the predicted state, and the evidence that led to uncertainty.
Set a review rule for ambiguous or consequential cases. A score alone may be poorly calibrated under a new camera or at night. Show the raw crop and recent frames when available, and let a person correct the state with a recorded reason. That correction can later inform evaluation, but it should not silently rewrite a test set.
Try cases with glare, partial vehicles, and empty spaces under strong shadow. Can a reviewer decide from the evidence, or is a new capture needed? Track how often review changes the decision. A human reviewer should see the uncertain camera frame, the proposed label, and why the system asked for help. If correcting an occupancy count takes more work than redoing it, the review path is poorly designed. Save the correction with enough context to learn from it later, but do not assume every uncertain example is a model failure. Some scenes simply hide the evidence needed for a reliable answer. Now check the idea on a concrete example: Send ambiguous cases to a person with a clear correction path. A possible trap is worth naming: A system must not present a guess as a verified observation. This supports a careful conclusion: Review workflows should expose the image region, confidence, and competing interpretations. Give the reviewer the uncertain space image and the system's proposed answer. If a shadow hides the car, allow an unknown state. Record corrections so future evaluation can distinguish an avoidable model error from missing visual evidence.
Next, we'll write annotation instructions so different people label these situations consistently.