Test Occlusion, Night, and Empty Scenes
The first tests miss vehicles at night and flag shadows as cars.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
The first tests miss vehicles at night and flag shadows as cars.
A practical question
At night, a parked car disappears into shadow. In another frame, a tree hides half a vehicle. A third frame shows an empty lot with reflections that resemble headlights. A credible prototype must face all three.
The useful idea
Hold challenging clips out of tuning. Evaluate per-space state errors by night, occlusion, glare, camera, and truly empty conditions. Inspect whether a wrong state persists long enough to mislead a driver. Report sample counts with rates so a rare condition isn't treated as certainty.
Work through the example
When you fix one condition, rerun the other tests. A lower threshold may catch dark cars but also mark empty shadows occupied. Build a review route for genuinely uncertain views. Build a difficult set with obscured vehicles, night glare, and truly empty spaces. Hold it aside while changing the model or decision rule. After each revision, compare errors by condition and inspect examples where the result changed. A rule that improves crowded daylight counts could create false occupied spaces at night. Report that trade-off and decide when human review or a temporary unknown state is safer than a confident count. Use this case to test your understanding: Hold a difficult set aside, then compare errors by condition after each revision. Here is the limitation to watch for: Tuning repeatedly on the final test set undermines its independence. The answer follows from that evidence: Evaluate hard cases including occlusion, empty scenes, glare, and night lighting. Hold glare, occlusion, and empty scenes out of tuning. After a revision, count newly fixed and newly broken examples within each group. The operator needs to see both, particularly when a false free space sends a driver to an occupied bay. Keep those frames as permanent regression checks after deployment.
Limits and next step
Next, we'll present the results and limitations so another team can check them.
Worked example
Hold a difficult set aside, then compare errors by condition after each revision.
Quick check: test your understanding
Try it yourself: Consider this case: Hold a difficult set aside, then compare errors by condition after each revision. Explain how you would check the result. In your answer, address this limitation: Tuning repeatedly on the final test set undermines its independence.
Check the reasoning
Evaluate hard cases including occlusion, empty scenes, glare, and night lighting. Tuning repeatedly on the final test set undermines its independence.
What is the main idea behind test occlusion, night, and empty scenes?
Evaluate hard cases including occlusion, empty scenes, glare, and night lighting.
How is it used in the example?
Hold a difficult set aside, then compare errors by condition after each revision.
What limitation should be checked?
Tuning repeatedly on the final test set undermines its independence.
Further reading
Programming Computer Vision with Python — Jan Erik Solem.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 79 transcript297 words · Narration ready for the upcoming videoOpen transcript
At night, a parked car disappears into shadow. In another frame, a tree hides half a vehicle. A third frame shows an empty lot with reflections that resemble headlights. A credible prototype must face all three.
Hold challenging clips out of tuning. Evaluate per-space state errors by night, occlusion, glare, camera, and truly empty conditions. Inspect whether a wrong state persists long enough to mislead a driver. Report sample counts with rates so a rare condition isn't treated as certainty.
When you fix one condition, rerun the other tests. A lower threshold may catch dark cars but also mark empty shadows occupied. Build a review route for genuinely uncertain views. Build a difficult set with obscured vehicles, night glare, and truly empty spaces. Hold it aside while changing the model or decision rule. After each revision, compare errors by condition and inspect examples where the result changed. A rule that improves crowded daylight counts could create false occupied spaces at night. Report that trade-off and decide when human review or a temporary unknown state is safer than a confident count. Use this case to test your understanding: Hold a difficult set aside, then compare errors by condition after each revision. Here is the limitation to watch for: Tuning repeatedly on the final test set undermines its independence. The answer follows from that evidence: Evaluate hard cases including occlusion, empty scenes, glare, and night lighting. Hold glare, occlusion, and empty scenes out of tuning. After a revision, count newly fixed and newly broken examples within each group. The operator needs to see both, particularly when a false free space sends a driver to an occupied bay. Keep those frames as permanent regression checks after deployment.
Next, we'll present the results and limitations so another team can check them.