Challenge It with Difficult Images
A prototype succeeds under bright light but fails at night.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
A prototype succeeds under bright light but fails at night.
A practical question
The inspection pipeline succeeds on bright, clean parts. At night, a little glare turns into a false scratch, and a real narrow mark disappears in blur. That is the moment to broaden the test set, not celebrate the earlier accuracy.
The useful idea
Keep difficult examples separate from the images used to tune thresholds. Group the evaluation by conditions such as shadow, glare, motion blur, surface finish, and scale. Inspect both missed defects and false alarms in each group. An overall score can hide a severe problem in one shift or camera setting.
Work through the example
If you change a filter to fix glare, rerun the entire evaluation. The fix might erase weak defects elsewhere. Ask whether a capture change, such as steadier lighting, solves more of the problem than an increasingly complicated correction. Difficult lighting, blur, and unusual surface texture can reveal failures hidden by easy examples. Keep a challenge set separate from the images used to choose settings. Compare errors by condition after each revision, not only an overall average. If one fix helps dark scenes but harms fine scratches, show both results. Before deployment, decide which failures demand human review and which require collecting better evidence. Now check the idea on a concrete example: Hold difficult images out of tuning and compare errors by condition. A possible trap is worth naming: Fixing one visible failure on the test set can bias the final assessment. The answer follows from that evidence: Challenge the pipeline with shadows, glare, blur, different surfaces, and scale changes. A dark scene and a blurred scene may fail for different reasons. Compare them separately after each change so a fix for one condition does not hide damage to the other.
Limits and next step
Record the conditions where the system should request human review or a new capture. Next, we'll present the result so an independent inspector can reproduce it and understand its limits.
Worked example
Hold difficult images out of tuning and compare errors by condition.
Quick check: test your understanding
Try it yourself: Consider this case: Hold difficult images out of tuning and compare errors by condition. Explain how you would check the result. In your answer, address this limitation: Fixing one visible failure on the test set can bias the final assessment.
Check the reasoning
Challenge the pipeline with shadows, glare, blur, different surfaces, and scale changes. Fixing one visible failure on the test set can bias the final assessment.
What is the main idea behind challenge it with difficult images?
Challenge the pipeline with shadows, glare, blur, different surfaces, and scale changes.
How is it used in the example?
Hold difficult images out of tuning and compare errors by condition.
What limitation should be checked?
Fixing one visible failure on the test set can bias the final assessment.
Further reading
Digital Image Processing, 4th edition — Rafael C. Gonzalez and Richard E. Woods.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 79 transcript310 words · Narration ready for the upcoming videoOpen transcript
The inspection pipeline succeeds on bright, clean parts. At night, a little glare turns into a false scratch, and a real narrow mark disappears in blur. That is the moment to broaden the test set, not celebrate the earlier accuracy.
Keep difficult examples separate from the images used to tune thresholds. Group the evaluation by conditions such as shadow, glare, motion blur, surface finish, and scale. Inspect both missed defects and false alarms in each group. An overall score can hide a severe problem in one shift or camera setting.
If you change a filter to fix glare, rerun the entire evaluation. The fix might erase weak defects elsewhere. Ask whether a capture change, such as steadier lighting, solves more of the problem than an increasingly complicated correction. Difficult lighting, blur, and unusual surface texture can reveal failures hidden by easy examples. Keep a challenge set separate from the images used to choose settings. Compare errors by condition after each revision, not only an overall average. If one fix helps dark scenes but harms fine scratches, show both results. Before deployment, decide which failures demand human review and which require collecting better evidence. Now check the idea on a concrete example: Hold difficult images out of tuning and compare errors by condition. A possible trap is worth naming: Fixing one visible failure on the test set can bias the final assessment. The answer follows from that evidence: Challenge the pipeline with shadows, glare, blur, different surfaces, and scale changes. A dark scene and a blurred scene may fail for different reasons. Compare them separately after each change so a fix for one condition does not hide damage to the other.
Record the conditions where the system should request human review or a new capture. Next, we'll present the result so an independent inspector can reproduce it and understand its limits.