Audit Inconsistent Mask Annotations
Annotators disagree whether a tree shadow belongs in its mask.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
Annotators disagree whether a tree shadow belongs in its mask.
A practical question
One annotator includes the shadow under a tree in the tree mask; another labels only visible foliage. A model trained on both receives contradictory instructions. Before improving the network, inspect the annotation rules.
The useful idea
Write a guide covering thin branches, occlusion, shadows, and ambiguous borders. Have two people label a small common sample and compare the differences. When they disagree, decide a consistent rule or explicitly mark cases that require review. A high score against inconsistent labels can be misleading.
Work through the example
Correct a small batch and inspect whether model errors change. Keep versioned examples of the rule so future annotators can apply it. A mask is a judgment built from image evidence, not a perfect physical truth. One annotator includes a tree's shadow as part of the canopy, while another excludes it. Give them the same small set of frames and a written rule for foliage, thin branches, and occlusion. Review disagreements and version the guide before expanding the dataset. A model cannot consistently learn two conflicting meanings for the same class. Keep a few borderline examples in the guide so future labels can be checked against a visible decision rather than a vague instruction. Now check the idea on a concrete example: Compare two tree masks and resolve shadows and branches with a written rule. A possible trap is worth naming: Training on inconsistent labels can make model evaluation misleading. This supports a careful conclusion: An annotation guide defines inclusion, occlusion, and boundary conventions. Review two annotators' masks on the same shadowed tree before retraining. If the guide is ambiguous, a model trained longer only learns inconsistent targets. Save the agreed boundary example so future reviewers can apply the same rule.
Limits and next step
Next, we'll use a known planar marker to estimate geometry for graphics placed in a camera view.
Worked example
Compare two tree masks and resolve shadows and branches with a written rule.
Quick check: test your understanding
Try it yourself: Consider this case: Compare two tree masks and resolve shadows and branches with a written rule. Explain how you would check the result. In your answer, address this limitation: Training on inconsistent labels can make model evaluation misleading.
Check the reasoning
An annotation guide defines inclusion, occlusion, and boundary conventions. Training on inconsistent labels can make model evaluation misleading.
What is the main idea behind audit inconsistent mask annotations?
An annotation guide defines inclusion, occlusion, and boundary conventions.
How is it used in the example?
Compare two tree masks and resolve shadows and branches with a written rule.
What limitation should be checked?
Training on inconsistent labels can make model evaluation misleading.
Further reading
Programming Computer Vision with Python — Jan Erik Solem.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 60 transcript297 words · Narration ready for the upcoming videoOpen transcript
One annotator includes the shadow under a tree in the tree mask; another labels only visible foliage. A model trained on both receives contradictory instructions. Before improving the network, inspect the annotation rules.
Write a guide covering thin branches, occlusion, shadows, and ambiguous borders. Have two people label a small common sample and compare the differences. When they disagree, decide a consistent rule or explicitly mark cases that require review. A high score against inconsistent labels can be misleading.
Correct a small batch and inspect whether model errors change. Keep versioned examples of the rule so future annotators can apply it. A mask is a judgment built from image evidence, not a perfect physical truth. One annotator includes a tree's shadow as part of the canopy, while another excludes it. Give them the same small set of frames and a written rule for foliage, thin branches, and occlusion. Review disagreements and version the guide before expanding the dataset. A model cannot consistently learn two conflicting meanings for the same class. Keep a few borderline examples in the guide so future labels can be checked against a visible decision rather than a vague instruction. Now check the idea on a concrete example: Compare two tree masks and resolve shadows and branches with a written rule. A possible trap is worth naming: Training on inconsistent labels can make model evaluation misleading. This supports a careful conclusion: An annotation guide defines inclusion, occlusion, and boundary conventions. Review two annotators' masks on the same shadowed tree before retraining. If the guide is ambiguous, a model trained longer only learns inconsistent targets. Save the agreed boundary example so future reviewers can apply the same rule.
Next, we'll use a known planar marker to estimate geometry for graphics placed in a camera view.