Write an Annotation Guide
Annotators disagree whether a vehicle entering a space counts as parked.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
Annotators disagree whether a vehicle entering a space counts as parked.
A practical question
One annotator marks a motorcycle as an occupied parking space; another marks it empty because the guide only mentions cars. A model cannot learn one stable policy from contradictory examples.
The useful idea
Write the annotation guide around observable decisions: what counts as occupancy, how to handle occlusion, partly visible spaces, glare, and frames with no reliable view. Include images of borderline cases and a route for unresolved questions.
Work through the example
Give several annotators the same small set and compare their labels. Discuss disagreements before expanding the dataset. Version the guide with the labels; changing the policy midway without tracking it can make old and new examples inconsistent. Write down whether a motorcycle occupies a space, how to treat a partly visible car, and what to do when a vehicle crosses a space without parking. Add actual borderline frames beside each rule. Have several annotators label the same sample and discuss differences before scaling up. If the policy changes, version it and review labels made under the older rule. Agreement is only meaningful when everyone is answering the same question. Use this case to test your understanding: Document how to mark partially visible cars and vehicles in the driving lane. Here is the limitation to watch for: Ambiguous labels reduce both training quality and honest evaluation. The answer follows from that evidence: An annotation guide defines categories, boundaries, edge cases, and examples before labeling starts. Have two annotators review a motorcycle and a car crossing a space. If they disagree, rewrite the occupancy rule with those actual frames attached. More labels without a shared rule would only make the training set larger and less coherent. Save the disputed examples in the guide for the next annotator.
Limits and next step
Once labels have a clear meaning, we'll split the data so evaluation tests a genuinely different camera or time rather than near-duplicate frames.
Worked example
Document how to mark partially visible cars and vehicles in the driving lane.
Quick check: test your understanding
Try it yourself: Consider this case: Document how to mark partially visible cars and vehicles in the driving lane. Explain how you would check the result. In your answer, address this limitation: Ambiguous labels reduce both training quality and honest evaluation.
Check the reasoning
An annotation guide defines categories, boundaries, edge cases, and examples before labeling starts. Ambiguous labels reduce both training quality and honest evaluation.
What is the main idea behind write an annotation guide?
An annotation guide defines categories, boundaries, edge cases, and examples before labeling starts.
How is it used in the example?
Document how to mark partially visible cars and vehicles in the driving lane.
What limitation should be checked?
Ambiguous labels reduce both training quality and honest evaluation.
Further reading
Programming Computer Vision with Python — Jan Erik Solem.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 69 transcript304 words · Narration ready for the upcoming videoOpen transcript
One annotator marks a motorcycle as an occupied parking space; another marks it empty because the guide only mentions cars. A model cannot learn one stable policy from contradictory examples.
Write the annotation guide around observable decisions: what counts as occupancy, how to handle occlusion, partly visible spaces, glare, and frames with no reliable view. Include images of borderline cases and a route for unresolved questions.
Give several annotators the same small set and compare their labels. Discuss disagreements before expanding the dataset. Version the guide with the labels; changing the policy midway without tracking it can make old and new examples inconsistent. Write down whether a motorcycle occupies a space, how to treat a partly visible car, and what to do when a vehicle crosses a space without parking. Add actual borderline frames beside each rule. Have several annotators label the same sample and discuss differences before scaling up. If the policy changes, version it and review labels made under the older rule. Agreement is only meaningful when everyone is answering the same question. Use this case to test your understanding: Document how to mark partially visible cars and vehicles in the driving lane. Here is the limitation to watch for: Ambiguous labels reduce both training quality and honest evaluation. The answer follows from that evidence: An annotation guide defines categories, boundaries, edge cases, and examples before labeling starts. Have two annotators review a motorcycle and a car crossing a space. If they disagree, rewrite the occupancy rule with those actual frames attached. More labels without a shared rule would only make the training set larger and less coherent. Save the disputed examples in the guide for the next annotator.
Once labels have a clear meaning, we'll split the data so evaluation tests a genuinely different camera or time rather than near-duplicate frames.