Semantic Labels for Every Pixel
The city needs a pixel-level map of roadside tree cover.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
The city needs a pixel-level map of roadside tree cover.
A practical question
A box around a roadside tree includes sky and pavement. If the task is to measure tree coverage, we need a label for each pixel rather than one rectangle around the whole object. Semantic segmentation assigns pixels to categories such as tree, road, and sky.
The useful idea
The result is a map of classes, not a measurement of individual tree identities. A branch boundary can be uncertain, especially under shadow or blur. Resizing its label mask requires category-preserving interpolation, and the annotation guide should explain how to label tiny gaps.
Work through the example
Overlay the predicted map on the original street image. Inspect a clear canopy and then a thin branch against bright sky. If the model calls a shadow 'tree', ask whether training labels and lighting examples cover that condition. A tree canopy occupies an irregular region that a rectangle cannot describe precisely. Overlay the predicted tree pixels on a clear canopy and a thin branch against bright sky. A shadow on pavement should not silently become tree simply because it has a similar tone. Compare the mask with a reviewed reference and state how unclear boundaries are labeled. Semantic labels describe categories per pixel; they do not say which of two neighboring trees owns a branch. Use this case to test your understanding: Overlay tree-class pixels on a canopy, a thin branch, and adjacent pavement. Here is the limitation to watch for: Pixels with a shared label need not belong to one connected object. The answer follows from that evidence: Semantic segmentation assigns class labels per pixel without distinguishing individual objects of the same class. A predicted tree mask may include its shadow on the sidewalk. Overlay the class map on the street photograph and compare with the annotation rule. The wrong shadow can distort a coverage measurement even when most canopy pixels are correct.
Limits and next step
Next, we'll separate multiple objects of the same class so two neighboring trees don't merge into one count.
Worked example
Overlay tree-class pixels on a canopy, a thin branch, and adjacent pavement.
Quick check: test your understanding
Try it yourself: Consider this case: Overlay tree-class pixels on a canopy, a thin branch, and adjacent pavement. Explain how you would check the result. In your answer, address this limitation: Pixels with a shared label need not belong to one connected object.
Check the reasoning
Semantic segmentation assigns class labels per pixel without distinguishing individual objects of the same class. Pixels with a shared label need not belong to one connected object.
What is the main idea behind semantic labels for every pixel?
Semantic segmentation assigns class labels per pixel without distinguishing individual objects of the same class.
How is it used in the example?
Overlay tree-class pixels on a canopy, a thin branch, and adjacent pavement.
What limitation should be checked?
Pixels with a shared label need not belong to one connected object.
Further reading
Programming Computer Vision with Python — Jan Erik Solem.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 57 transcript320 words · Narration ready for the upcoming videoOpen transcript
A box around a roadside tree includes sky and pavement. If the task is to measure tree coverage, we need a label for each pixel rather than one rectangle around the whole object. Semantic segmentation assigns pixels to categories such as tree, road, and sky.
The result is a map of classes, not a measurement of individual tree identities. A branch boundary can be uncertain, especially under shadow or blur. Resizing its label mask requires category-preserving interpolation, and the annotation guide should explain how to label tiny gaps.
Overlay the predicted map on the original street image. Inspect a clear canopy and then a thin branch against bright sky. If the model calls a shadow 'tree', ask whether training labels and lighting examples cover that condition. A tree canopy occupies an irregular region that a rectangle cannot describe precisely. Overlay the predicted tree pixels on a clear canopy and a thin branch against bright sky. A shadow on pavement should not silently become tree simply because it has a similar tone. Compare the mask with a reviewed reference and state how unclear boundaries are labeled. Semantic labels describe categories per pixel; they do not say which of two neighboring trees owns a branch. Use this case to test your understanding: Overlay tree-class pixels on a canopy, a thin branch, and adjacent pavement. Here is the limitation to watch for: Pixels with a shared label need not belong to one connected object. The answer follows from that evidence: Semantic segmentation assigns class labels per pixel without distinguishing individual objects of the same class. A predicted tree mask may include its shadow on the sidewalk. Overlay the class map on the street photograph and compare with the annotation rule. The wrong shadow can distort a coverage measurement even when most canopy pixels are correct.
Next, we'll separate multiple objects of the same class so two neighboring trees don't merge into one count.