Semantic Segmentation and Pixel Labels
Use a beach survey to understand pixel labels, aligned target masks, and the difference between categories and individual objects.
Semantic Segmentation and Pixel Labels explained visually
Use a beach survey to understand pixel labels, aligned target masks, and the difference between categories and individual objects.
What you will learn
Assign pixel labels and compare fully convolutional, U-Net, and SegNet decoders.
Move from image categories to pixel labels
Imagine looking at an aerial photograph of a beach. The image contains water, sand, and rocks. Calling it a beach photograph tells us something useful, but it does not show where each region begins and ends.
Semantic segmentation provides that additional detail. It assigns a category to every pixel in an image.
In the previous lesson, we explored image classification with Alex Net. We will now move from a prediction about the whole image to predictions across its surface.
For our beach example, the categories could be water, sand, and rock. The model predicts which category belongs to each pixel.
Interpret labels and display colors
We can display those predictions using different colors. Blue might represent water, yellow might represent sand, and gray might represent rock. These colors display the labels. They do not need to match the original colors in the photograph.
The word semantic refers to the meaning of each category. If the photograph contains several separate rocks, their pixels can all receive the same rock label.
Ordinary semantic segmentation does not assign a separate identity to each individual rock.
Train with accurate aligned masks
During supervised training, the model receives images and their matching label masks. A label mask records the correct category at each location.
The image and mask must line up. If a water label is shifted onto the sand, the training example gives the model incorrect guidance about that boundary.
Context also matters. A pale pixel might belong to dry sand, sea foam, or a light colored rock. Nearby patterns help the network decide which interpretation fits.
Count predictions and identify regions
For a tiny example, imagine an image with four rows and four columns. It contains sixteen pixels. Semantic segmentation produces sixteen category labels, one for each pixel.
We can then identify both the type of region and its visible extent.
In the next lesson, we will explore how a fully convolutional network produces these predictions across an image.
Quick check: test your understanding
What does semantic segmentation predict?
A category label at each target pixel.
How many labels are needed for a 4 × 4 image?
4 × 4 = 16 labels.
Does a shared rock label distinguish every individual rock?
No. Semantic categories do not by themselves assign separate instance identities.
Accessible referenceLesson 57 transcript311 words · Matches the published narration and captionsOpen transcript
Imagine looking at an aerial photograph of a beach. The image contains water, sand, and rocks. Calling it a beach photograph tells us something useful, but it does not show where each region begins and ends. Semantic segmentation provides that additional detail. It assigns a category to every pixel in an image.
In the previous lesson, we explored image classification with Alex Net. We will now move from a prediction about the whole image to predictions across its surface. For our beach example, the categories could be water, sand, and rock. The model predicts which category belongs to each pixel.
We can display those predictions using different colors. Blue might represent water, yellow might represent sand, and gray might represent rock. These colors display the labels. They do not need to match the original colors in the photograph. The word semantic refers to the meaning of each category. If the photograph contains several separate rocks, their pixels can all receive the same rock label.
Ordinary semantic segmentation does not assign a separate identity to each individual rock. During supervised training, the model receives images and their matching label masks. A label mask records the correct category at each location. The image and mask must line up. If a water label is shifted onto the sand, the training example gives the model incorrect guidance about that boundary.
Context also matters. A pale pixel might belong to dry sand, sea foam, or a light colored rock. Nearby patterns help the network decide which interpretation fits. For a tiny example, imagine an image with four rows and four columns. It contains sixteen pixels. Semantic segmentation produces sixteen category labels, one for each pixel.
We can then identify both the type of region and its visible extent. In the next lesson, we will explore how a fully convolutional network produces these predictions across an image.