Fully Convolutional Networks for Segmentation
Follow an aerial farm image through shared spatial processing, class scores, and an encoder–decoder segmentation network.
Fully Convolutional Networks for Segmentation explained visually
Follow an aerial farm image through shared spatial processing, class scores, and an encoder–decoder segmentation network.
What you will learn
Assign pixel labels and compare fully convolutional, U-Net, and SegNet decoders.
Share work across overlapping regions
Imagine a drone photographing farmland. We want a map showing crops, bare soil, and water. A single label for the whole photograph would leave out most of the information we need.
We need a prediction at every location, supported by the surrounding context.
One possible approach would examine a small image patch around each pixel separately. However, neighboring patches overlap. Processing them independently would repeat much of the same work.
A fully convolutional network shares that computation across the image. It processes spatial feature maps and produces class scores at many locations.
Count spatial scores and labels
Instead of flattening the entire image into one vector for a single classification, the network keeps a spatial arrangement of predictions.
Consider a prediction grid with sixty four rows and sixty four columns. It contains four thousand ninety six locations.
Suppose we predict three categories at every location. We need three scores per location, giving twelve thousand two hundred eighty eight scores altogether.
For a task with one category per pixel, softmax can turn the scores at each location into probabilities. Selecting the largest probability gives one label at that location.
The result is a grid of four thousand ninety six labels.
Connect encoder and decoder roles
Keeping every intermediate feature map at full image resolution can require substantial computation and memory. Many segmentation networks therefore use an encoder and a decoder.
The encoder extracts features while reducing spatial dimensions. The decoder increases spatial resolution to produce predictions aligned with the target image grid.
The phrase fully convolutional does not mean that every operation must be a convolution. The network can also contain activations, pooling, and upsampling.
Keep the prediction spatial
The key idea is that it produces spatial predictions without requiring the whole image to become one fixed classification vector.
This approach allows the model to learn both useful features and how to turn those features into a segmentation map.
In the next lesson, we will examine how a decoder makes a feature map larger through upsampling and max unpooling.
Quick check: test your understanding
How many locations are in a 64 × 64 prediction grid?
64 × 64 = 4,096.
How many scores are needed for three classes at every location?
4,096 × 3 = 12,288 scores, reduced to 4,096 labels by class selection.
Must every operation in a fully convolutional network be a convolution?
No. It may also use activations, pooling, and upsampling.
Accessible referenceLesson 58 transcript324 words · Matches the published narration and captionsOpen transcript
Imagine a drone photographing farmland. We want a map showing crops, bare soil, and water. A single label for the whole photograph would leave out most of the information we need. We need a prediction at every location, supported by the surrounding context. One possible approach would examine a small image patch around each pixel separately. However, neighboring patches overlap. Processing them independently would repeat much of the same work.
A fully convolutional network shares that computation across the image. It processes spatial feature maps and produces class scores at many locations. Instead of flattening the entire image into one vector for a single classification, the network keeps a spatial arrangement of predictions. Consider a prediction grid with sixty four rows and sixty four columns. It contains four thousand ninety six locations.
Suppose we predict three categories at every location. We need three scores per location, giving twelve thousand two hundred eighty eight scores altogether. For a task with one category per pixel, softmax can turn the scores at each location into probabilities. Selecting the largest probability gives one label at that location.
The result is a grid of four thousand ninety six labels. Keeping every intermediate feature map at full image resolution can require substantial computation and memory. Many segmentation networks therefore use an encoder and a decoder. The encoder extracts features while reducing spatial dimensions. The decoder increases spatial resolution to produce predictions aligned with the target image grid.
The phrase fully convolutional does not mean that every operation must be a convolution. The network can also contain activations, pooling, and upsampling. The key idea is that it produces spatial predictions without requiring the whole image to become one fixed classification vector. This approach allows the model to learn both useful features and how to turn those features into a segmentation map. In the next lesson, we will examine how a decoder makes a feature map larger through upsampling and max unpooling.