Fully Connected Layers and Class Predictions
Combine image features into class scores, count weights and biases, and interpret softmax predictions carefully.
Fully Connected Layers and Class Predictions explained visually
Combine image features into class scores, count weights and biases, and interpret softmax predictions carefully.
What you will learn
Follow filters, pooling, flattening, and classification through complete CNN examples.
Combine evidence with learned weights
A fruit grader may consider shape, color, and texture together. A fully connected layer combines learned numerical features rather than applying a hand-written fruit rule.
Each output unit receives every input feature. It computes a weighted sum and adds a bias. The weights and biases are learned during training.
Count the parameters
Connect 12 input features directly to 3 output units. There are 12 × 3 = 36 weights. One bias per output adds 3 parameters, giving 39 in total.
This count describes that one fully connected layer. It does not count parameters in earlier convolutional layers or any additional hidden layers.
Distinguish logits from probabilities
The three raw class scores are logits. They may be positive or negative and are not probabilities by themselves. For a single-label classification task, softmax converts the scores into nonnegative values that sum to 1.
An example prediction is banana 80%, orange 15%, and apple 5%. The highest probability selects banana. These illustrative percentages sum to 100%; they are not measured performance results.
Interpret confidence and training correctly
A high predicted probability does not guarantee that the answer is correct. Unfamiliar objects, lighting changes, and different backgrounds can cause errors.
During training, a loss compares predictions with the correct labels. Backpropagation supplies gradients for the classifier and earlier trainable layers. Architectures may include hidden fully connected layers before the final scores.
Quick check: test your understanding
How many parameters connect 12 features to 3 outputs with one bias per output?
39: 36 weights plus 3 biases.
Are logits probabilities?
No. They are unconstrained raw scores. Softmax can convert them into a normalized distribution for single-label classification.
Does an 80% prediction prove the classification is correct?
No. It expresses the model’s prediction and may be wrong or poorly calibrated.
Accessible referenceLesson 54 transcript314 words · Matches the published narration and captionsOpen transcript
Imagine sorting fruit into apples, oranges, and bananas. Shape alone may not be enough. You may also consider color and texture before making a decision. A neural network can learn to combine several kinds of evidence in a similar way. Convolutional layers extract features from an image. A fully connected layer can then combine those features to produce class scores.
In a fully connected layer, every output receives information from every input feature. Each connection has a weight. The layer multiplies inputs by their weights, adds the results, and includes a bias for each output. Suppose we have twelve input features and three output classes.
Each output needs twelve weights. Three outputs therefore need thirty six weights. Adding one bias for each output gives thirty nine learned parameters in total. The three outputs could correspond to apple, orange, and banana. These raw class scores are called logits. They are not probabilities. A logit can be positive or negative, and the scores do not have to add up to one.
For a task where each image belongs to one class, we can apply softmax to convert the scores into a probability distribution. For example, the model might assign an eighty percent probability to banana, fifteen percent to orange, and five percent to apple. Choosing the largest probability would produce a banana prediction.
However, a confident prediction can still be wrong. Unfamiliar fruit, poor lighting, or a misleading background may cause mistakes. During training, the prediction is compared with the correct label. Backpropagation calculates gradients, and the training process updates the classifier and the earlier layers. Some networks also use hidden fully connected layers before the final class scores. These layers provide additional learned transformations of the features.
We now have the main pieces needed for a complete image classifier. In the next lesson, we will connect them in a small network inspired by Le Net.