Categorical Cross-Entropy
Learn how a model chooses one class from several options.
Categorical Cross-Entropy explained visually
Use an automated fruit sorter to understand class probabilities and multi-class loss.
What you will learn
Connect classification losses with the complete training loop, batches, epochs, and convergence.
Several possible classes
A fruit sorter may choose apple, banana, or orange. The output layer assigns a probability to every class, and the probabilities add up to one.
One correct target
A one-hot target marks the correct class. Categorical cross-entropy examines the probability assigned to that class.
Mistakes guide training
High probability on the correct class creates small loss. Low probability creates large loss and a stronger correction.
From examples to reliability
Averaging loss across varied fruit examples helps the model learn color, shape, and texture instead of relying on one clue.
Quick check: test your understanding
When is categorical cross-entropy useful?
When one example must be assigned to one of several classes.
What should class probabilities total?
One.
What produces a small loss?
High probability on the correct class.
Accessible referenceLesson 13 transcript319 words · Matches the published narration and captionsOpen transcript
Binary cross-entropy helps a model choose between two classes. Many real problems contain more choices. Imagine an automated fruit-sorting machine. Apples travel to one container, bananas to another, and oranges to a third. A camera captures each fruit. The neural network then produces three probabilities.
For an apple, it might predict seventy percent apple, twenty percent orange, and ten percent banana. The three probabilities add up to one. The correct label uses one-hot encoding. An apple becomes one, zero, zero. A banana becomes zero, one, zero. An orange becomes zero, zero, one.
Categorical cross-entropy focuses on the probability assigned to the correct class. If the fruit is an apple and the model assigns a high probability to apple, the loss remains small. If the model gives the apple only a small probability, the loss becomes large, even when another class receives a confident prediction.
Watch three fruits pass through the system. The first fruit produces a loss of approximately zero point two two three. The second is more uncertain, producing a loss of approximately zero point five one zero. The third produces a loss of approximately zero point three five seven.
We add those losses and divide by three. The mean categorical cross-entropy is approximately zero point three six three. This single number summarizes how well the model handled that group of examples. During training, the network repeatedly adjusts its weights to reduce this value. As the probability of the correct class increases, the loss normally decreases.
Categorical cross-entropy supports many practical systems. A wildlife camera can distinguish different animal species. A recycling system can separate paper, plastic, glass, and metal. A medical model can compare several possible findings. In every case, the model distributes probability across the available classes. The loss checks how much probability reached the correct answer. Cross-entropy trains probabilities. Another classification loss focuses on distance from a decision boundary. That method is called hinge loss.