Binary Cross-Entropy
See how AI learns reliable yes-or-no decisions.
Binary Cross-Entropy explained visually
Follow a smart porch camera as probability, confidence, and mistakes become one useful training loss.
What you will learn
Connect classification losses with the complete training loop, batches, epochs, and convergence.
A two-choice problem
Package present or package absent is a binary classification task. The model returns a probability between zero and one rather than only a hard answer.
Confidence changes the penalty
Binary cross-entropy gives a small penalty to confident correct predictions and a large penalty to confident wrong predictions. This discourages dangerous overconfidence.
Learning from every example
The loss is calculated for each image and averaged across a batch. Training then changes the network weights to reduce future mistakes.
A practical result
False alarms and missed deliveries both become feedback. With varied examples, the camera's probabilities become more dependable.
Quick check: test your understanding
What kind of task has two possible labels?
Binary classification.
What receives the largest penalty?
A confident prediction that is wrong.
What does a lower loss indicate?
The prediction better matches the correct label.
Accessible referenceLesson 12 transcript357 words · Matches the published narration and captionsOpen transcript
In the previous lesson, we learned how Huber loss balances sensitivity and stability for regression. But classification asks a different kind of question. Instead of predicting a number, a classification model chooses between categories. Imagine a smart camera watching a front porch. For every image, it answers one question: is there a package at the door?
The correct answer has two possible values. One means a package is present. Zero means it is not. But the neural network does not begin with a simple yes or no. It produces a probability. For one image, it might say there is a ninety percent chance of a package. For another, it might say twenty percent.
Binary cross-entropy compares that probability with the correct answer. When the correct label is one, the loss becomes smaller as the prediction approaches one. When the correct label is zero, the loss becomes smaller as the prediction approaches zero. This means a prediction of zero point nine for a real package receives a small penalty. A prediction of zero point one for that same package receives a much larger penalty.
The logarithm inside the equation creates an important effect. Confident and correct predictions receive very little loss. Confident but incorrect predictions receive a severe penalty. Consider five porch images. Their correct labels are one, zero, one, one, and zero. The model predicts zero point nine, zero point two, zero point eight, zero point seven, and zero point three.
Each prediction produces its own loss. We calculate all five values and then find their average. For this example, the mean binary cross-entropy is approximately zero point two five three. A lower value means the predicted probabilities agree more closely with the correct labels. Binary cross-entropy works for many two-class problems. A medical image may show a condition or no condition. An email may be spam or legitimate. A machine may be operating normally or showing a fault.
The model makes a probability estimate, and binary cross-entropy tells it how costly that estimate was. But what happens when the model must choose among more than two classes? That requires categorical cross-entropy, which we will explore next.