L1 Regularization
Encourage a simpler neural network by shrinking unnecessary weights to zero.
L1 Regularization explained visually
Pack for a long hike to understand weight penalties, sparsity, and feature selection.
What you will learn
Recognize overfitting and improve generalization with dropout, regularization, and augmentation.
Add a weight penalty
L1 adds the absolute values of the weights to the original task objective.
Create sparsity
Weak or unnecessary connections can shrink exactly to zero and stop influencing predictions.
Control with lambda
Lambda determines the strength of regularization and should be selected using validation results.
Avoid too much pressure
Excessive regularization can erase useful information and cause underfitting.
Quick check: test your understanding
What values does L1 penalize?
The absolute values of model weights.
What is sparsity?
A condition where many available weights are zero.
What controls penalty strength?
Lambda.
Accessible referenceLesson 20 transcript237 words · Matches the published narration and captionsOpen transcript
Imagine packing equipment for a long hike. Every item adds weight, so you keep the tools that truly help and remove items that contribute little.
L1 regularization gives a neural network a similar pressure. Training still minimizes the original task loss, but it also adds a penalty based on the absolute values of the weights.
Large collections of unnecessary weights increase that penalty. During optimization, some weak or unhelpful weights can shrink exactly to zero. A zero weight means that connection no longer influences the prediction.
This produces sparsity: many possible connections exist, but only a smaller group remains active. In models where input features have clear meanings, that behavior can also support feature selection.
Consider a house-price model with useful inputs such as location and floor area, plus several noisy measurements. L1 regularization may reduce some noisy feature weights to zero while preserving the stronger signals.
The symbol lambda controls the strength of the penalty. A small lambda changes the original task only slightly. A very large lambda can erase useful information and cause underfitting. We choose it by comparing performance on validation data.
L1 does not guarantee that every zero weight identifies a universally useless feature, especially when several features carry similar information. Still, it provides a practical way to encourage a simpler model.
In the next lesson, we will compare this with L2 regularization, which usually keeps connections but reduces their magnitude more smoothly.