Linear Regression as a Useful Baseline
A straight-line model can be a strong starting point even when the real world is complicated.
Linear Regression as a Useful Baseline explained visually
A straight-line model can be a strong starting point even when the real world is complicated. English captions are optional and start off. The complete transcript follows the written lesson.
What you will learn
A straight-line model can be a strong starting point even when the real world is complicated.
Fit a numerical relationship
A straight-line model can be a strong starting point even when the real world is complicated. Linear regression predicts a numerical target by combining features with coefficients and an intercept. Its simplicity makes it easier to inspect, reproduce, and compare with more flexible models.
Interpret coefficients carefully
With one feature, the coefficient describes the model's predicted change in the target for a one-unit feature increase. With multiple features, the interpretation holds other included features fixed. It does not automatically establish a causal effect. Correlated features, omitted variables, and the data-collection process can make a coefficient misleading if it is interpreted as a statement about what would happen after an intervention.
Inspect residuals
Least-squares fitting minimizes the sum of squared residuals, where a residual is the observed value minus the prediction. Large residuals receive substantial influence. Plot residuals against predictions and relevant features to look for curvature, changing spread, or unusual points. Evaluate on held-out data and compare with a simple constant predictor. A line that fits the training examples neatly may still extrapolate badly beyond their range.
A worked example
Suppose a fitted model predicts cost as ten dollars plus two dollars for each kilometer. At five kilometers, the prediction is twenty dollars. This is a model-based estimate within its intended context; it does not prove that every additional kilometer always causes exactly two dollars of real-world cost.
Worked example
Suppose a fitted model predicts cost as ten dollars plus two dollars for each kilometer. At five kilometers, the prediction is twenty dollars. This is a model-based estimate within its intended context; it does not prove that every additional kilometer always causes exactly two dollars of real-world cost.
Quick check: test your understanding
Try it yourself: Calculate the prediction at eight kilometers and explain one risk of applying the model at 5,000 kilometers.
Check the reasoning
The prediction is 26 dollars. At 5,000 kilometers the model is extrapolating far beyond an ordinary local-delivery range, where the relationship may change.
Does a regression coefficient by itself prove causation?
No. It describes a fitted relationship under the model and data assumptions.
How would you solve the practice case?
The prediction is 26 dollars. At 5,000 kilometers the model is extrapolating far beyond an ordinary local-delivery range, where the relationship may change.
What does the worked example show?
Suppose a fitted model predicts cost as ten dollars plus two dollars for each kilometer. At five kilometers, the prediction is twenty dollars. This is a model-based estimate within its intended context; it does not prove that every additional kilometer always causes exactly two dollars of real-world cost.
Further reading
Machine Learning — Tom M. Mitchell.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 5 transcript220 words · Video transcript · Matches the published narration and captionsOpen transcript
A straight-line model can be a strong starting point even when the real world is complicated. Linear regression predicts a numerical target by combining features with coefficients and an intercept. Its simplicity makes it easier to inspect, reproduce, and compare with more flexible models.
With one feature, the coefficient describes the model's predicted change in the target for a one-unit feature increase. With multiple features, the interpretation holds other included features fixed. It does not automatically establish a causal effect. Correlated features, omitted variables, and the data-collection process can make a coefficient misleading if it is interpreted as a statement about what would happen after an intervention.
Least-squares fitting minimizes the sum of squared residuals, where a residual is the observed value minus the prediction. Large residuals receive substantial influence. Plot residuals against predictions and relevant features to look for curvature, changing spread, or unusual points. Evaluate on held-out data and compare with a simple constant predictor. A line that fits the training examples neatly may still extrapolate badly beyond their range.
Suppose a fitted model predicts cost as ten dollars plus two dollars for each kilometer. At five kilometers, the prediction is twenty dollars. This is a model-based estimate within its intended context; it does not prove that every additional kilometer always causes exactly two dollars of real-world cost.