The Bias-Variance Tradeoff
In Lesson 1 we turned a fuzzy business question into a precise prediction problem. Now we meet the single idea that decides whether your model actually works on new data.
Maya, a real-estate analyst, wants to predict a home's sale price from its size. Her first model is a flat rule: "every house is worth about $380k." It is steady, and steadily wrong. Her second model is a wildly flexible curve that passes through every single house she has on record. It is perfect on her records, and useless on the next house that comes up for sale. Both failed, in opposite ways. The space between them is what this lesson is about.
By the end you will be able to:
- Name bias and variance as two separate sources of a model's error
- Explain why making a model more flexible trades one for the other, giving the famous U-shaped error curve
- Spot overfitting and underfitting from the gap between training and test error, and measure it in R
Prerequisites: you can run R and read its output, and you know what a training set and a test set are (Lesson 1).
A model can be wrong in two different ways
Think of the dartboard above. The bullseye is the truth: a home's real price. Each dart is one model's prediction. Now picture something subtle but important: we never train on "the" data, we train on one sample of homes that happened to land in our spreadsheet. Draw a different sample next month and we would fit a slightly different model, throwing a slightly different dart.
That gives two completely separate ways to miss:
- Bias is aim. If every dart lands in the same spot but that spot is off to one side, the model is consistently, systematically wrong. Maya's flat "$380k for everyone" rule has high bias: it ignores size, so it is always too high for tiny homes and too low for mansions, no matter which sample she trains on.
- Variance is spread. If the darts scatter all over the board, the model is wildly sensitive to which exact homes it saw. Maya's wiggly curve has high variance: feed it next month's listings and it draws a completely different shape.
Slide the two controls below. Move bias to shove the cluster off the bullseye; move variance to scatter the darts. Watch the readout split the total error into a bias part and a variance part.
The goal, the tight cluster on the bullseye, needs both low. The trouble is that the usual ways of lowering one tend to raise the other.