Confidence intervals by bootstrap
Today let's find out how sure you can really be about a typical number, when the usual formula for its uncertainty does not even exist.
Mo runs a small coffee cart. One Saturday morning she timed how long each of 16 customers waited in line, in minutes: 4, 5, 5, 6, 6, 7, 7, 7, 8, 8, 9, 10, 11, 13, 19, 27. Two of those waits, 19 and 27 minutes, happened because the register jammed partway through the morning.
The typical wait, the median, works out to 7.5 minutes. But that number came from just 16 customers on just one Saturday. Mo wants to know how far off 7.5 could really be.
Drag the marker above and watch the two shaded slices shrink or grow as it moves. That shrinking and growing, how much of a spread gets cut off at either end once you pick a point, is exactly the idea this lesson turns into a real interval for Mo's 7.5 minutes: cut a slice off each end and see what is left in the middle.
A statistic with no formula for its spread
For a mean, there is a direct formula for how much it would wobble from one sample to the next: the standard deviation of the data, divided by the square root of the sample size. That is the standard error, and once you know it, you already know roughly how far the mean could be off.
Mo's median does not come with that same shortcut. Compute all three numbers on her 16 wait times and see for yourself.
The mean, 9.5 minutes, sits well above the median, 7.5, because the two long waits from the register jam pull the mean up without moving the median at all. sd(wait_times) comes out to 5.94, the spread of the 16 wait times themselves. Divide that by the square root of 16 and you get 1.49 minutes: the mean's standard error, how much the mean of a fresh batch of 16 customers would typically wobble around.
Now try asking the same question about the median. Is there a line of algebra that turns Mo's 16 numbers into the median's standard error, the way sd(wait_times) / sqrt(16) just did for the mean?
There is not, not a clean one. The median only looks at whichever two numbers happen to sit in the middle once the data are sorted, and no simple formula describes how much that middle pair would move around from one batch of 16 customers to the next. Every number in the sample pulls on the mean, so a tidy formula falls out of that. The median only cares about ranking, and ranking does not reduce to a formula half as easily.
So if there is no formula, how does anyone put a number on how uncertain the median really is? That is what the bootstrap is for.