The sampling distribution of a statistic

Today let's find out whether the mean is the only statistic that settles into a predictable spread, or whether a share, a median and a variance do too.

Kiran runs a small online bookstore. Orders come in all day, some for a single paperback, some for a stack of books at once, so the amounts land all over the place. Here is one day of 30 orders, in dollars, in the order they arrived.

Most of those orders sit well under the dashed line, Kiran's own $35 average order value, but a few tower over it, one even past $150. That is a right skewed shape, the kind with a long thin tail stretching off to the right.

The mean of a sample like this settles into a predictable, bell shaped spread as you average more and more days like it. Whether a share of big orders, a median order and a variance of orders, each built from a chart just like the one above, settle just as predictably is the question, checked one statistic at a time.

The standard error is the spread of the statistic, not of the data

Kiran's bookstore models its daily order values as exponential with a rate of 1/35. That single rate pins down two numbers at once: the population's mean order value, $35, and for this particular distribution, its standard deviation too, also $35.

Draw the spread of Kiran's 30 orders, then draw the spread of their mean once you repeat the sampling many times, and set the two side by side.

RInteractive R
# Compare the spread of one day's 30 orders with the spread of the mean's sampling distribution set.seed(1) orders_day <- rexp(30, rate = 1/35) round(sd(orders_day), 2) #> [1] 37.37 set.seed(2) means30 <- replicate(2000, mean(rexp(30, rate = 1/35))) round(sd(means30), 2) #> [1] 6.42 round(35 / sqrt(30), 2) #> [1] 6.39

  

orders_day holds that one day's 30 orders, and sd() on them comes out to 37.37, close to the population's own standard deviation of $35. That is the data's spread: how far a single order tends to sit from the average order.

means30 is a different thing entirely. replicate(2000, mean(rexp(30, rate = 1/35))) draws a fresh 30 orders, averages them, and repeats that 2000 times, so it holds 2000 separate daily averages, one per simulated day. Its spread, 6.42, is nowhere near 37.37, because an average of 30 orders moves around far less than a single order does.

This spread of the mean itself, not of the raw data behind it, has a name: the standard error. For the mean, it follows a simple formula, the population's standard deviation over the square root of the sample size. 35 divided by the square root of 30 comes out to 6.39, matching the simulated 6.42 closely.

Key Insight
A statistic's standard error is the standard deviation of its OWN sampling distribution, the spread you would see if you could compute that statistic over and over on fresh samples. It is not the spread of the data that went into it. Kiran's orders spread by 37, their mean spreads by only 6.4.