The four distributions inference runs on
Today let's understand the four distributions that nearly every statistical test leans on: the normal, the t, the chi-squared and the F.
Renu runs a small coffee roastery. One of her machines, Machine A, fills bags labelled 200 grams. It has run long enough that she trusts its long-run bag-to-bag standard deviation: 5 grams. One morning she pulls 16 bags off the line and weighs each one.
Here is what those 16 bags weighed, against the 200 gram label.
Some bags ran over the label, some under. None of them read exactly 200.
That is the shape every one of these four distributions deals with. Each one is not a shape to memorise, but the distribution OF a quantity someone actually computes from data like Renu's: a mean, a mean measured against its own estimated uncertainty, a sum of squared deviations, or a ratio of two of those. This lesson builds all four, in that order, straight out of Renu's own 16 bags.
The normal: the distribution of the sample mean
Start with the plainest of the four questions: does the average of these 16 bags look like what a 200 gram, 5 gram standard deviation machine should produce?
Average the 16 readings and you get 200.675 grams. That number alone cannot tell you much, because any 16 bags from a machine centred at 200 would almost never average to exactly 200. The question is whether 200.675 is an ordinary amount to be off by, or a surprising one.
To answer that, you need to know how much a mean of 16 bags normally wobbles. If Machine A's long-run standard deviation really is 5 grams, the mean of 16 bags has its own, smaller spread, called the standard error: 5 divided by the square root of 16, which is 1.25 grams. A single bag can easily land 5 grams off target, but a mean of 16 of them bunches up much closer to 200, because the high and low bags tend to cancel out.
With that standard error in hand, you can turn 200.675 into an ordinary distance: how many standard errors away from 200 it sits. That distance is called a z statistic, and it is just (the mean minus the target) divided by the standard error.
z works out to 0.54. Renu's mean sits just over half a standard error above 200, which is nowhere near unusual.
The histogram backs that up directly. It is 2,000 simulated 16-bag means, each one drawn the same way Renu's 16 bags were, and they pile up in the familiar bell shape the normal distribution describes: centred almost exactly on 200, with a spread close to the 1.25 you just computed by formula. The dashed line marks Renu's own mean, sitting comfortably inside that pile, not out in a thin tail.
So the normal distribution, here, is the distribution of the sample mean, once you already know sigma. The catch is in that "once you already know sigma". Renu does not actually know Machine A's true sigma. She only has one guess at it: the standard deviation of these same 16 bags. What changes when sigma has to come from the sample itself is next.
The t distribution: estimating sigma from the sample
In the real world, Renu does not get handed a trustworthy, long-run sigma for free. All she has is this one batch of 16 bags, so the only standard deviation available is the one those 16 bags happen to show.
The sample's own standard deviation, s, comes out to 6.114, a little higher than Machine A's assumed long-run 5. Divide that by the square root of 16 and the estimated standard error is 1.528, wider than the known 1.25 from the last step. Run the same distance calculation as before, mean minus target over standard error, and you get 0.442 instead of 0.54.
That ratio, a mean measured against its own estimated standard error, is called a t statistic, and it follows the t distribution. A t distribution always carries a number attached to it called its degrees of freedom, written df, which here is 16 minus 1, or 15. The minus one is the cost of having used the sample itself to estimate s; one of the 16 numbers' worth of information went into pinning down the mean you are now measuring against.
Why build a whole separate distribution for this, instead of just reusing the normal? Because s itself is only an estimate, and estimates vary from sample to sample. A different 16 bags from the very same machine would hand back a slightly different s, so the ratio built from an estimated standard error is noisier, and has fatter tails, than one built from a standard error you actually know.
The widget shows exactly that. With df set to 15, Renu's own degrees of freedom, the solid t curve sits a little lower in the middle and a little fatter in the tails than the dashed standard normal drawn behind it. Read the cutoff it reports for a 95% interval: wider than the normal's 1.960. That gap is not a correction someone invented. It is what honesty costs when sigma is estimated from the same small sample that produced the mean.
Drag the df slider up. As df climbs toward 30 and beyond, the solid t curve settles down onto the dashed normal, and the cutoff it reports drifts back toward 1.960. A bigger sample makes an estimated sigma behave more and more like a known one, which is exactly why the gap between t and normal only matters when your sample is small.
Quick check: why is the t cutoff wider than the normal's
The chi-squared distribution: a sum of squared normals
Now ask a different question about those same 16 bags: is their spread, their variance, itself an ordinary amount, or an unusually large or small one for a machine with sigma = 5?
To answer that you need to know how a sample variance behaves, and the cleanest way to see that is to build it from scratch. Take 15 independent standard normal values, square each one, and add the squares up. Do that over and over, and the distribution those sums settle into is called the chi-squared distribution, with 15 degrees of freedom here because 15 squared values went into each sum.
The 5th, 50th and 95th percentiles of 20,000 simulated sums, 7.29, 14.36 and 24.97, land almost exactly on qchisq()'s theoretical answer: 7.26, 14.34 and 25.00. That confirms it. qchisq() is not a separate formula someone handed down. It is the exact same distribution you just built by hand, computed without having to simulate it.
Now connect that to Renu's 16 bags. Her sample variance, scaled by (16 minus 1) and divided by sigma squared (5 squared, or 25), turns out to be exactly this kind of sum of 15 squared standard normals, as long as Machine A's sigma really is 5. Run that scaling and you get 22.426. Check that against the 95% band for a chi-squared with 15 degrees of freedom, 6.262 to 27.488 by qchisq(), and 22.426 sits comfortably inside it. Nothing about Machine A's spread today looks unusual.
The chi-squared shape, and where Renu's variance falls
That is 500 draws from the exact chi-squared(15) you built by hand two steps back, the same sum of 15 squared standard normals, just run 500 times instead of 20,000. The histogram leans right: it floors out at zero, since you cannot sum squares to a negative number, and it thins out slowly on the right instead of mirroring that thinness on the left. Its mean sits near 15, its own degrees of freedom, and its spread (the square root of 2 times 15) is close to 5.48.
Switch to the boxplot and you see the same shape from a different angle: the box itself sits left of centre in the range the whisker covers, with the longer whisker reaching up, not down. That asymmetry is the right skew again, just drawn as a box instead of bars.
Renu's own statistic, 22.426, would land a little past the middle of this pile, maybe a third of the way toward the right edge, nowhere near the thin stretch past 30 where a genuinely unusual variance would show up. That matches exactly what the 95% band already told you: an ordinary result, not a surprising one.
The F distribution: a ratio of two variance estimates
Renu has a second, newer machine, Machine B, and she wants to know if it is more variable than Machine A. Not whether its mean is off, its spread.
She samples 10 bags off Machine B. That gives her a second sample variance to set against Machine A's.
Machine A's sample variance is 37.377, and Machine B's is 63.492, nearly double. Divide one by the other and you get a ratio: 1.699. That ratio, one variance estimate over another, is called an F statistic, and it follows the F distribution.
You already have the machinery for this, from the chi-squared step. A chi-squared sum divided by its own degrees of freedom is called a mean square, and the F distribution is the ratio of two independent mean squares. Here it carries two degrees-of-freedom numbers instead of one, because two separate samples went into it: 9 from Machine B's 10 bags, and 15 from Machine A's 16.
If the two machines are truly equally variable, this ratio should hover near 1, drifting up or down only by sampling noise. The 95% cutoff for an F with 9 and 15 degrees of freedom is 2.588, by qf(). Renu's ratio, 1.699, stays under that cutoff.
So even though Machine B's raw variance looks almost double Machine A's, the gap between them is not yet big enough to call it distinguishable from chance, with only 10 bags sampled from Machine B. More bags from Machine B would pin that ratio down with less noise, and could still tip it either way.
Closing quiz: which distribution answers which question
A third machine, Machine C, comes along, and Renu wants to know whether it is more variable than Machine A. Which distribution answers that question directly?
Your turn: build and read a chi-squared and an F statistic
Machine C turns out to have been sampled at n = 12 bags, with a sample variance of 50. Fill in both numbers below, then run the block to build its chi-squared statistic against Machine A's known sigma, and its F ratio against Machine A's own sample variance.
Show answer
# Machine C's chi-squared statistic and F ratio, with its real numbers filled in
n_C <- 12
var_C <- 50
chisq_C <- (n_C - 1) * var_C / sigma_known^2
round(chisq_C, 3)
#> [1] 22
round(qchisq(c(0.025, 0.975), df = n_C - 1), 3)
#> [1] 3.816 21.920
F_C <- var_C / var(bagsA)
round(F_C, 3)
#> [1] 1.338
round(qf(0.95, df1 = n_C - 1, df2 = length(bagsA) - 1), 3)
#> [1] 2.507Notice the two answers do not agree. Against Machine A's known sigma of 5, Machine C's chi-squared statistic of 22 just clears the upper cutoff of 21.920, so that comparison flags it as unusual. But the F ratio, Machine C's variance of 50 over Machine A's observed 37.377, comes to only 1.338, well under its own cutoff of 2.507.
That is not a contradiction. The two statistics are answering two different questions: one asks whether 50 is unusual against a known sigma of 5, the other asks whether 50 is unusual next to Machine A's own observed variance, which has sampling noise of its own. Picking the right distribution means being clear about which of those two questions you are actually asking.
References
A few places to go deeper on where these four distributions come from and how they connect:
- Casella, G. and Berger, R., Statistical Inference (2nd ed., Chapman and Hall/CRC), Chapter 5, Properties of a Random Sample, including the derivations of the t, chi-squared and F distributions from the normal
- Wasserman, L., All of Statistics (Springer), Chapter 5, The Normal and Related Distributions
- NIST/SEMATECH e-Handbook of Statistical Methods, section 1.3.6, Gallery of Distributions - the Gaussian, t, chi-square and F entries, free to read.
Four distributions, one shared machinery
Look back at everything Renu's 16 bags produced today. A z of 0.54 measured her mean against a known sigma. A t of 0.442 measured the same mean against an estimated one, on 15 degrees of freedom. A chi-squared of 22.426 measured whether her sample variance itself was ordinary. An F of 1.699 measured her variance against Machine B's.
None of the last three needed a new assumption on top of the first. Every one of them is standard normal values, assembled differently: squared and summed for chi-squared, two of those sums ratioed for F, a mean divided by an estimated spread for t. Once you see that, the four distributions stop being four separate things to memorise, and become four honest answers to four different questions about the same kind of data.
The next lesson in this course turns to counting outcomes instead of measuring them: the binomial and Poisson distributions.