Unequal Variance in Peer Review

Unequal group variances rarely bias the difference you are estimating. What they distort is its standard error, and with it the p-value and the confidence interval. The usual fix, a Welch correction, drops the equal-variance assumption altogether rather than relying on it, so for a good number of analyses the honest answer is that the assumption was never required.

What the reviewer wrote

The two groups appear to have quite different variances, so a Student's t-test may not be appropriate here.

Homogeneity of variance has not been demonstrated. The authors should formally test this assumption before comparing the group means.

I also wondered about the spread of the outcome, which looks a good deal wider in the treatment arm than in the control in Figure 2, and whether that has any bearing on the test that was chosen.

What they actually mean

The reviewer is pointing at an assumption behind the pooled Student's t-test and the classic one-way ANOVA: that every group shares a single common variance. When the spreads differ, those tests pool them into one number, and that pooled number describes no group well.

Two different situations get worded the same way. In a comparison of groups the concern is unequal variance between the groups. In a regression it is the related idea that the residual variance changes across the fitted values, usually called heteroscedasticity; if that is your case the check and the remedy are different, and Heteroscedasticity in R covers them. This chapter takes the group-comparison reading, which is what "the variances are not equal" almost always means.

The reviewer is not claiming that a difference in spread is itself a mistake, and is not asking whether your outcome is normally distributed. Both are separate questions. What is in doubt is only whether the standard error your test reported is the right one.

Why they are asking

A pooled test borrows strength across groups by assuming they vary by the same amount. When that assumption holds, pooling makes the comparison more powerful. When it fails, the pooled standard error is the wrong size, and the p-value and confidence interval built on it are wrong with it, so a result can look more or less significant than the data actually support.

How much it matters depends on the group sizes. With roughly equal group sizes the pooled test is remarkably tolerant of unequal variance, and the p-value barely moves. With unequal sizes the errors stop cancelling; the distortion can go in either direction depending on which group carries the larger variance, and it can be big enough to change a conclusion. The situation to worry about is unequal variances together with unequal group sizes.

The mechanics of the tests are covered in Test Normality and Equal Variance in R. This chapter is about deciding whether you have a problem and what to say.

How to check it

Compare the group variances and back them with a formal test. The insect-spray counts make a clear example.

The variances span from 3.0 to 38.6, more than a tenfold range, and Bartlett's test rejects equality at p = 0.00009. Bartlett's test has one weakness worth knowing: it is sensitive to non-normality, so on skewed data it can report unequal variances when the real issue is the shape of the distribution. A test that does not lean on normality is the safer check here.

The Fligner-Killeen test uses ranks and tolerates non-normal data, and it still rejects at p = 0.013, so the unequal variance here is genuine rather than an artefact of skew. Levene's test is the version most reviewers name; it behaves like Fligner-Killeen but needs the car package, whereas fligner.test is built into base R. None of these tests has a threshold worth trusting on its own, because a small sample will miss real differences and a large one will flag differences too small to matter, so read the test next to the group variances rather than in place of them.

What to do about it

You are fine

Either the check did not reject, or your design protects you. On the tooth-growth data the two delivery methods have variances of 43.6 and 68.3, and an F-test does not reject equality.

The p-value of 0.23 gives no reason to abandon a pooled test, so you report the check and move on. There is a second, easier way to be fine that many authors miss: R's t.test applies the Welch correction by default, so if you called t.test without setting var.equal = TRUE, you never assumed equal variances at all and the objection does not touch your analysis. Check the degrees of freedom you reported. If they are fractional, the test was already Welch, and pointing that out is the reply.

It is fixable

The variances differ and the correction is a one-line change: use the Welch version, which estimates a separate variance for each group and adjusts the degrees of freedom instead of pooling. Fuel economy in mtcars splits into two transmission groups whose variances differ by about two and a half times, 14.7 against 38.0, so it shows what the switch does.

The pooled test set the degrees of freedom at 30 and the p-value at 0.000285. The Welch test cut the degrees of freedom to 18.3, widened the confidence interval from (-10.85, -3.64) to (-11.28, -3.21), and moved the p-value to 0.001374. The estimated difference in means did not move at all, because Welch corrects the standard error and leaves the estimate alone. The manual-transmission advantage holds either way, so here the correction costs nothing but a little precision. For three or more groups the same idea is oneway.test(y ~ group, var.equal = FALSE), the Welch one-way ANOVA described in Welch's ANOVA in R.

It is a real problem

Sometimes the unequal variance is not noise to correct but a sign that the model is the wrong one. When the variance grows with the mean, a Welch correction patches the test and leaves the cause untouched. The insect-spray counts show the pattern.

The groups with the smallest mean counts, C and E, have the smallest variances, and the group with the largest mean, F, has the largest, because count data carry a built-in link between their mean and their variance. A Welch test on these numbers would run, but comparing arithmetic means of counts whose spread rises with their level answers a distorted question. The honest route is a model that expects the pattern: a square-root or log transform stabilises the variance, and a Poisson or quasi-Poisson model handles the counts directly, as in Poisson regression in R. If instead the different spread is your actual result, say a treatment that makes responses more variable rather than simply larger, then report it as a finding in its own right instead of hiding it inside a test of means.

How to word your response

If you are fine

We thank the reviewer for this point. The comparison in Table 2 uses Welch's t-test, which estimates a separate variance for each group and so does not assume homogeneity of variance. For completeness we also report a test of equal variances (F-test, p = 0.23), which shows no departure worth acting on. The choice of test and this check are now stated in the Methods (page X).

If it was fixable

The reviewer is correct that the group variances differ. We have replaced the pooled t-test with Welch's t-test, which does not assume equal variances. The estimated difference in means is unchanged and the effect remains significant, with a slightly wider confidence interval (now -11.3 to -3.2). The revised test and interval appear in the Results, and the change of method is noted in the Methods (page X).

If it is a real problem

We agree that the variances differ across groups, and on inspection the variance rises with the group mean, as is expected for count data. Rather than adjust the test alone, we have re-analysed the outcome with a quasi-Poisson model that accounts for this mean-variance relationship directly. The main conclusions are unchanged, though one contrast that was borderline under the original analysis is no longer significant, and we have revised that claim accordingly (Results, page X; Methods, page X).

Practice

A reviewer writes: "Bartlett's test on your three groups is highly significant, so the equal-variance assumption is violated and the ANOVA results as reported cannot be trusted." You run the check:

Which of the three outcomes applies, and what do you write?