5.2 What does a difference of means do across samples?

We know how \(\bar{X}\) behaves. How does \(\bar{X} - \bar{Y}\) behave?

Take a sample \(x_1, \ldots, x_n\) from a population with mean \(\mu_x\) and standard deviation \(\sigma_x\), and an independent sample \(y_1, \ldots, y_m\) from a population with mean \(\mu_y\) and standard deviation \(\sigma_y\). The sample sizes need not be equal.

The null hypothesis is that the two populations have the same mean:

\[H_0 : \mu_x = \mu_y \qquad H_A : \mu_x \neq \mu_y\]

It is more useful to write the same thing as a statement about the difference:

\[H_0 : \mu_x - \mu_y = 0\]

because that turns a comparison of two quantities into a claim about one, and we already know how to test a claim about one quantity.

The quantity we will study is therefore \(\bar{X} - \bar{Y}\). Two things need establishing: where its distribution sits, and how wide it is.

Where it sits. Expectation passes through a difference:

\[E[\bar{X} - \bar{Y}] = E[\bar{X}] - E[\bar{Y}] = \mu_x - \mu_y\]

So the difference of sample means is an unbiased estimator of the difference of population means. Under \(H_0\) it is centred on zero.

How wide it is. Variance does not pass through a difference in the same way. For independent quantities,

\[ \begin{aligned} \mathrm{Var}[\bar{X} - \bar{Y}] &= \mathrm{Var}[\bar{X}] + \mathrm{Var}[-\bar{Y}] \\[4pt] &= \mathrm{Var}[\bar{X}] + (-1)^2\mathrm{Var}[\bar{Y}] \\[4pt] &= \frac{\sigma_x^2}{n} + \frac{\sigma_y^2}{m} \end{aligned} \]

using \(\mathrm{Var}(aX) = a^2\mathrm{Var}(X)\) from Unit 2, with \(a = -1\). Taking the square root gives the standard error of the difference:

\[\mathrm{se}(\bar{X} - \bar{Y}) = \sqrt{\frac{\sigma_x^2}{n} + \frac{\sigma_y^2}{m}}\]

Independence is needed for the first line, exactly as it was in Unit 2. It is what allows the variance of a sum to be the sum of the variances.

The variances add, even though the means subtract.

This is the single most important line in the unit. Squaring \(-1\) gives \(+1\), so subtracting a second noisy quantity makes the result more variable, not less.

Comparing two groups is therefore harder than measuring one. Both estimates wobble, and both wobbles end up in the answer.

That has a practical consequence worth seeing now rather than later. Suppose two groups each have \(\sigma = 10\) and each is sampled \(n = 100\) times. The standard error of either mean alone is \(10/\sqrt{100} = 1\). The standard error of their difference is

\[\sqrt{\frac{100}{100} + \frac{100}{100}} = \sqrt{2} \approx 1.41\]

about 40% larger. Detecting a difference between two groups needs more data than estimating either group’s mean to the same precision.