5.1 Comparing two groups

Two groups, two sample means, and they are not equal. What have we learned?

Suppose a claim is made that two populations have the same mean. Male and female students at a university consume the same number of cigarettes on average; conventional and high-yielding wheat produce the same yield per hectare; two districts have the same average household income.

We could settle it by measuring everyone in both populations. That is expensive in exactly the way Unit 2 described, and nobody does it.

So we draw a sample from each group and compute both sample means. Unlike the one-sample case, we now have two numbers, each carrying its own sampling variation.

Suppose they differ. Have we shown the populations differ?

No — and for the same reason as in Unit 4, only more so.

Each sample mean would miss its population mean even if we had measured perfectly. Now there are two of them, each free to land high or low. A difference between two sample means is guaranteed to appear even when the two populations are identical.

So the question is the one we have asked all along, adapted:

If the two populations really had the same mean, how often would two samples come out as far apart as ours did?

If that would happen often, we have learned nothing about the populations. If it would almost never happen, then either something unusual happened to us or the two populations are not the same.

Everything that follows is machinery for answering that question. And as before, the machinery needs a sampling distribution — this time of a difference.