4.14 Looking ahead

We have now completed the basic toolkit of statistical inference for a single mean. Given one sample, we can

• estimate an unknown population mean,
• quantify the uncertainty around that estimate with a confidence interval, and
• test a specific claim about the population with a hypothesis test.

The last two, as Section 4.6 showed, are the same object seen from two sides.

Every test in this unit compared a sample against a claimed number — a figure from a government report, a manufacturer’s specification, a previous year’s estimate. That number had to come from somewhere outside the data.

Most questions worth asking do not arrive in that form.

Did the households that received the transfer end up better off than those that did not? Do girls in the scheme attend school more than girls outside it? Did the districts that got the road see wages rise more than the districts that did not? In each case there is no external claim to test against. There are two groups, each with its own sample mean, and the question is whether the difference between them is larger than sampling variation alone would produce.

That is the subject of the next unit. It is also where the difficulty changes character. Deciding whether two groups differ is a statistical problem, and we will solve it. Deciding whether one group differs because of the treatment is a question about how the groups came to be formed — and no test statistic can answer it.