3.1 One sample, one number

We have one sample and one number. What are we entitled to say?

The previous unit answered an important question: why do statisticians use the sample mean to estimate the population mean? We found that it has two desirable properties. First, it is unbiased — over repeated samples it is centred on the true population mean. Second, its variability decreases as the sample size grows. Larger samples therefore tend to produce estimates closer to the truth.

There was, however, one feature of that argument which is impossible in practice.

To judge the behaviour of the sample mean we repeatedly drew thousands of samples from a population whose true mean we already knew. By watching the sampling distribution emerge, we could see that the sample mean was centred correctly and became more precise as the sample grew.

Real surveys do not work that way.

A labour force survey, a household expenditure survey, a school attendance audit — each is conducted once. We observe one sample, calculate one estimate, and the data collection ends. We never see the thousands of alternative samples that might have been drawn.

Suppose a survey of students reports an average attendance of 26.1 classes.

Is the population mean exactly 26.1? Almost certainly not. How close is it? The data alone do not tell us.

Simply reporting the estimate is therefore incomplete. We already know that every sample mean varies from sample to sample; some will be larger than the population mean and others smaller. Without a measure of that variability, the estimate has no context.

The natural response is to report the estimate together with its standard error. For the sample mean we found that

\[\mathrm{se}(\bar{X}) = \frac{\sigma}{\sqrt{n}}\]

This is a useful result, and it immediately raises another difficulty. The formula contains the population standard deviation \(\sigma\), which is unknown. If we are estimating the population mean because it is unknown, we can hardly assume we know the population standard deviation.

One missing quantity has become two.

Nor are all questions about population means. Many surveys seek to estimate proportions instead.

  • What proportion of households live below the poverty line?
  • What proportion of workers are employed in the informal sector?
  • What proportion of children are fully vaccinated?

These require estimators of a different kind.

Finally, even once we can calculate both an estimate and its standard error, we still face the central question of statistical inference. Suppose two surveys both estimate average attendance at 26.1 classes. One has a standard error of 0.4; the other has a standard error of 3.2.

How should those numbers change our confidence in the estimate? How uncertain is “uncertain”?

Answering that requires moving beyond a single number. Rather than reporting only a point estimate, we will construct interval estimates — ranges of plausible values for the unknown parameter, together with a statement of how reliable such ranges are.

This unit develops the three ingredients needed for statistical inference.

  1. How to estimate unknown population variability.
  2. How to extend estimation from means to proportions.
  3. How to combine point estimates and standard errors into confidence intervals.

By the end of the unit we will be able to replace a statement such as

The average attendance is 26.1 classes.

with one that is considerably more informative:

We estimate the average attendance to lie between 24.4 and 27.8 classes, with 95% confidence.

That is the central goal of this unit.