2.1 The problem of estimation

We want to know something about a population we cannot observe. What exactly are we going to do?

The previous unit described how a statistic behaves: repeated samples produce different averages, and those averages have a distribution of their own.

It did not tell us what to do.

Consider the position of anyone conducting a real survey. There is a quantity they want to know — the average annual earnings of Indian wage workers, the proportion of households with piped water, the average yield per hectare of paddy in Punjab. That quantity is a fixed number. It exists. It is simply not observed, because nobody has surveyed all 1.4 billion people, and nobody ever will.

What they have instead is a sample, and a decision to make: which number computed from that sample should be put forward as the answer?

A parameter is a fixed, unknown feature of the population, such as \(\mu\) or \(\sigma^2\).

An estimator is a rule for computing a guess at a parameter from a sample. Because it is computed from a sample, it is a random variable, and we write it with a capital letter: \(\bar{X}\).

An estimate is the number that rule produces from the sample actually drawn: \(\bar{x} = 54{,}300\).

The distinction between the last two is worth pausing on, because the words are often used interchangeably in ordinary speech. An estimator is the rule: take the observations, add them together and divide by \(n\). An estimate is the number produced after the rule has been applied to a particular sample.

The difference matters. An estimate can never be judged directly, because we almost never know the true population mean. If a survey estimates average monthly income to be ₹50,000, we do not know whether the truth is ₹49,800, ₹50,100 or ₹55,000. The truth is precisely what we are trying to discover.

The estimator, however, can be judged. Although we never learn whether one particular estimate is close to the truth, we can ask how the estimator behaves across many hypothetical samples. Does it systematically overestimate the population mean? Does it systematically underestimate it? Does it produce very different answers from one sample to the next? These are questions about the procedure, not about any single outcome.

Since the parameter is never known, an estimator cannot be judged by any single estimate. It is judged by how it behaves across the many samples that might have been drawn.

Evaluating procedures rather than outcomes is the idea that drives the whole of this unit.

There are many possible estimators of the population mean. We could use the sample median. We could take the midpoint of the largest and smallest observations. We could even ignore the data entirely and always report ₹50,000. Each is an estimator of \(\mu\), because each provides a rule for producing an estimate from a sample.

Why, then, has statistics settled almost universally on the sample mean?

That is the question for the rest of this unit.