3.6 How large a sample do we need?

A survey has not yet been run. How many households should it cover?

Everything so far took the sample as given. In practice the question usually arrives earlier, from someone who has to budget for the fieldwork.

The margin of error for a mean is

\[E = z_{\alpha/2}\,\frac{\sigma}{\sqrt{n}}\]

which can be solved for \(n\):

\[n = \left(\frac{z_{\alpha/2}\,\sigma}{E}\right)^{2}\]

The squaring is the \(\sqrt{n}\) rule from the previous unit, arriving from the other direction: halving the margin of error quadruples the sample.

For a proportion the same rearrangement gives

\[n = \frac{z_{\alpha/2}^{2}\;p(1-p)}{E^{2}}\]

with one difficulty. It requires \(p\), which is what the survey exists to find out. The standard resolution is to use the worst case. Since \(p(1-p)\) is maximised at \(p = 0.5\), setting \(p = 0.5\) gives a sample size large enough whatever the truth turns out to be.

sample_size_prop <- function(margin, level = 0.95, p = 0.5) {
  z <- qnorm(1 - (1 - level) / 2)
  ceiling(z^2 * p * (1 - p) / margin^2)
}

data.frame(margin_of_error = c("5%", "3%", "2%", "1%"),
           n_at_95 = sapply(c(0.05, 0.03, 0.02, 0.01), sample_size_prop),
           n_at_90 = sapply(c(0.05, 0.03, 0.02, 0.01), sample_size_prop,
                            level = 0.90))

#>   margin_of_error n_at_95 n_at_90
#> 1              5%     385     271
#> 2              3%    1068     752
#> 3              2%    2401    1691
#> 4              1%    9604    6764

These numbers explain something otherwise puzzling about published surveys. A national opinion poll covering 1.4 billion people typically samples one or two thousand — and the table shows why that is sufficient rather than negligent. The population size does not appear anywhere in the formula. What determines precision is the number of people asked, not the fraction of the country they represent.

This calculation answers a narrow question: how large a sample is needed for the standard error to reach a target, assuming the sample is drawn properly.

It says nothing about the failures of Section 2.2, and the sample size it returns will not repair any of them. A survey of two thousand urban households is not a survey of India, however carefully the two thousand was computed.

Intervals for a proportion

Collecting the pieces, an interval for a population proportion is

\[\hat{p} \;\pm\; z_{\alpha/2}\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}\]

Note that \(z\) appears here rather than \(t\). The \(t\) correction exists because \(\sigma\) was estimated separately from \(\mu\); for a proportion, \(\hat{p}\) determines the standard error on its own, so there is no second estimate to account for.

Suppose that in a random sample of 1,000 secondary school teachers in Karnataka, 518 are women.

x <- 518
n <- 1000

p_hat <- x / n
se    <- sqrt(p_hat * (1 - p_hat) / n)

round(c(p_hat = p_hat, se = se,
        lower = p_hat - 1.96 * se,
        upper = p_hat + 1.96 * se), 4)

#>  p_hat     se  lower  upper 
#> 0.5180 0.0158 0.4870 0.5490

So we report that between 48.7% and 54.9% of secondary school teachers in Karnataka are women, with 95% confidence.

That interval contains 0.5, which is worth noticing. The sample proportion is above half, but the data are also consistent with women being a minority of the profession. A headline reading “most secondary school teachers are women” would be going beyond what a sample of a thousand can support.