9.7 Does adding a variable ever cost anything?

If omitting a relevant variable causes bias, why not include everything available?

That is the second decision, and the instinct behind it is understandable. Section 8.7 showed that leaving out a relevant variable biases the estimate. Including it fixes the problem. Why not include everything?

Start with the easy case: a variable that does not belong in the model at all. Its true coefficient is zero.

Including an irrelevant variable does not bias anything.

The unbiasedness proof of Section 7.4 does not require that every included variable matter. If \(\beta_3 = 0\), then OLS estimates \(\beta_3\) as zero on average and leaves the other coefficients centred on the truth.

So the cost is not bias. It is precision, and how much precision depends entirely on one thing.

set.seed(7)

out <- replicate(3000, {
  x1  <- rnorm(100)
  irr_alone <- rnorm(100)                                   # unrelated to x1
  irr_close <- 0.9 * x1 + sqrt(1 - 0.81) * rnorm(100)       # correlated 0.9 with x1
  y <- 1 + 2 * x1 + rnorm(100, sd = 2)                      # neither belongs in the model

  c(b_none  = coef(lm(y ~ x1))[2],
    b_alone = coef(lm(y ~ x1 + irr_alone))[2],
    b_close = coef(lm(y ~ x1 + irr_close))[2],
    se_none  = coef(summary(lm(y ~ x1)))[2, 2],
    se_alone = coef(summary(lm(y ~ x1 + irr_alone)))[2, 2],
    se_close = coef(summary(lm(y ~ x1 + irr_close)))[2, 2])
})

round(rowMeans(out), 4)

#>  b_none.x1 b_alone.x1 b_close.x1    se_none   se_alone   se_close 
#>     2.0001     1.9999     1.9896     0.2023     0.2033     0.4675

The true slope is 2. All three estimates average 2, exactly as promised — adding a variable with no effect costs nothing in bias whether or not it is related to \(x_1\).

The standard errors tell a different story. Adding the unrelated variable moves the standard error from 0.202 to 0.203, which is nothing. Adding the correlated one moves it to 0.468, more than doubling it.

This is the \((1-R_j^2)\) factor of Section 8.4, and it is worth seeing that it does not care whether the added variable matters.

A control that duplicates the variable of interest consumes its variation whether or not it has any effect on the outcome. The regression is left estimating a coefficient from whatever variation survives, and pays for the privilege in precision.