9.9 What R-squared cannot tell you

The model explains 54% of the variation. Is it a good model?

There is no answer to that question, because \(R^2\) was never a measure of whether a model is correct.

\(R^2\) answers exactly one question: how much of the variation in \(y\) does the fitted model account for in this sample?

It says nothing about whether the coefficients are unbiased, whether the standard errors are right, whether the functional form is sensible, or whether anything can be interpreted causally.

Four specific limitations are worth stating plainly.

A high \(R^2\) does not mean the estimate is unbiased. Omitted variable bias depends on what is missing from the model. A regression can account for 95% of the variation and still have every coefficient wrong, because the variable that would have corrected them was never included. Nothing in \(R^2\) can detect a variable that is not there.

A low \(R^2\) does not mean the model is useless. Individual behaviour is mostly idiosyncratic. Cross-sectional regressions on household data routinely report \(R^2\) of 0.1 or 0.2, and the coefficients may still be precisely estimated and economically important. Section 9.4 explains 32% of the variation in log earnings; the return to schooling is not thereby 32% true.

\(R^2\) cannot be compared across different dependent variables. The earnings regression in levels and the same regression in logs have different \(R^2\) because they are explaining the variation in different quantities. Choosing between functional forms on \(R^2\) compares two things that are not comparable.

\(R^2\) rewards prediction, not explanation. A variable measured after the outcome, or one that is a consequence of it, will improve the fit spectacularly and destroy the interpretation. That is the subject of the next section.

The two purposes of regression are worth separating here.

If the goal is prediction — forecasting next year’s output, flagging which loans will default — then fit is close to the whole point, and \(R^2\) is a reasonable thing to look at.

If the goal is explanation — estimating what a year of schooling is worth — then fit is nearly beside the point. What matters is whether the comparison being made is the right one, and \(R^2\) is silent about that.

Most of economics is the second kind of question.