6.9 Practice problems

Work each one before opening the solution. The question bank that follows has no answers.

Concept checks

6.1 Why does regression estimate a conditional mean rather than predict individual outcomes?

Because \(y\) is not determined by \(x\). At any given attendance level, students obtain a wide range of grades, because grades also depend on effort, ability and much else — all of which sit in the error term.

What is stable across the population is the average of \(y\) at each value of \(x\). That is what \(E(y \mid x)\) describes and what the regression line estimates.

The scatter around the line is therefore not a failure of the method. It is the part of \(y\) that \(x\) does not account for, and a model claiming otherwise would be claiming that attendance alone determines grades.

6.2 Why is minimising the sum of residuals a bad criterion?

Because positive and negative residuals cancel, so a line that misses badly in both directions can have residuals summing to zero.

Misses of \(+8, -8, +8, -8\) sum to zero, and so do misses of \(+1, -1, +1, -1\). By that criterion the two lines are equally good, though the first is far worse.

Squaring removes the cancellation and makes large misses count more than small ones, which is what we want from a measure of fit.

6.3 A student writes: “the regression line shows what GPA a student who attended 25 classes will get.” Correct the statement.

The line gives the average GPA among students who attended 25 classes, which in these data is 6.10. It does not give what any particular such student will get.

Forty-nine students attended exactly 25 classes, scoring between 1.58 and 9.08. A prediction for an individual would be wrong for almost all of them.

The confusion is between the conditional mean and the individual outcome — which is the difference the error term exists to describe.

Explain

6.4 State the zero conditional mean assumption for a regression of earnings on years of schooling, in concrete terms, and say whether you believe it.

The model is \(\textit{earnings} = \beta_0 + \beta_1\,\textit{schooling} + u\), where \(u\) holds everything else affecting earnings — ability, family connections, school quality, motivation, region.

\(E(u \mid \textit{schooling}) = 0\) says that people with more schooling are, on average, no more able, no better connected, and no better taught than people with less.

That is not believable. Schooling is not distributed at random: children of wealthier and better-connected families obtain more of it, and those advantages raise earnings on their own. So \(u\) is positively related to schooling, and \(\hat{\beta}_1\) overstates the return to education.

This does not make the regression useless. It correctly describes how average earnings differ across schooling levels. It simply does not measure what an extra year of schooling would do to a given person.

6.5 Distinguish the error \(u_i\) from the residual \(\hat{u}_i\).

The error is the deviation of \(y_i\) from the population regression line: \(u_i = y_i - \beta_0 - \beta_1 x_i\). It involves the true parameters and is therefore unobservable.

The residual is the deviation from our fitted line: \(\hat{u}_i = y_i - \hat{\beta}_0 - \hat{\beta}_1 x_i\). It is computed from the data and can be printed.

The relationship mirrors \(\mu\) against \(\bar{x}\). Residuals are estimates of errors, and confusing the two leads to claims like “the errors are uncorrelated with \(x\), so CLRM4 holds” — which is wrong, because the residuals are uncorrelated with \(x\) by construction, whatever the errors do.

6.6 Explain why \(y_i = \beta_0 + \beta_1 x_i + u_i\) is not an extra assumption once the conditional expectation function has been defined.

Because \(u_i\) is defined as the gap between what individual \(i\) obtained and the average for their value of \(x\):

\[u_i = y_i - E(Y \mid X = x_i)\]

Rearranged, that reads \(y_i = E(Y \mid X = x_i) + u_i\), which is an identity. It says only that an observation equals its conditional mean plus however far it sits from that mean.

Substituting the linear CEF gives \(y_i = \beta_0 + \beta_1 x_i + u_i\). The genuine assumption was made earlier, when we chose to treat the conditional mean as linear. Writing the model down adds nothing but a name for the remainder.

6.7 Two sections of the same class produce fitted slopes of 0.300 and 0.162. A student concludes that one section must contain an arithmetic error. Evaluate.

No error is implied. Each section is a different sample from the same population, and every sample produces a different sample regression function — exactly as every sample produced a different \(\bar{x}\) in Unit 2.

Neither slope is \(\beta_1\). Both estimate it, and with thirty students each they estimate it imprecisely. The whole class of 680 gives 0.189, which is also an estimate rather than the truth.

The useful response is not to hunt for a mistake but to ask how much \(\hat{\beta}_1\) varies from sample to sample — a question about its sampling distribution, and the subject of the next unit.

Numerical and conceptual

6.8 A population has these conditional means of \(y\) given \(x\):

\(x\)	1	2	3	4
\(E(Y \mid X = x)\)	5	9	13	17

1. Is the population regression function linear here?
2. Write down \(\beta_0\) and \(\beta_1\).
3. An individual with \(x = 3\) has \(y = 11\). What is their \(u_i\)?
4. What is \(E(u \mid X = 3)\), and why?

(i) Yes. The conditional mean rises by exactly 4 for each unit increase in \(x\), so one straight line passes through all four points.

(ii) \(\beta_1 = 4\). Working back from \(E(Y \mid X = 1) = 5\) gives \(\beta_0 = 5 - 4 = 1\), so the PRF is \(E(Y \mid X = x) = 1 + 4x\).

(iii) \(u_i = y_i - E(Y \mid X = 3) = 11 - 13 = -2\). This individual sits two units below the average for their value of \(x\).

(iv) Zero. The conditional mean is the average of \(y\) among all individuals with \(X = 3\), so deviations from it must average to zero within that group. That is why \(E(u \mid x) = 0\) is natural when the linear CEF is correct.

6.9 Explain why a regression of \(y\) on \(x\) cannot be estimated when every observation has the same value of \(x\) — using the definition of the conditional mean rather than any formula.

The PRF describes how the average of \(y\) changes as \(x\) changes. If \(x\) never changes, there is exactly one conditional mean to observe and no information about any other value.

Concretely: if all 680 students had attended 25 classes, we could compute the average GPA of students attending 25 classes and nothing else. Whether GPA would be higher at 30 classes is unanswerable from such data — not because the arithmetic breaks down, but because the comparison the question requires was never observed.

Variation in the explanatory variable is not a technical requirement. It is the evidence.