Summary
- Regression asks how one variable is related to another rather than whether two
groups differ. Splitting attendance at the median gave \(t = 11.8\); using the
whole variable gave \(t = 17.6\).
- Individuals with the same \(x\) obtain very different \(y\). Forty-nine students
attended exactly 25 classes and scored between 1.58 and 9.08, so no line can
pass through the individual points.
- The population regression function is the conditional mean
\(E(Y \mid X = x)\). It is not a fitted line, and it exists whether or not we
ever estimate it.
- Since each conditional mean is a function of \(x\), the PRF is also the
conditional expectation function. It says how the average of \(y\) moves
with \(x\), not what any individual obtains.
- Taking the CEF to be linear is a modelling choice, made after the target is
defined rather than before it.
- Writing \(y_i = E(Y \mid X = x_i) + u_i\) and substituting the linear CEF gives
\(y_i = \beta_0 + \beta_1 x_i + u_i\) — the PRF with a name attached to the part
it does not describe, not a new assumption.
- The error term is not a mistake and not measurement noise. It is the rest
of the world: effort, ability, teaching, health, luck.
- \(E(u) = 0\) costs nothing, since a non-zero average can be absorbed into the
intercept. \(E(u \mid x) = 0\) costs everything, cannot be tested, and is what
separates a description from an effect.
- \(\beta_0\) and \(\beta_1\) are population parameters; \(\hat{\beta}_0\) and
\(\hat{\beta}_1\) are sample estimates defining the sample regression
function. Two sections of thirty students gave slopes of 0.300 and 0.162.
- The error \(u\) is the deviation from the true line and is unobservable. The
residual \(\hat{u}\) is the deviation from the fitted line and can be
printed.
- Minimising \(\sum \hat{u}_i\) fails because positive and negative misses cancel.
Ordinary least squares minimises \(\sum \hat{u}_i^2\).