Summary
- OLS minimises \(\sum \hat{u}_i^2\). Setting both partial derivatives to zero
gives the normal equations, whose solution is
\(\hat{\beta}_1 = \sum(x_i-\bar{x})(y_i-\bar{y}) / \sum(x_i-\bar{x})^2\) and
\(\hat{\beta}_0 = \bar{y} - \hat{\beta}_1\bar{x}\).
- The slope is the sample covariance of \(x\) and \(y\) over the sample variance of
\(x\). That reading explains its sign, and it explains why \(x\) must vary: with
no variation in \(x\) the denominator is zero.
- By construction \(\sum\hat{u}_i = 0\), \(\sum x_i\hat{u}_i = 0\), and the fitted
line passes through \((\bar{x}, \bar{y})\). These are algebraic identities and
are not evidence about the model.
- \(\text{SST} = \text{SSE} + \text{SSR}\), and
\(R^2 = \text{SSE}/\text{SST} = 1 - \text{SSR}/\text{SST}\) is the share of the
variation in \(y\) the regression accounts for. In a simple regression it equals
the squared correlation between \(x\) and \(y\).
- \(R^2\) measures fit, not correctness. Attendance accounts for 31% of the
variation in semester GPA; values of 0.1 or 0.2 are unremarkable in economics.
- The six standard assumptions are known collectively as the classical linear
regression model. Under CLRM1–CLRM4, \(E(\hat{\beta}_1) = \beta_1\). The
derivation runs through
\(\hat{\beta}_1 = \beta_1 + \sum(x_i-\bar{x})u_i/\text{SST}_x\), which is the
most useful line in the unit.
- Adding CLRM5, \(\mathrm{Var}(\hat{\beta}_1) = \sigma^2/\sum(x_i-\bar{x})^2\). Small
errors and widely spread \(x\) both buy precision, which is why spread in the
explanatory variable is worth seeking out.
- \(\hat{\sigma}^2 = \sum\hat{u}_i^2/(n-2)\), the divisor reflecting the two
parameters estimated on the way.
- Testing \(H_0: \beta_1 = 0\) uses
\(t = \hat{\beta}_1/\mathrm{se}(\hat{\beta}_1)\) on \(n-2\) degrees of freedom,
exactly as in Unit 4. The interval is the more informative
report.
- CLRM6, normality of the errors, makes that \(t\) distribution exact in small
samples. It is the weakest of the six: the Central Limit Theorem makes it
unnecessary in large ones, whereas no sample size repairs a failure of CLRM4.
- When \(E(u \mid x) \neq 0\) the second term in the unbiasedness expression does
not vanish. That is omitted variable bias, and nothing in the regression
output reveals it.