6.1 From comparing groups to modelling relationships

We can compare two groups. Why is that not enough?

Unit 5 compared students above and below the median attendance. Those who attended more had higher average grades, by more than sampling variation alone would explain.

But knowing that two groups differ is only the beginning. We still do not know how grades change as attendance changes, or whether attendance itself is responsible for the difference.

That second gap is the deeper one, and it is worth seeing clearly.

The experiment we cannot run

What we would really like is a thought experiment. Take one student and observe them twice: once attending 20 classes and once attending 30, with everything else about them unchanged — the same ability, the same motivation, the same teacher, the same week. The difference in their grades would isolate the effect of attendance itself.

That experiment is impossible. A student cannot attend both 20 and 30 classes. Instead, we observe different students who differ in many ways at once. The challenge of empirical economics is to separate the effect of attendance from all the other differences between those students.

Simple regression does not solve that problem. Section 7.8 returns to this point in detail. If a reader finishes this unit believing that regression automatically uncovers causal effects, then this book has failed them.

What regression does is change the question we ask of the data, and build the machinery that later units extend towards the thought experiment above.

The question has changed

Instead of asking whether students with high attendance differ from those with low attendance, we now ask whether grades tend to rise as attendance rises.

Attendance is no longer a group label. It becomes a numerical variable whose entire range carries information.

That is the conceptual step from a \(t\)-test to a regression, and everything technical in this unit follows from it.

The price of the old question

To compare two groups, Unit 5 took a variable recording how many classes each of 680 students attended — a number running from 2 to 32 — and replaced it with a label saying “high” or “low”. A student who attended 32 classes and one who attended 28 were treated as identical, while one who attended 28 and one who attended 27 were placed in opposite groups.

We can quantify the price of that simplification.

students <- read.csv("data/attendance-grades.csv")
students$high <- students$attendance >= median(students$attendance)

c(split_t      = abs(t.test(sem_gpa ~ high, data = students)$statistic),
  regression_t = summary(lm(sem_gpa ~ attendance, data = students))$coefficients[2, 3])

#>    split_t.t regression_t 
#>        11.80        17.59

The same data. The same outcome. Almost the same question. Splitting attendance into two groups produces a \(t\) statistic of 11.8; using the full attendance variable produces one of 17.6.

The relationship was always present — the median split simply discarded much of the evidence for it.

Regression does not merely give us a better answer. It asks a different question — and in doing so improves on a two-sample comparison in two ways.

It uses the full explanatory variable rather than reducing it to two groups.
It estimates a slope — the expected change in \(y\) for a one-unit increase in \(x\) — instead of merely comparing two averages.

In the next unit we will add more explanatory variables, allowing us to compare observations while holding other measured characteristics fixed. That brings us closer to the counterfactual comparison we would like to make, although — as we shall see — it does not eliminate the problem entirely.