The Key Concept
Difference-in-Differences
In 2023/24, the Premier League introduced new timekeeping rules to increase effective playing time. Did it lead to more goals?
Goals per club per game spiked from 1.43 to 1.64 in the first season with the new rules — then fell back to 1.47. But can we credit the timekeeping rule for any of that change? Other things change over time too — tactics evolve, squad compositions shift, other rules are tweaked.
How do we know if the spike were due to the implementation of the new timekeeping rule?
And not due to other things that changes over time?
Many things happen along the time between the pre and post-intervention. They can confound the relationship between the treatment and the outcome.
Remember, a causal effect is the difference between the outcome resulting from the cause (factor) and the potential outcome had it not been affected while keeping all other factors constant.
So how can we measure that?
The solution? Bring in a control group — another group that is never treated.
At first, this seems like we've made things worse. We've added a new back door through Group. The control group may be fundamentally different from the treated group!
With Difference-in-Differences, we can close the back door from Time and Group, and focus on the path from the Treatment to the Outcome.
Now let's bring in the control group — The Bundesliga. And see what it tells us.
The Bundesliga moved quite steadily. It drifted up slightly in 23/24 then drifted back. The Premier League, however, spiked sharply in the same season the timekeeping rule kicked in.
If the two leagues were trending similarly before the rule, then the Bundesliga tells us what the EPL would have done without it. However the changing trend for the Premier League indicates that it may have something to do with the new timekeeping rule!
Difference-in-Differences isolates the treatment effect in two steps. Let's walk through them.
Pre- and post-treatment averages for each group. Dots represent goals per club per game, collapsed to a single before/after mean.
For this to work, we need the parallel trends assumption.
The parallel trends assumption says: if no treatment had occurred, the difference between the treated and control groups would have stayed the same over time.
It's okay that the groups have different levels — what matters is that they would have changed by the same amount.
The dashed blue line shows where the Premier League would have been without the timekeeping rule, if it had followed the same trend as the Bundesliga. The gap between the actual and counterfactual lines is the DiD effect.
Parallel trends is inherently unobservable — it's about a counterfactual we can never see. We can look for suggestive evidence (more on that in Chapter 3), but we can never prove it.
Beyond parallel trends, DiD also relies on a few other key assumptions.
SUTVA (Stable Unit Treatment Value Assumption)
The treatment status of one unit doesn't affect the outcomes of another. In our example: the EPL adopting timekeeping rules shouldn't change how goals are scored in the Bundesliga.
No Anticipation
The treated group doesn't change its behavior before the treatment is actually implemented. Clubs shouldn't start playing differently in 22/23 just because they know the timekeeping rule is coming in 23/24.
DiD uses the untreated group's change over time as a stand-in for the counterfactual — what would have happened to the treated group without treatment.