Navigate
Courses Tutorials Practice Pricing Tools Upgrade
Lesson 6 of 11

Instrumental Variables and 2SLS

Lesson 5 ended on a puzzle. When a cutoff only nudges the chance of treatment, you recover the effect by scaling the jump in the outcome by the jump in take-up. That ratio, outcome-move over treatment-move, has a name and a whole method behind it: instrumental variables.

Here is the problem it solves. You want to know whether doing more lessons on a language app really raises exam scores. But the keen students do more lessons and score higher anyway, so a plain regression credits the lessons with a boost that was really just keenness. Instrumental variables finds a lever that pushes some students to do more lessons for a reason unrelated to how keen they are, and reads the true effect off that push.

By the end of this lesson you will be able to:

  • Explain why regressing the outcome on the treatment is biased when a hidden trait drives both
  • Say what makes a variable a valid instrument: relevance, exclusion, and independence
  • Build the two-stage least squares estimator by hand in R and watch it recover the true effect
  • Spot the weak-instrument trap with the first-stage F before it quietly wrecks your estimate

Prerequisites: Lesson 1 (confounding and the potential-outcomes gap), Lesson 3 (why a naive comparison is biased), and Lesson 5 (the fuzzy-cutoff ratio you are about to generalize). You can fit lm and read its coefficients in base R.

The setup

A language app, and a slope that flatters

Meet Lingo, a Spanish-learning app. Every user does some number of lessons, and at the end takes a standard exam scored 0 to 100. The product team wants one number: how many exam points does one extra lesson actually buy? If lessons genuinely lift scores, they will push notifications, streaks, the works, to get people doing more of them.

We will build Lingo's data ourselves, so we know the true answer to check against. In our simulated app one extra lesson truly adds 0.4 points to the exam. There is also a hidden trait, call it motivation (grit, ambition, spare time), that makes a student do more lessons and score higher for reasons of their own. And there is the lever we will need later: the app randomly switches on daily reminders for half its users. Each lesson runs in a fresh R session, so we create everything here.

RInteractive R
set.seed(6) n <- 2000 motivation <- rnorm(n) # hidden trait: grit, ambition, spare time reminder <- rbinom(n, 1, 0.5) # the app randomly switches daily reminders ON lessons <- round(30 + 8 * reminder + 5 * motivation + rnorm(n, 0, 4)) score <- round(40 + 0.4 * lessons + 8 * motivation + rnorm(n, 0, 4), 1) app <- data.frame(reminder, lessons, score) head(app) #> reminder lessons score #> 1 1 46 57.8 #> 2 1 41 47.1 #> 3 1 48 62.0 #> 4 1 44 69.5 #> 5 0 35 54.2 #> 6 0 29 47.9

  

  





The obvious way to answer the team's question is to regress the exam score on the number of lessons and read the slope.




  

    
RInteractive R

    
naive <- lm(score ~ lessons, data = app)
round(coef(naive), 2)
#> (Intercept)     lessons
#>       16.05        1.10

    

      
      
    

    

  

  





The slope says 1.10 points per lesson. Nearly three times the 0.4 that lessons truly deliver. Take it at face value and you would promise a student that grinding out ten more lessons lifts their exam by eleven points, when the honest figure is four. Plot the data and the tempting, too-steep line is right there.




  

    
RInteractive R

    
library(ggplot2)
p_raw <- ggplot(app, aes(lessons, score)) +
  geom_point(alpha = 0.25, size = 0.8, colour = "#1c2c4f") +
  geom_smooth(method = "lm", se = FALSE, colour = "#c0392b") +
  labs(x = "lessons completed", y = "final exam score",
       title = "More lessons, higher scores: but how much is really the lessons?")
print(p_raw)