Navigate
Courses Tutorials Practice Pricing Tools Upgrade
Lesson 7 of 8

Logistic Regression Done Properly

In Lesson 6 you separated the two jobs a regression can do: explaining which predictors matter, and predicting a number for a new day, each with its own kind of interval. Every one of those models answered a question whose answer was a number: how many cups will Priya sell?

This lesson changes the question. Priya, still running her iced-coffee cart, no longer wants a cup count. She wants to know one yes-or-no thing each morning: will I sell out today? On a freezing day, almost never. On a scorcher, almost certainly. The honest answer is not "yes" or "no" but a probability, a number between 0 and 1 that slides smoothly from one to the other as the day heats up. The curve below is exactly that answer, and building it correctly is what this lesson is about.

By the end you will be able to:

  • Explain why ordinary linear regression is the wrong tool for a yes/no outcome
  • Turn a probability into odds and log-odds, and back again through the S-shaped logistic curve
  • Fit a logistic regression in R with glm(), and read its coefficients as odds ratios
  • Turn the model into a probability, pick a decision threshold, and judge whether it was done properly

Prerequisites: Lessons 1 to 6 (you can fit a line with lm(), read its coefficient table, and you know a probability is a number between 0 and 1). Every new term is defined as it appears.

The new question

The outcome is now yes or no

For six lessons the thing on the left of the ~ was a count: cups sold, a number that could be 48 or 63 or 71. Now it is a label: soldout is 1 on days Priya ran out and 0 on days she had cups left over. There is no in-between. You cannot sell out 0.6 of the way.

A fresh R session starts empty, so we build four weeks and a bit of her trading right here. Each row is one day: its high temperature, whether it was a weekend, and whether she sold out.

RInteractive R
set.seed(7) n <- 60 temp <- round(runif(n, 16, 34), 1) # each day's high, in Celsius weekend <- rbinom(n, 1, 2/7) # 1 on Saturdays and Sundays # nature's hidden rule: hotter days (and weekends) are likelier to sell out lp <- -14 + 0.5 * temp + 1.2 * weekend # the true log-odds, which we never see soldout <- rbinom(n, 1, 1 / (1 + exp(-lp))) # 1 = sold out, 0 = cups left over coffee <- data.frame(temp, weekend, soldout) table(coffee$soldout) #> #> 0 1 #> 36 24

  

  





Twenty-four sell-out days, thirty-six with cups to spare. Our whole job is to turn a day's temperature into the probability of that 1. The rule that generated the data is hidden inside lp (do not worry about it yet; that expression is precisely what the lesson will teach you to read). What matters now is the shape of the target: a column of 0s and 1s, nothing else.