Navigate
Courses Tutorials Practice Pricing Tools Upgrade
Lesson 9 of 13

Zero-Inflated and Hurdle Models

In Lesson 8 you gave counts the model they deserve: a Poisson regression on the log scale, and a negative binomial when the spread ran wider than a Poisson allowed. Both fixed how far the counts spread. This lesson is about a different kind of trouble, one that lives entirely at a single value: zero.

Meet Nadia, a ranger at Blue Lake State Park. As each visiting group packs up to leave, she writes down one number: how many fish they caught that day. Some groups caught six or seven. But when she tallies a month of clipboards, one thing jumps out: nearly half of all the groups caught nothing at all. A wall of zeros.

Here is the catch that makes those zeros special. A "0" on Nadia's clipboard hides two completely different stories. One group is a family who came to picnic and swim and never once put a line in the water. Another is a patient angler who fished all afternoon and simply got unlucky. Same number on the page, two utterly different reasons. A single count model, even a generous negative binomial, has no way to tell them apart, and as you will see, that is exactly why it cannot fit this data.

By the end of this lesson you will be able to:

  • Explain why a negative binomial still under-predicts a pile of zeros
  • Tell a structural zero (never at risk) from a sampling zero (at risk, but landed on 0)
  • Fit and read a zero-inflated model and a hurdle model in R, and choose the right one

Prerequisites: you can fit and read a Poisson or negative binomial count model (Lesson 8, the direct prerequisite), and you have met the GLM idea and read a logistic model as odds. Comfortable reading exp(coef) as a multiplier.

The problem

Even a negative binomial is not enough

Let us build Nadia's season so we can run everything on this page. Each row is one visiting group. Two hidden stories decide its catch: whether the group ever fishes at all, and, for those who do, how many fish they land.

RInteractive R
set.seed(2027) n <- 500 persons <- sample(1:4, n, replace = TRUE) # people in the group child <- sample(0:2, n, replace = TRUE) # young children along camper <- rbinom(n, 1, 0.6) # 1 = staying overnight to camp # story 1: some groups never fish (more likely with young kids, less if camping) never_fishes <- rbinom(n, 1, plogis(-0.5 + 1.1 * child - 1.3 * camper)) # story 2: groups that DO fish land a Poisson catch that grows with group size + camping catch <- rpois(n, exp(0.2 + 0.5 * persons + 0.4 * camper)) fish <- data.frame( persons = persons, child = child, camper = factor(camper, labels = c("day", "camp")), count = ifelse(never_fishes == 1, 0, catch) # a non-fisher records 0 ) c(groups = n, zeros = sum(fish$count == 0), busiest = max(fish$count)) #> groups zeros busiest #> 500 243 19

  

  





Of 500 groups, 243 caught nothing. That is 49% zeros. Now fit the two best single-equation count models from Lesson 8 and ask each one a simple question: how many zeros do you expect?




  

    
RInteractive R

    
library(MASS)
nb <- glm.nb(count ~ persons + child + camper, data = fish)

# expected number of zeros under each one-equation model, vs the 243 we saw
pois_all <- glm(count ~ persons + child + camper, data = fish, family = poisson)
round(c(
  observed = sum(fish$count == 0),
  poisson  = sum(dpois(0,   fitted(pois_all))),
  negbin   = sum(dnbinom(0, mu = fitted(nb), size = nb$theta))
))
#> observed  poisson   negbin 
#>      243       80      197

    

      
      
    

    

  

  





A plain Poisson expects a laughable 80 zeros. The negative binomial, our Lesson 8 fix for overdispersion, does better and reaches 197, but it is still 46 zeros short of what Nadia actually recorded. It gave up too. The extra spread of a negative binomial helps with the long tail, but it cannot manufacture this pile of zeros, because all of its zeros are tied to the same single mean as its other counts.