Navigate
Courses Tutorials Practice Pricing Tools Upgrade
Lesson 5 of 6

Fairness Basics

Lessons 2 to 4 asked how the model behaves: which feature moved a prediction, and what shape its effect traced. This final pair of lessons asks the question that matters most once a model decides something about a person: for whom does it work?

To make that concrete we switch to a decision that lands squarely on people. A bank runs a model that approves or denies small loans. Split its applicants into two groups, A and B, and a gap jumps out: group A is approved far more often than group B. Is that unfair? It depends entirely on what "fair" means, and it turns out you cannot have every version of fairness at once.

By the end of this lesson you will be able to:

  • Compute the three group metrics every audit starts with: selection rate, true-positive rate, and false-positive rate
  • State the three main fairness definitions and say which one a given gap violates
  • Explain the impossibility result: when two groups repay at different rates, some fairness definitions cannot hold together
  • Choose the definition that matches the real harm, apply one mitigation, and see honestly what gap it leaves behind

Prerequisites: you can fit and use a model in R (for example a random forest) and read a predict() output, and you have done Lesson 4: Partial Dependence, ICE, and ALE.

The setup

From "how it decides" to "for whom"

Every tool so far explained the model's behaviour in the abstract. Fairness is different: it needs a decision that helps or harms real people, and a notion of who deserved the good outcome. Loan approval has both. The good outcome is being approved; the deserving applicants are the ones who would have repaid.

So each applicant carries two facts. repaid is the ground truth, 1 if they would repay the loan (learned later, from who actually paid). approved is the model's decision, 1 if it said yes. We have 400 applicants, 200 per group. Build them once, from exact counts, so this page runs on its own.

RInteractive R
# 400 loan applicants across two groups. For each we record two facts: # repaid = 1 if they WOULD repay the loan (ground truth, the "deserving" applicant) # approved = 1 if the model APPROVED them (the decision) group <- rep(c("A", "B"), each = 200) # group A: 120 of 200 would repay; the model approved 96 of those 120, # plus 16 of the 80 who would have defaulted. # group B: 80 of 200 would repay; the model approved 48 of those 80, # plus 24 of the 120 who would have defaulted. repaid <- c(rep(1, 120), rep(0, 80), # A: 120 repay, 80 default rep(1, 80), rep(0, 120)) # B: 80 repay, 120 default approved <- c(rep(1, 96), rep(0, 24), rep(1, 16), rep(0, 64), # A's decisions rep(1, 48), rep(0, 32), rep(1, 24), rep(0, 96)) # B's decisions loans <- data.frame(group, repaid, approved) table(loans$group, approved = loans$approved) #> approved #> 0 1 #> A 88 112 #> B 128 72

  

  





Read the counts. In group A, 120 of 200 applicants would repay, and the model approved 112 of the 200. In group B, only 80 of 200 would repay, and it approved 72. Group A is approved more, but group A also holds more good borrowers to begin with. That difference in how often each group would repay, 60% versus 40%, is real and common, and untangling it from the model's decisions is the whole job of a fairness audit.