Survival Data and Censoring

You have modeled numbers with regression and categories with classification. This course is about a third kind of outcome: how long until something happens.

Dr. Meera Rao has spent two years testing a new heart-failure drug. She enrolled 30 patients, 15 on the standard drug and 15 on the new one (each group is called an arm of the trial), and followed every patient for up to 24 months, recording the months from enrollment to death. The chart below summarizes her trial: each curve tracks the fraction of an arm still alive, month by month, and the small vertical ticks mark patients who slipped out of view while still alive. Those ticks, the patients whose story has no ending, are what this lesson is about.

By the end of this lesson you will be able to:

• Say exactly what a censored row of data does and does not tell you
• Show, with real numbers, why dropping censored patients or counting them as deaths biases results low, and why ordinary regression inherits the same problem
• Encode follow-up times and event indicators as a survival outcome in R
• Read a survival curve S(t), find its median, and connect it to the hazard

Prerequisites: you can fit and read a simple regression, you know probability as a long-run fraction, and you can work with a data frame in R.

The outcome is a waiting time

Take one of Dr. Rao's patients: Bhavna enrolled on 3 March, started the standard drug that day, and died 3.2 months later. Her outcome is not a category and not an ordinary measurement. It is a waiting time: the length of the gap between a defined starting point and a defined event.

Every time-to-event question has the same three ingredients:

• a time origin: the moment the clock starts (here, the day a patient enrolls)
• a time scale: the units the clock counts (here, months since enrollment)
• an event: the thing that stops the clock (here, death)

Swap the ingredients and the same machinery answers very different questions: months until a subscriber cancels, hours until a compressor fails, days until a loan defaults. Medicine named the field, so the outcome is called survival time even when nothing is alive.

Here are 10 of Dr. Rao's standard-arm patients. Each lesson runs in a fresh interactive R session, so we build the data right here (run this first):

The months column records how long Dr. Rao watched each patient. The status column is the twist this whole lesson is about. Hold that thought for one step.