Introduction to Survival Analysis
Introduction to Survival Analysis
Time Until Something Happens
How long until a machine part fails? How long will a customer keep their subscription? How long does it take for a patient to recover after a treatment? These questions are all about the time until a specific event occurs. Survival analysis is the set of statistical tools we use to answer them.
Despite its name, it’s not just about life and death. It's used in engineering, economics, and marketing to study 'lifetimes' of all kinds. The 'event' could be anything from a device breaking down to a person unsubscribing from a service.
Time-to-event data
noun
A type of data that measures the duration from a defined starting point to the occurrence of a specific event.
This data has two main components: a time duration and an event indicator. The indicator is a simple flag, usually a 1 or 0, that tells us whether the event happened at the recorded time (1) or if observation stopped for some other reason (0).
The Problem of Incomplete Data
Real-world studies are often messy. We can't always wait for the event to happen for every subject in our study. Imagine a medical study tracking patients for five years. When the five years are up, some patients might not have experienced the event yet. Others may have moved to a different city or simply stopped participating.
We can't just ignore these subjects. Their data is still valuable because we know they 'survived' for at least a certain amount of time. This situation of having incomplete observation time is called censoring.
Censoring
noun
A condition in which the value of a measurement is only partially known. In survival analysis, it occurs when a subject’s time-to-event is not fully observed.
The most common type is right-censoring. This happens when a subject's follow-up period ends before the event is observed. For example, a study on a new type of battery might end after two years. Any batteries still working at that point are right-censored. We know they lasted at least two years, but we don't know their exact failure time.
Another, less common, form of incomplete data is truncation. This occurs when subjects are only included in the study if they meet a certain condition. For instance, if we study patients who have already survived for one year after a diagnosis, our data is left-truncated. We've excluded anyone who had the event within the first year.
Survival and Hazard Functions
To analyze this kind of data, we use two key mathematical concepts: the survival function and the hazard function. They are two sides of the same coin, offering different perspectives on the time-to-event process.
The survival function looks at the big picture of survival over time, while the hazard function zooms in on the risk of an event at a specific moment.
The survival function, denoted as , gives the probability that a subject survives past a specific time . It answers the question: "What's the probability the event has not occurred by this time?"
Here, is the time when the event occurs. The survival function always starts at 1 (since the probability of surviving past time 0 is 100%) and decreases over time as more subjects experience the event. It can never go up.
The hazard function, , is a bit more subtle. It represents the instantaneous risk of the event occurring at time , given that the subject has survived up to that time. It answers the question: "If a subject has made it this far, what is the risk the event happens right now?"
Unlike the survival function, the hazard function can go up, down, or stay constant. For example, the risk of a car part failing might increase with age. Conversely, the risk of post-surgery complications might be very high immediately after the operation and then decrease over time as the patient recovers.
Understanding these core ideas—time-to-event data, censoring, and the survival and hazard functions—is the first step into the powerful world of survival analysis. These concepts allow us to properly handle incomplete data and draw meaningful conclusions about how long things last.