So far, we've used regression to predict continuous values, like the price of a house or the temperature tomorrow. The goal is to draw a line (or a curve) that best fits the data. But what if the outcome isn't a number on a spectrum? What if it's a simple yes or no?

This is where classification comes in. Instead of predicting a quantity, we're predicting a category or a label. Will a customer churn or not? Is this email spam or not spam? Is a tumor malignant or benign? These are all classification problems with two possible outcomes, known as binary classification.

You might wonder why we can't just use linear regression for this. It's a fair question. We could label one outcome as 0 (e.g., "not spam") and the other as 1 (e.g., "spam") and try to fit a line. The problem is that a straight line can easily predict values below 0 or above 1, which makes no sense for probabilities. We need a model that's built specifically for this kind of yes/no question. That model is logistic regression.

Logistic regression works by taking the output of a linear equation and feeding it into a special function called the logistic function, or sigmoid function. The magic of this function is that it can take any number, from negative infinity to positive infinity, and squeeze it into a value between 0 and 1.

This S-shaped curve is perfect for representing probability. If the output of our linear equation is a large positive number, the logistic function gives a value close to 1. If it's a large negative number, the function gives a value close to 0. A value of 0 from the linear equation results in a probability of exactly 0.5.

Here, $P(Y=1)$ is the probability of the outcome being '1' (e.g., spam). The variable $z$ is the familiar linear combination of our input features: $z = \beta_0 + \beta_1x_1 + \beta_2x_2 + \dots$. The model learns the best values for the coefficients (the $\beta$ values) to map the inputs to the correct probability.

While probabilities are intuitive, they have a tricky, non-linear relationship with our input variables. To make the math work out nicely, logistic regression uses a few transformations. It all starts with probability.

The first step is to convert this probability into odds. Odds represent the ratio of the probability of an event happening to the probability of it not happening.

Odds are useful, but they still aren't linear. The final step is to take the natural logarithm of the odds. This gives us the log-odds, also known as the logit.

This transformation is the key. The log-odds has a direct, linear relationship with our input variables. In fact, the log-odds is exactly what our linear equation, $z$, calculates.

So, logistic regression is essentially a linear model for the log-odds. By modeling the log-odds, we can use the logistic function to back-calculate the probability and make a final, categorical prediction.

Let's test your understanding of these core ideas.

Understanding this progression from probability to log-odds provides the foundation for interpreting how logistic regression makes its decisions.