Imagine you want to predict a number, like the price of a house or the temperature tomorrow. Linear regression is a great tool for that. It draws the best-fitting straight line through your data to make predictions.

But what if you don't want to predict a number? What if you want to predict an answer to a yes-or-no question? For example:

• Is this email spam or not spam?
• Will this customer click on the ad or not?
• Is this tumor malignant or benign?

These are all binary outcomes. They have only two possible results. A straight line doesn't really work here. If we try to fit a line, it can give us predictions like 1.5 or -0.2, which don't make sense as probabilities. Probabilities have to be between 0 and 1.

Instead of a straight line, logistic regression uses a special S-shaped curve to make its predictions. This curve ensures that the output is always a probability between 0 and 1.

The core of logistic regression is the logistic function, also known as the sigmoid function. It's a neat mathematical trick that takes any real number and squeezes it into the range between 0 and 1.

This S-shaped curve is perfect for modeling probability. When the input value is very large and positive, the function gets close to 1. When it's very large and negative, it gets close to 0. Right in the middle, at an input of 0, the function outputs 0.5, representing a 50/50 chance.

The formula for this function looks like this, where $z$ is the input from our linear model (more on that later):

This equation elegantly turns a linear combination of inputs into a sensible probability. The model calculates a value $z$, and the logistic function converts that value into the probability of the event happening.

To really understand how logistic regression works, we need to talk about odds. You've probably heard the term in betting or sports. It's just another way of expressing the likelihood of an event.

Probability is the chance of an event happening divided by the total number of outcomes. If the probability of winning is 0.8 (or 80%), the probability of losing is 0.2 (or 20%).

So, if the probability of winning is 0.8, the odds are:

This means the event is 4 times more likely to happen than not happen. We can say the odds are 4 to 1.

Logistic regression actually works with something one step further: the log-odds. This is simply the natural logarithm of the odds.

This might seem complicated, but it's what allows the model to work. The model calculates the log-odds as a straight-line equation, and then the logistic function converts those log-odds back into a probability between 0 and 1.

Because it's great at answering yes/no questions, logistic regression is used everywhere.

In medicine, it can help predict the likelihood of a patient having a certain disease based on their symptoms and lab results. Banks use it to determine whether a loan applicant is likely to default or not. Tech companies use it to predict if a user will churn (cancel their subscription) or to filter out spam from your inbox. It’s a versatile and powerful tool for binary classification problems.

Now let's test your understanding of these core concepts.

Logistic regression provides a solid foundation for understanding many other classification models in machine learning.