Probability is a way to measure how likely something is to happen. It's a number between 0 and 1. If the probability of an event is 0, it’s impossible. If it's 1, it's a sure thing. Everything else falls somewhere in between.

There are a few ways to think about where these numbers come from.

The classical approach is what we use for games of chance. When you flip a fair coin, there are two equally likely outcomes: heads or tails. The probability of getting heads is 1 out of 2, or 0.5.

The relative frequency approach comes from data. If you want to know the probability that a bus is late, you could track it for 100 days. If it’s late on 15 of those days, you could estimate the probability of it being late as 15/100, or 0.15.

Finally, there's the subjective approach. This is about personal belief. An expert might say there's an 80% chance a certain startup will succeed based on their experience and intuition. It's not based on a perfect mathematical model, but on informed judgment.

To calculate probabilities, we first need to know all the possible outcomes. This complete set of possibilities is called the sample space.

For a single six-sided die, the sample space is {1, 2, 3, 4, 5, 6}. For a coin flip, it's {Heads, Tails}. It's a simple list of everything that could happen.

An event is any specific outcome or group of outcomes you're interested in. It's a subset of the sample space. For example, rolling an even number on a die is an event. This event corresponds to the outcomes {2, 4, 6}.

Probability follows three fundamental rules, often called axioms. They're quite intuitive.

1. Probabilities are always between 0 and 1. A probability can't be negative, and it can't be greater than 100%.
2. The probability of the entire sample space is 1. If you roll a die, you are guaranteed to get a number between 1 and 6. The chance of something happening is 100%.
3. If two events can't happen at the same time, the probability that one or the other occurs is the sum of their individual probabilities. For example, you can't roll a 1 and a 3 on a single die at the same time. The probability of rolling a 1 is 1/6, and the probability of rolling a 3 is 1/6. The probability of rolling a 1 or a 3 is simply 1/6 + 1/6 = 2/6.

This addition rule applies only when events are mutually exclusive, meaning they have no outcomes in common.

Sometimes, knowing one thing has happened changes the probability of something else. This is called conditional probability.

Let's go back to our die. The probability of rolling a 4 is 1/6. But what if I told you the number rolled was even? Now the sample space has shrunk. We know the outcome must be in the set {2, 4, 6}. Given this new information, the probability of it being a 4 is now 1 out of 3. We write this as $P(\text{roll is 4} \mid \text{roll is even}) = 1/3$. The vertical bar "|" means "given that".

This idea leads us to one of the most important formulas in probability: Bayes' theorem. It provides a formal way to update our beliefs in light of new evidence.

The formula looks like this:

Let's break that down with an example.

Imagine a medical test for a rare disease. Let's say the disease affects 1% of the population. The test is 99% accurate: if you have the disease, it will be positive 99% of the time. And if you don't have the disease, it will be negative 99% of the time.

So, you take the test and it comes back positive. What's the probability you actually have the disease?

It's not 99%! Let's use Bayes' theorem.

• $A$ is "having the disease." $P(A) = 0.01$.
• $B$ is "testing positive."
• We want to find $P(A|B)$.
• We know $P(B|A) = 0.99$ (the test's accuracy for sick people).
• What about $P(B)$? A person can test positive in two ways: they have the disease and test positive, or they don't have the disease and get a false positive. The chance of a false positive is 1% (since the test is 99% accurate for healthy people). The chance of not having the disease is 99%. So, $P(\text{false positive}) = 0.99 \times 0.01$. The chance of a true positive is $0.01 \times 0.99$. So, $P(B) = (0.01 \times 0.99) + (0.99 \times 0.01) = 0.0198$.

Now we plug everything into the formula:

Even with a positive test, there's only a 50% chance you have the disease! This is because the disease is so rare that false positives, while infrequent, make up a large portion of the total positive results. Bayes' theorem helps us see this clearly and avoid jumping to the wrong conclusion.

Let's check your understanding of these core concepts.

These foundational ideas are the building blocks for understanding a world filled with uncertainty.