Probability is a way to measure uncertainty. It tells us how likely an event is to happen, on a scale from 0 to 1. A probability of 0 means the event is impossible, while a probability of 1 means it's absolutely certain. Most things in life fall somewhere in between.

To calculate probability, we first need to understand the sample space. This is just a list of all possible outcomes of an experiment. For a single six-sided die, the sample space is all the numbers that can come up.

$S = \{1, 2, 3, 4, 5, 6\}$

An event is a specific outcome or a set of outcomes we're interested in. Let's say we want to know the probability of rolling an even number. Our event, which we can call $A$, is the set of even outcomes.

The probability of event $A$ is the number of outcomes in $A$ divided by the total number of outcomes in the sample space $S$.

Probability isn't just guesswork; it follows three strict rules, called axioms. These are the foundations upon which everything else is built.

Think back to our die. The event of rolling a 1 and the event of rolling a 6 are mutually exclusive. You can't do both on a single roll. The probability of rolling a 1 is $1/6$, and the probability of rolling a 6 is $1/6$. So, the probability of rolling a 1 or a 6 is:

$P(1 \text{ or } 6) = P(1) + P(6) = \frac{1}{6} + \frac{1}{6} = \frac{2}{6} = \frac{1}{3}$

Probabilities can change as we learn more. This is the idea behind conditional probability. It asks, what is the probability of an event $A$ happening, given that we know another event $B$ has already happened? We write this as $P(A|B)$, which is read as "the probability of A given B."

The formula for conditional probability is:

In this formula, $P(A \text{ and } B)$ is the probability that both events $A$ and $B$ happen. Let's use our die example again. What is the probability of rolling an even number, given that we know the roll was greater than 3?

• Event $A$: The roll is even. $A = \{2, 4, 6\}$
• Event $B$: The roll is greater than 3. $B = \{4, 5, 6\}$

First, we need the probability of $B$. There are 3 outcomes in $B$ out of 6 total, so $P(B) = 3/6 = 0.5$.

Next, we need the probability of $A$ and $B$ happening together. This means the roll is both even and greater than 3. The outcomes that satisfy this are $A \text{ and } B = \{4, 6\}$. There are 2 such outcomes, so $P(A \text{ and } B) = 2/6 = 1/3$.

Now we can find the conditional probability.

So, once we know the roll was greater than 3, the probability of it being even increases from $0.5$ to about $0.67$.

Conditional probability leads directly to one of the most important ideas in this field: Bayes' Theorem. This theorem gives us a way to formally update our beliefs in light of new evidence. It's the engine behind many machine learning algorithms and scientific studies.

The theorem connects $P(A|B)$ with $P(B|A)$.

You take the test and it comes back positive. What's the probability you actually have the disease?

Let's define our events:

• $A$: You have the disease. $P(A) = 0.01$.
• $B$: You test positive.

We want to find $P(A|B)$. Using Bayes' Theorem, we need to find the other pieces:

• $P(B|A)$: The probability of testing positive given you have the disease. This is the test's accuracy. $P(B|A) = 0.99$.
• $P(B)$: The overall probability of testing positive. This is where it gets tricky. A person can test positive in two ways: having the disease and testing positive (true positive), or not having it and testing positive (false positive).

This calculation uses the Law of Total Probability. It says we can find the probability of an event by summing the probabilities of its different scenarios.

Now we have everything for Bayes' Theorem.

Surprisingly, even with a positive result from a 99% accurate test, you only have about a 50% chance of actually having the disease. This counterintuitive result happens because the disease is so rare. The small number of false positives from the large healthy population is about equal to the number of true positives from the small sick population.

Now, let's check your understanding of these core concepts.