Naive Bayes Explained Simply
Introduction to Probability
The Language of Chance
Probability is simply a way to measure the chance that something will happen. We use it every day, whether we're checking the weather forecast or just guessing how long a traffic light will stay green. It gives us a framework for thinking about uncertainty.
At its heart, probability is a ratio. It compares the number of ways a specific outcome can occur to the total number of possible outcomes.
Let's take a simple example: rolling a standard six-sided die. The set of all possible outcomes, called the sample space, is {1, 2, 3, 4, 5, 6}. There are six total outcomes.
What's the probability of rolling a 4? There's only one way for that to happen. So, the probability is:
Probabilities are always expressed as a number between 0 and 1. A probability of 0 means the event is impossible (like rolling a 7 on a six-sided die). A probability of 1 means the event is certain (like rolling a number less than 7).
When Information Changes Things
Sometimes, we get new information that changes the odds. This is where the idea of conditional probability comes in. It's the probability of an event happening, given that another event has already occurred.
Imagine a standard deck of 52 playing cards. The probability of drawing a King is:
But what if a friend draws a card, peeks at it, and tells you, "The card is a face card"? (A face card is a King, Queen, or Jack). Now, what is the probability that the card is a King?
Our sample space has shrunk. We're no longer considering all 52 cards. We're only looking at the 12 face cards (4 Kings, 4 Queens, 4 Jacks). Within this new, smaller sample space, there are 4 Kings.
The vertical bar "|" is read as "given". So, means "the probability of A happening, given that B has happened." The new information dramatically increased the probability of the card being a King.
Conditional probability is about updating our beliefs in light of new evidence. The denominator, or our total set of possibilities, changes.
Independent Events
What if one event has absolutely no effect on another? These are called independent events. Flipping a coin is the classic example. If you flip a coin and get heads, what's the probability of getting heads on the next flip? It's still $1/2$. The first flip doesn't influence the second one.
Two events, A and B, are independent if knowing the outcome of one doesn't change the probability of the other. Formally, A and B are independent if:
Let's go back to our deck of cards. Let's define two events:
- Event A: Drawing a Heart.
- Event B: Drawing a King.
The probability of drawing a Heart is .
Now, let's find the conditional probability of drawing a Heart given that we drew a King, or . Our new sample space is the 4 Kings. Within that set, one of them is a Heart (the King of Hearts).
Since , the two events are independent. Knowing a card is a King doesn't change the likelihood that it's a Heart, and vice versa. This makes intuitive sense, as each suit has one King.
On the other hand, drawing a King and drawing a face card are not independent. As we saw earlier, knowing a card is a face card makes it much more likely to be a King.
Understanding these basic rules of probability—how to calculate chances, how new information refines them, and how to know when they don't affect each other—is the foundation for working with more complex models of uncertainty.