No history yet

Introduction to Probability

The Language of Chance

Probability is the mathematics of uncertainty. It gives us a way to talk about and measure how likely something is to happen. When we can't know an outcome for sure, we can still analyze the possibilities. The complete set of possible outcomes and their associated likelihoods is called a probability distribution.

Probability Distribution

noun

A mathematical function that lists all possible outcomes of an experiment and their corresponding probabilities. The sum of all probabilities in a distribution must equal 1.

For a simple event like rolling a six-sided die, the distribution is straightforward. There are six possible outcomes (1, 2, 3, 4, 5, 6), and each has an equal chance of occurring. So, the probability for each is $1/6$. This is an example of a discrete probability distribution, where the outcomes are distinct, countable values.

Summarizing Possibilities

A full distribution tells us everything about the probabilities of an event, but sometimes we need a simpler summary. Two of the most important summary metrics are the expected value and the variance.

The expected value tells us the long-term average outcome we should anticipate. It’s like a forecast for a random event.

To calculate the expected value, you multiply each possible outcome by its probability and then sum all those values. The formula for a discrete random variable XX is written as:

E[X]=∑ixiP(xi)E[X] = \sum_{i} x_i P(x_i)

Let’s find the expected value for our six-sided die roll. We multiply each outcome (xix_i) by its probability (P(xi)=1/6P(x_i) = 1/6) and add them up.

E[X]=1(16)+2(16)+3(16)+4(16)+5(16)+6(16)=1+2+3+4+5+66=216=3.5\begin{aligned}E[X] &= 1(\frac{1}{6}) + 2(\frac{1}{6}) + 3(\frac{1}{6}) \\ & \quad + 4(\frac{1}{6}) + 5(\frac{1}{6}) + 6(\frac{1}{6}) \\ &= \frac{1+2+3+4+5+6}{6} = \frac{21}{6} = 3.5\end{aligned}

Of course, you can't actually roll a 3.5. The expected value isn't necessarily a possible outcome. Instead, it's the average result you would expect to see if you rolled the die many, many times.

While the expected value gives us the center of a distribution, the variance tells us how spread out the outcomes are. A low variance means the outcomes are clustered tightly around the expected value. A high variance means they are scattered far and wide.

Variance measures the average squared difference of each possible outcome from the expected value. It quantifies the amount of risk or unpredictability.

The formula for variance is:

Var(X)=∑i(xi−E[X])2P(xi)Var(X) = \sum_{i} (x_i - E[X])^2 P(x_i)

For the die roll, with E[X]=3.5E[X] = 3.5:

Var(X)=(1−3.5)2(16)+(2−3.5)2(16)+...+(6−3.5)2(16)=16((−2.5)2+(−1.5)2+(−0.5)2+(0.5)2+(1.5)2+(2.5)2)=16(6.25+2.25+0.25+0.25+2.25+6.25)=17.56≈2.917\begin{aligned}Var(X) &= (1-3.5)^2(\frac{1}{6}) + (2-3.5)^2(\frac{1}{6}) + ... \\ &\quad + (6-3.5)^2(\frac{1}{6}) \\ &= \frac{1}{6}((-2.5)^2 + (-1.5)^2 + (-0.5)^2 \\ &\quad + (0.5)^2 + (1.5)^2 + (2.5)^2) \\ &= \frac{1}{6}(6.25+2.25+0.25+0.25+2.25+6.25) \\ &= \frac{17.5}{6} \approx 2.917 \end{aligned}

A higher variance would mean the outcomes are less predictable. For example, a game with payouts of $0 or $100 has a much higher variance than a game with payouts of $49 or $51, even if both have the same expected value.

The Law of Large Numbers

The concepts of expected value and variance are tied together by a powerful idea: the law of large numbers. This principle states that as you repeat an experiment a large number of times, the average of the results will get closer and closer to the expected value.

Think of flipping a coin. If you flip it 10 times, you might get 7 heads (70%). But if you flip it 10,000 times, the proportion of heads will be extremely close to the expected value of 50%.

Lesson image

The law of large numbers is the reason casinos and insurance companies are profitable. They engage in a massive number of independent events (bets placed, policies sold). While they might lose money on any single event, they can be confident that over millions of trials, their average results will closely match the calculated expected value, ensuring their business model works.

Now, let's test your understanding of these core concepts.

Quiz Questions 1/6

What does a probability distribution describe?

Quiz Questions 2/6

You are playing a game where you flip a coin. If it's heads, you win 10.Ifit′stails,youlose10. If it's tails, you lose 4. What is the expected value of one coin flip?

These foundational ideas—distributions, expected value, variance, and the law of large numbers—are the building blocks for understanding more complex systems involving randomness.