Life is full of uncertainty. Will it rain tomorrow? Will my favorite team win the championship? Will I roll a six? Probability is the branch of mathematics that gives us a framework for dealing with this uncertainty. It's a way of putting a number on how likely something is to happen.

At the heart of probability is the idea of an experiment, which is any process with an uncertain outcome. Flipping a coin, rolling a die, or measuring the daily rainfall are all experiments. The result of such an experiment is called an outcome.

To talk about these outcomes mathematically, we use something called a random variable.

A random variable, often represented by a capital letter like $X$, links the outcomes of an experiment to a numerical value. For example, if we roll a six-sided die, we can let the random variable $X$ be the number that comes up. The possible values for $X$ are 1, 2, 3, 4, 5, and 6.

Random variables come in two main flavors: discrete and continuous.

Discrete random variables can only take on a countable number of distinct values. Think of things you can count. The number of heads in three coin flips (0, 1, 2, or 3), the number you roll on a die (1, 2, 3, 4, 5, 6), or the number of cars that pass through an intersection in an hour are all discrete.

Continuous random variables, on the other hand, can take on any value within a given range. These are things you measure, not count. A person's height, the temperature of a room, or the exact time it takes to run a race are continuous. The height could be 1.75 meters, 1.751 meters, or even 1.7512 meters, depending on how precisely you measure it. There's an infinite number of possibilities within the range.

So we have a random variable. Now what? We want to know the probability of each of its possible values. That’s where a probability distribution comes in. A probability distribution is a function or a table that tells us the probability for each outcome of a random variable.

For a discrete variable, this is often called a Probability Mass Function (PMF). Let’s look at the PMF for our fair six-sided die. Since the die is fair, each of the six outcomes is equally likely.

Notice that if you add up all the probabilities, you get 1. This is always true for any probability distribution. The total probability of all possible outcomes must equal 1, because one of those outcomes must happen.

For continuous variables, the concept is similar, but we use a Probability Density Function (PDF). Instead of giving the probability of a specific outcome (which is technically zero for a continuous variable), the PDF tells us about the likelihood of an outcome falling within a certain range. The total area under the curve of a PDF is always 1.

Sometimes, it's useful to have a single number that summarizes the typical or average outcome of a random variable. This is called the expected value.

The expected value is a weighted average of all possible values, where the weights are the probabilities. You can think of it as the average result you'd get if you performed the experiment over and over again, an infinite number of times.

The formula for the expected value of a discrete random variable $X$, written as $E[X]$, is:

This means you take each possible value $x_i$, multiply it by its probability $P(x_i)$, and then add up all those products.

Let’s calculate the expected value for our die roll:

The expected value is 3.5. Of course, you can't actually roll a 3.5 on a die. That’s okay. The expected value doesn't have to be a possible outcome. It just represents the long-term average.

Now that you've got the basics down, let's test your knowledge.

These concepts—random variables, distributions, and expected values—are the fundamental building blocks for understanding more complex ideas in probability and information theory.