Probabilistic Foundations of Deep Learning for Mathematicians
Measure Theoretic Probability
From Measure to Probability
You're familiar with the general concept of a measure space from real analysis. It's a triplet (X, oldsymbol{rak{A}}, oldsymbol{eta}) consisting of a set, a -algebra of its subsets, and a measure. To formalize probability, we simply specialize this idea.
A probability space is a measure space where the total measure of the space is 1. We use slightly different notation to reflect the new context, but the underlying structure is identical.
An event is simply a measurable subset of the sample space. The measure for an event A oldsymbol{\in} \mathcal{F} is the probability of that event occurring. Because , we know the probability of some outcome happening is certain, which aligns perfectly with our intuition.
Random Variables Reimagined
In elementary probability, a random variable is often loosely described as a variable whose value is a numerical outcome of a random phenomenon. Measure theory gives us a precise definition: a random variable is a measurable function.
Specifically, a real-valued random variable is a function that maps outcomes from the sample space to the real numbers, . For to be a random variable, it must be -measurable. This means that for any Borel set on the real line, its pre-image under must be an event in our -algebra .
This measurability condition is crucial. It guarantees that questions like "What is the probability that is between and ?" are well-posed, because the set of outcomes where is an event in and thus has a defined probability under .