Introduction to MGFs and CFs

Functions That Define Distributions

In probability, we often work with probability density functions (PDFs) or probability mass functions (PMFs) to describe a random variable. But there are other tools that can capture all the information about a distribution in a single, compact function. These are the moment-generating function (MGF) and the characteristic function (CF).

Think of them as a unique fingerprint for a probability distribution. If two random variables have the same MGF or CF, they must have the same distribution. This property makes them incredibly useful for identifying distributions and proving theorems.

The Moment-Generating Function

The moment-generating function, as its name suggests, is used to find the moments of a random variable. A "moment" is just a specific quantitative measure of the shape of the distribution. For example, the first moment is the mean (the expected value), and the second central moment is the variance.

Moment-Generating Function

noun

A function M_X(t) that captures all the moments of a random variable X. It's defined as the expected value of e^(tX).

M_X(t) = E[e^{tX}]

For a continuous random variable with a PDF $f(x)$ , the MGF is found by an integral. For a discrete random variable with a PMF $p(x)$ , it's found by a summation.

\begin{aligned} \\ M_X(t) &= \int_{-\infty}^{\infty} e^{tx} f(x) \, dx \quad \text{(Continuous)} \\ M_X(t) &= \sum_{i} e^{tx_i} p(x_i) \quad \text{(Discrete)} \\ \end{aligned}

The key property of the MGF is that the $n$ -th derivative with respect to $t$ , evaluated at $t=0$ , gives the $n$ -th moment of $X$ .

E[X^n] = \frac{d^n}{dt^n} M_X(t) \bigg|_{t=0}

A major limitation of the MGF is that it doesn't always exist. For some distributions, the integral or sum that defines it might not converge to a finite number for any $t$ other than $t=0$ . This is where the characteristic function comes in.

The Characteristic Function

The characteristic function (CF) is closely related to the MGF and serves a similar purpose. However, it has one huge advantage: it always exists for any random variable.

This is because it uses a complex exponential instead of a real one. The complex exponential $e^{itX}$ involves Euler's formula ( $e^{i\theta} = \cos(\theta) + i\sin(\theta)$ ), whose components, sine and cosine, are bounded between -1 and 1. This bounding ensures the expectation always converges.

\phi_X(t) = E[e^{itX}] = E[\cos(tX) + i\sin(tX)]

Like the MGF, the CF can be used to find moments, though the process involves the imaginary unit $i$ .

E[X^n] = \frac{1}{i^n} \frac{d^n}{dt^n} \phi_X(t) \bigg|_{t=0}

Both MGFs and CFs share another powerful property. If $X$ and $Y$ are independent random variables, the MGF (or CF) of their sum $Z = X+Y$ is simply the product of their individual MGFs (or CFs).

\begin{aligned} \\ M_{X+Y}(t) &= M_X(t)M_Y(t) \\ \phi_{X+Y}(t) &= \phi_X(t)\phi_Y(t) \\ \end{aligned}

A Comparison

Let's compare these two functions side-by-side.

Feature	Moment-Generating Function (MGF)	Characteristic Function (CF)
Definition	$M_X(t) = E[e^{tX}]$	$\phi_X(t) = E[e^{itX}]$
Existence	Not guaranteed to exist	Always exists
Uniqueness	Uniquely determines the distribution	Uniquely determines the distribution
Sum of Ind. Vars	$M_{X+Y}(t) = M_X(t)M_Y(t)$	$\phi_{X+Y}(t) = \phi_X(t)\phi_Y(t)$

Because the characteristic function is more general, it is the preferred tool in advanced probability theory. However, when the MGF exists, it is often easier to work with because it avoids complex numbers.

Let's test your understanding of these new concepts.

Quiz Questions 1/5

What is the primary reason that moment-generating functions (MGFs) and characteristic functions (CFs) are considered unique "fingerprints" for a probability distribution?

Quiz Questions 2/5

A major limitation of the moment-generating function (MGF) is that it does not always exist for every random variable.

Advanced MGFs and CFs in Finance