Introduction to Functional Analysis

Beyond Finite Dimensions

In linear algebra, you worked with vector spaces like $\mathbb{R}^2$ and $\mathbb{R}^3$ . These are comfortable, finite-dimensional worlds where vectors are just arrows with a length and direction. Functional analysis takes these ideas and applies them to spaces that are infinitely large. In these spaces, the "vectors" are often functions.

Imagine the set of all continuous functions you can draw on the interval from 0 to 1. This set forms a vector space. You can add two functions together, or multiply a function by a scalar, and you'll still get a continuous function. But this space is infinite-dimensional. You can't describe every function in it by just a handful of basis vectors. Functional analysis gives us the tools to work with these enormous spaces.

Functional analysis is essentially infinite-dimensional linear algebra.

Measuring Functions

To talk about things like distance and convergence, we first need a way to measure the "size" or "length" of our vectors. In functional analysis, this concept is called a norm. A norm is a function that assigns a strictly positive length to each vector, except for the zero vector, which has a length of zero.

norm

noun

A function that assigns a positive length or size to each vector in a vector space.

A vector space equipped with a norm is called a normed vector space. Any valid norm, denoted as $\|x\|$ , must satisfy three key properties for any vectors $x, y$ and scalar $\alpha$ :

\begin{aligned} \\ &1. \|x\| \ge 0 \quad (\text{and } \|x\| = 0 \iff x = 0) \\ &2. \|\alpha x\| = |\alpha| \|x\| \\ &3. \|x + y\| \le \|x\| + \|y\| \\ \end{aligned}

Let's consider our vector space of continuous functions on the interval $[a, b]$ , denoted $C[a, b]$ . A common way to define a norm for a function $f(x)$ in this space is the supremum norm, which is just the maximum absolute value the function reaches on the interval.

\|f\|_{\infty} = \sup_{x \in [a, b]} |f(x)|

Complete Spaces

Having a norm allows us to talk about sequences of functions getting closer to each other. A sequence where the elements get arbitrarily close together is called a Cauchy sequence. You might assume that if the terms of a sequence are getting closer and closer, they must be converging to something. But that something isn't guaranteed to be within the space you're working in.

Think about the rational numbers. The sequence 3, 3.1, 3.14, 3.141, ... is a Cauchy sequence of rational numbers. The terms are bunching up. But they converge to $\pi$ , which is an irrational number. The sequence's limit is not in the space of rational numbers.

A space is called complete if every Cauchy sequence in it converges to a limit that is also in that space. The real numbers are complete, which is why they form a continuous line with no holes.

A normed vector space that is also complete is called a Banach space. These spaces are incredibly useful because they guarantee that the limits of our sequences of functions will themselves be functions within the same space. The space $C[a, b]$ with the supremum norm is a Banach space.

A Banach space is a complete normed vector space. It's a place where sequences that should converge actually do.

Things get even more interesting when we add more structure. A Hilbert space is a special type of Banach space where the norm comes from an inner product. The inner product is a generalization of the dot product from Euclidean space. It lets us talk about angles between vectors, and specifically, orthogonality (when vectors are perpendicular).

For functions, an inner product might look like this:

\langle f, g \rangle = \int_{a}^{b} f(x) g(x) \, dx

The norm is then defined from this inner product:

\|f\| = \sqrt{\langle f, f \rangle} = \left( \int_{a}^{b} f(x)^2 \, dx \right)^{1/2}

Hilbert spaces are the main setting for quantum mechanics and are fundamental in signal processing and the study of partial differential equations. They provide a rich geometric structure for infinite-dimensional spaces.

Functions of Functions

The final piece of our foundation is the linear operator. In linear algebra, you studied matrices, which are linear transformations that take a vector and map it to another vector. A linear operator does the same thing, but in function spaces. It's a function that takes a function as input and produces another function as output.

An operator $L$ is linear if it satisfies two conditions for any vectors (functions) $f, g$ and any scalar $\alpha$ :

\begin{aligned} \\ L(f+g) &= L(f) + L(g) \\ L(\alpha f) &= \alpha L(f) \\ \end{aligned}

A classic example of a linear operator is the differentiation operator, $D = \frac{d}{dx}$ . If you apply it to a function $f(x)$ , you get a new function, its derivative $f'(x)$ . You can verify that differentiation is a linear operation:

$D(f+g) = (f+g)' = f' + g' = D(f) + D(g)$ $D(\alpha f) = (\alpha f)' = \alpha f' = \alpha D(f)$

Integration is another common linear operator. These concepts—normed spaces, complete spaces, and operators—form the language of functional analysis. They allow us to apply the powerful geometric intuition of linear algebra to solve problems involving functions, like differential equations.

Let's check your understanding of these foundational ideas.

Quiz Questions 1/6

What is the primary conceptual shift when moving from linear algebra to functional analysis?

Quiz Questions 2/6

Which of the following is NOT a required property for a function to be a norm, denoted $\Vert x \Vert$ ?

With these building blocks, we can begin to explore how functional analysis provides a powerful framework for solving complex mathematical problems.

Functional Analysis for ODEs