In linear algebra, we work with vectors as arrows in space, like points in $\mathbb{R}^2$ or $\mathbb{R}^3$. We can add them, scale them, and measure their lengths. Functional analysis takes these ideas and applies them to more abstract objects: functions. Here, an entire function, with its infinite collection of points, is treated as a single vector in a vast, infinite-dimensional space. To navigate these new spaces, we first need a way to measure the "size" or "length" of these function-vectors.

This is where the concept of a norm comes in. A normed vector space is a vector space equipped with a norm, which is a function that assigns a strictly positive length to each vector. A norm, denoted by $\|x\|$, must satisfy three basic properties:

Consider the space of all continuous functions on the interval $[0, 1]$, which we call $C[0, 1]$. One way to define a norm for a function $f(x)$ in this space is the supremum norm, which finds the function's peak absolute value.

In a normed space, we can talk about sequences of vectors getting closer. A Cauchy sequence is one where the vectors get arbitrarily close to each other as the sequence progresses. We expect such sequences to converge to a limit. But what if the limit isn't in the space? For example, a sequence of rational numbers can converge to an irrational number like $\sqrt{2}$, which isn't in the space of rational numbers.

A space is complete if every Cauchy sequence in it converges to a limit that is also in the space. There are no "holes."

The space $C[0, 1]$ with the supremum norm is a Banach space. If you have a Cauchy sequence of continuous functions, their limit will also be a continuous function. This property of completeness is crucial for many proofs and applications, especially in solving differential equations.

Banach spaces give us length, but what about angles and orthogonality (perpendicularity)? For that, we need an extra piece of structure: an inner product.

An inner product, denoted $\langle x, y \rangle$, takes two vectors and produces a scalar. It generalizes the dot product from Euclidean space. With an inner product, we can define a norm naturally.

This lets us define the angle $\theta$ between two vectors, and say that two vectors are orthogonal if their inner product is zero. A complete inner product space is called a Hilbert space.

All Hilbert spaces are Banach spaces, but not all Banach spaces are Hilbert spaces. The extra geometric structure of Hilbert spaces makes them particularly useful. For instance, they allow us to use tools like Fourier series, which breaks down complex functions into a sum of simpler, orthogonal sine and cosine functions. This is the foundation of modern signal processing.

Finally, we want to see how these vector spaces interact. A linear operator is a function $T$ that maps vectors from one vector space $V$ to another, $W$, while preserving the underlying structure. Specifically, it must satisfy two conditions for all vectors $x, y$ in $V$ and any scalar $\alpha$:

Linear operators are the functions of functional analysis. They can be thought of as transformations, like rotations and scaling in $\mathbb{R}^2$, but they can also represent more complex actions.

Functional analysis studies these operators, asking questions like: Is an operator continuous? Does it have an inverse? What are its eigenvalues? The answers to these questions have profound implications in fields ranging from physics, where operators represent physical observables in quantum mechanics, to engineering, where they model systems that transform signals.

By building this framework of normed spaces, Banach spaces, Hilbert spaces, and linear operators, functional analysis provides a powerful language to describe and solve problems involving infinite-dimensional systems.

Time to test your understanding of these foundational ideas.