Linear algebra operates within a structured environment called a vector space. You're already familiar with vectors as arrows in space or lists of numbers. A vector space is the set of all possible vectors in a given context, along with two key rules: you can add any two vectors in the space and the result will still be in the space, and you can multiply any vector by a scalar and it will also remain in the space. These properties are known as closure under addition and scalar multiplication.

A subspace is simply a vector space that lives inside a larger vector space. For example, a line passing through the origin is a subspace of the 2D plane. A plane passing through the origin is a subspace of 3D space. The key is that it must contain the zero vector and be closed under addition and scalar multiplication on its own.

In machine learning, high-dimensional datasets are treated as vectors within a vector space. A subspace might represent a simplified, lower-dimensional version of the data that captures its most important features. In quantum computing, the possible states of a quantum system form a complex vector space, and operations on qubits are transformations within that space.

Linear transformations, represented by matrices, can stretch, shrink, rotate, or shear vectors. Amidst all this change, some vectors are special. When a matrix acts on them, they don't change their direction; they only change their length. These special vectors are called eigenvectors.

The scalar factor by which an eigenvector is stretched or shrunk is its corresponding eigenvalue. The relationship is captured by a foundational equation in linear algebra.

Think of a transformation that stretches everything horizontally. Any horizontal vector is an eigenvector because its direction doesn't change, it just gets longer. Its eigenvalue would be the stretch factor. A vertical vector would also be an eigenvector, but its eigenvalue would be 1 since it wasn't stretched.

Eigenvectors and eigenvalues are critical for understanding the behavior of a matrix. In machine learning, the algorithm for Principal Component Analysis (PCA) finds the eigenvectors of a dataset's covariance matrix to identify the directions of greatest variance. In quantum mechanics, physical properties like energy or spin are represented by operators (matrices). The possible results of a measurement are the eigenvalues of the operator, and the state of the system after measurement is the corresponding eigenvector.

Just as we can factor a number into its prime components (like 12 = 2 × 2 × 3), we can decompose a complex matrix into simpler, more fundamental matrices. This process, called matrix decomposition or factorization, simplifies complex problems and reveals underlying properties of the transformation the matrix represents.

One of the most important decompositions is the Singular Value Decomposition (SVD). SVD asserts that any matrix $A$ can be factored into three other matrices.

SVD is incredibly versatile. It provides a geometrically intuitive breakdown of a transformation into a rotation ($V^T$), a scaling along axes ($\Sigma$), and another rotation ($U$). It's used extensively in machine learning for dimensionality reduction, collaborative filtering in recommendation systems (like those used by Netflix), and image compression. By keeping only the largest singular values in $\Sigma$, you can create a lower-rank approximation of the original matrix that preserves its most significant information.

While vectors and matrices are powerful, sometimes we need to combine vector spaces to describe more complex systems. This is where the tensor product comes in. The tensor product, denoted by the symbol $\otimes$, is a way to create a larger vector space from two smaller ones.

If you have a vector $v$ from space $V$ and a vector $w$ from space $W$, their tensor product $v \otimes w$ is an element in a new, larger space $V \otimes W$. For example, the tensor product of two 2-dimensional vectors results in a 4-dimensional vector.

This operation is fundamental to quantum computing. The state of a single qubit can be described by a 2D vector. To describe a system of two qubits, you don't just add their vectors—you take their tensor product. The result is a 4D vector that lives in a combined state space. This mathematical structure is what allows for quantum phenomena like entanglement, where the state of the combined system contains more information than the states of the individual parts.

In deep learning, tensors (which are the multi-dimensional arrays you often hear about) are the primary data structures. While not always explicitly using the tensor product operation in the same way as quantum mechanics, the concept of multi-dimensional arrays that can be transformed and manipulated is central to how neural networks process complex data like images and video.

These advanced concepts form the mathematical language that powers many of the most sophisticated algorithms in AI and quantum computing.