At its core, linear algebra is the study of vectors and the rules for manipulating them. In machine learning, we use vectors to represent data. A vector is just an ordered list of numbers. You could represent a house with a vector containing its square footage, number of bedrooms, and asking price. Or you could represent a user with a vector of their ratings for different movies.

Each number in a vector is a component, and the total number of components gives the vector's dimension. A vector with two components, like $\begin{bmatrix} 3 \\ 2 \end{bmatrix}$, can be visualized as an arrow in a 2D plane, starting at the origin and ending at the point (3, 2).

Vectors can be added together or multiplied by a single number, called a scalar. Adding two vectors, $\vec{u} + \vec{v}$, results in a new vector that represents the combined effect of both. Multiplying a vector by a scalar, like $2\vec{v}$, scales its length without changing its direction.

The collection of all possible vectors of a certain dimension is called a vector space. A 2D vector space is like an infinite sheet of graph paper; a 3D vector space is the space we live in. These spaces provide the stage where all the action of linear algebra takes place.

A matrix is a rectangular grid of numbers, arranged in rows and columns. You can think of a matrix as a collection of vectors stacked together. For example, a dataset of 100 houses, each with 3 features, could be represented by a 100x3 matrix.

Matrices are powerful because they don't just store data; they can also act on it. A matrix can transform a vector by rotating, stretching, or shearing it. This is the foundation for many machine learning techniques.

Like vectors, matrices have rules for addition, subtraction, and multiplication. Addition and subtraction are straightforward: you just add or subtract the corresponding elements. But matrix multiplication is different. Multiplying two matrices combines their transformations into a single new transformation.

For example, if matrix A rotates a vector and matrix B stretches it, the matrix product BA is a single matrix that does both the rotation and the stretching.

How can we summarize the effect of a matrix transformation? One way is with the determinant. The determinant is a single number that tells you how much a matrix scales area (in 2D) or volume (in 3D).

If a 2x2 matrix has a determinant of 3, it means it makes any area 3 times larger. A negative determinant means the transformation also flips the orientation of space, like looking in a mirror. A determinant of zero is special: it means the matrix squishes space into a lower dimension, like collapsing a 2D plane into a single line.

While the determinant gives a general sense of scaling, eigenvalues and eigenvectors tell a more specific story. During a transformation, most vectors get knocked off their original direction. But some special vectors, called eigenvectors, only get stretched or shrunk. They stay pointed in the same direction.

The factor by which an eigenvector is stretched or shrunk is its corresponding eigenvalue. If an eigenvector has an eigenvalue of 2, the transformation doubles its length. If the eigenvalue is -0.5, the vector is flipped and its length is halved.

Complex matrices can be hard to understand. Matrix decomposition, or factorization, is the process of breaking a matrix down into a product of simpler, more fundamental matrices. It's like finding the prime factors of a number (e.g., $12 = 2 \times 2 \times 3$). By studying the factors, we can better understand the original.

Several types of decomposition exist, each useful for different tasks.

These decompositions are the workhorses behind many machine learning algorithms. They provide numerically stable and efficient ways to solve complex problems by breaking them into simpler parts.

That's a quick tour of the core concepts in linear algebra. These building blocks—vectors, matrices, and their transformations—are essential for representing data and understanding how algorithms learn from it.