In machine learning, data is rarely a single number. An image is a grid of pixels, a user profile is a collection of preferences, and a sentence is a sequence of words. We need a way to represent this high-dimensional data, and that's where linear algebra comes in. It provides the structure and tools to work with data as vectors and matrices.

A single data point, like a user's ratings for three movies, can be a vector:

A collection of these data points, our entire dataset, becomes a matrix, where each column (or row) is a different user.

Matrices don't just store data; they transform it. Multiplying a data vector by a matrix can rotate, stretch, or shear it. Within these transformations, certain vectors are special. They are the eigenvectors. When a matrix transformation is applied, an eigenvector's direction doesn't change. It only gets scaled by a factor, called the eigenvalue.

Think of eigenvectors as the skeleton of a transformation. They point in the directions where the transformation's behavior is simplest—pure stretching or compressing. This idea is fundamental to many machine learning techniques that aim to find the most important features or directions in the data.

Singular Value Decomposition (SVD) takes this concept further. It's a powerful factorization that breaks any matrix $A$ into three other matrices:

In essence, SVD tells us that any linear transformation can be broken down into a rotation, a scaling along the axes, and another rotation. The singular values in $\Sigma$ are ordered by magnitude, telling us which dimensions hold the most information. For dimensionality reduction, we can simply discard the smallest singular values (and corresponding parts of $U$ and $V$) to get a compressed version of our data that preserves the most important structural information. This is the core principle behind Principal Component Analysis (PCA).

If linear algebra gives us the language to describe data, calculus gives us the tools to learn from it. Most machine learning models have parameters (weights) that we need to tune. The goal of 'training' is to find the parameter values that make the model's predictions as accurate as possible. We measure inaccuracy with a 'loss function'. The lower the loss, the better the model.

Finding the lowest point of a loss function is an optimization problem. In a model with millions of parameters, the loss function is a high-dimensional surface. How do we navigate it to find the minimum? We use derivatives.

The gradient tells us which way is 'up'. To minimize our loss, we simply take a small step in the opposite direction. This is the core idea of Gradient Descent. The size of that step is determined by the learning rate.

For complex models like neural networks, the loss function isn't a direct function of a single weight. It's a nested function—the output of one layer is the input to the next. To find the gradient, we need the chain rule. It allows us to calculate the derivative of a composite function by multiplying the derivatives of its inner and outer parts. Backpropagation is essentially a clever, recursive application of the chain rule to compute the gradients for all weights in a network efficiently.

Data is rarely perfect; it's noisy and incomplete. Probability and statistics are the tools we use to quantify uncertainty and make inferences from limited data. One of the most important concepts is Bayes' Theorem.

Bayes' Theorem provides a formal way to update our beliefs in light of new evidence. This is the foundation of Bayesian machine learning, where we don't just find a single best value for a model parameter, but an entire probability distribution of likely values.

When modeling continuous variables, like the height of a person or the error of a prediction, the Gaussian (or Normal) distribution is ubiquitous. It's defined by its mean ($\\,mu$) and standard deviation ($\\,sigma$), and its bell-shaped curve describes how data is often centered around an average value.

Statistics helps us make decisions based on this probabilistic view of data. For instance, after training a new model, we might want to know: is it actually better than the old one, or is the performance difference just due to random chance?

This is where hypothesis testing comes in. We start with a null hypothesis ($H_0$), which usually states there is no effect (e.g., "the new model is not better than the old one"). We then calculate a p-value, which is the probability of observing our data (or more extreme data) if the null hypothesis were true. If the p-value is very small (typically below a chosen significance level, or alpha, like 0.05), we reject the null hypothesis and conclude that the observed effect is statistically significant.

Let's review the main ideas we've covered.

Time to check your understanding.

These mathematical pillars—linear algebra for representation, calculus for optimization, and probability for uncertainty—are the bedrock of modern machine learning. Understanding them moves you from just using tools to truly understanding why they work.