In machine learning, we often work with large amounts of data. To handle this data efficiently, we need a way to organize and manipulate it. This is where vectors and matrices come in. Think of them as specialized containers for numbers, allowing us to perform powerful calculations on entire datasets at once.

Let's start with the simpler of the two: vectors.

A vector is an ordered list of numbers. In machine learning, a vector can represent a single data point with multiple features. For example, if we were describing a house, a vector could hold its features like area, number of bedrooms, and age.

Vectors are typically written as a column of numbers enclosed in brackets. This vector $\vec{v}$ has three dimensions, or components:

Like regular numbers, vectors can be added, subtracted, and multiplied. These operations are the foundation for more complex tasks.

Let's say we have two vectors, $\vec{a}$ and $\vec{b}$.

To add them, we just add their corresponding components. The first component of $\vec{a}$ gets added to the first component of $\vec{b}$, and so on. Visually, this is like placing the vectors tip-to-tail to find the resulting path.

Subtraction works the same way, just with subtraction instead of addition.

We can also multiply a vector by a single number, called a scalar. This operation scales the vector, making it longer or shorter. If the scalar is negative, it also reverses the vector's direction.

Multiplying our vector $\vec{a}$ by the scalar 3 stretches it to three times its original length.

If a vector is a list of numbers, a matrix is a grid of numbers, arranged in rows and columns. You can think of a matrix as a collection of vectors stacked together. This structure is perfect for representing entire datasets. For instance, each row could be a different house (a data point), and each column could be a different feature (area, bedrooms, age).

The size, or dimension, of a matrix is given by its number of rows and columns. A matrix with 2 rows and 3 columns is a $2 \times 3$ matrix.

Just like vectors, matrices have their own set of operations.

Matrix addition is straightforward, but it has one rule: you can only add matrices that have the exact same dimensions. The addition happens element-wise, just like with vectors.

Matrix multiplication is a bit more involved. To multiply two matrices, the number of columns in the first matrix must equal the number of rows in the second matrix. If you multiply an $m \times n$ matrix by an $n \times p$ matrix, the result will be an $m \times p$ matrix.

Let's see an example. We'll multiply a $2 \times 3$ matrix by a $3 \times 2$ matrix. The result will be a $2 \times 2$ matrix.

After doing the arithmetic:

One important property of matrix multiplication is that it's not commutative. In other words, $A \times B$ is generally not the same as $B \times A$. The order matters!

Finally, the transpose of a matrix is what you get when you swap its rows and columns. The first row becomes the first column, the second row becomes the second column, and so on. The transpose of matrix $A$ is written as $A^T$.

Transposing is a common operation used to get matrices into the right shape for multiplication and other calculations.

Ready to check your understanding?

Understanding how to work with vectors and matrices is the first major step into the mathematics behind machine learning. These tools give us a powerful way to represent and transform data.