Vectors and Matrices

The Building Blocks: Vectors

Imagine you're giving someone directions. You might say, "Walk two blocks east and then one block north." That instruction has both a distance (magnitude) and a direction. In mathematics, we have a special object for that: a vector.

A vector is essentially a list of numbers that represents a point in space, giving it both magnitude and direction from a starting point (the origin). We usually write them vertically in brackets, like this:

\vec{v} = \begin{bmatrix} 2 \\ 1 \end{bmatrix}

This vector, $\vec{v}$ , represents the instruction "go 2 units along the x-axis and 1 unit up the y-axis."

Working with Vectors

Just like numbers, we can perform operations on vectors. The rules are simple and intuitive.

Vector Addition: To add two vectors, you just add their corresponding components. If you have vector $\vec{a}$ (go 2 east, 1 north) and vector $\vec{b}$ (go 1 east, 3 north), adding them gives you a new vector that represents the total displacement.

\vec{a} + \vec{b} = \begin{bmatrix} 2 \\ 1 \end{bmatrix} + \begin{bmatrix} 1 \\ 3 \end{bmatrix} = \begin{bmatrix} 2+1 \\ 1+3 \end{bmatrix} = \begin{bmatrix} 3 \\ 4 \end{bmatrix}

Geometrically, this is like placing the tail of the second vector at the tip of the first. The new vector goes from the start of the first to the end of the second.

Scalar Multiplication: What if you wanted to go twice as far in the same direction? You'd multiply your vector by 2. This is called scalar multiplication, because you're scaling the vector with a single number (a scalar). You just multiply every component in the vector by that scalar.

2 \cdot \vec{a} = 2 \begin{bmatrix} 2 \\ 1 \end{bmatrix} = \begin{bmatrix} 2 \cdot 2 \\ 2 \cdot 1 \end{bmatrix} = \begin{bmatrix} 4 \\ 2 \end{bmatrix}

Multiplying by a negative scalar flips the vector's direction.

The Dot Product: Sometimes we need to multiply two vectors to get a single number, not another vector. This is what the dot product does. You multiply the corresponding components of two vectors and then sum the results. It tells you how much one vector points in the direction of the other.

\vec{a} \cdot \vec{b} = \begin{bmatrix} 2 \\ 1 \end{bmatrix} \cdot \begin{bmatrix} 1 \\ 3 \end{bmatrix} = (2 \cdot 1) + (1 \cdot 3) = 2 + 3 = 5

The result, 5, is a scalar. This operation is fundamental in machine learning, especially for measuring similarity between data points.

Meet the Matrix

If a vector is a list of numbers, a matrix is a grid of numbers, arranged in rows and columns. Matrices are incredibly useful for organizing data and, as we'll see later, for performing complex transformations on vectors.

A matrix's size is described by its number of rows and columns. For example, this is a 2x3 matrix (2 rows, 3 columns):

A = \begin{bmatrix} 5 & 8 & 2 \\ 3 & 1 & 7 \end{bmatrix}

Matrices can be added and multiplied, too, but the rules are a bit stricter.

Matrix Addition: You can only add two matrices if they have the exact same dimensions. The process is just like vector addition: you add the corresponding elements.

\begin{bmatrix} 5 & 8 \\ 3 & 1 \end{bmatrix} + \begin{bmatrix} 1 & 0 \\ 2 & 4 \end{bmatrix} = \begin{bmatrix} 5+1 & 8+0 \\ 3+2 & 1+4 \end{bmatrix} = \begin{bmatrix} 6 & 8 \\ 5 & 5 \end{bmatrix}

Matrix Multiplication: This is the most complex operation. To multiply two matrices, the number of columns in the first matrix must equal the number of rows in the second. Each entry in the resulting matrix is the dot product of a row from the first matrix and a column from the second.

For example, to multiply a 2x3 matrix by a 3x2 matrix, the result will be a 2x2 matrix.

One crucial property of matrix multiplication is that it's not commutative. In general, $A \times B \neq B \times A$ .

Matrix Transpose: The transpose of a matrix, denoted as $A^T$ , is what you get when you flip the matrix over its main diagonal. The rows become columns and the columns become rows.

A = \begin{bmatrix} 5 & 8 & 2 \\ 3 & 1 & 7 \end{bmatrix} \quad A^T = \begin{bmatrix} 5 & 3 \\ 8 & 1 \\ 2 & 7 \end{bmatrix}

Special Kinds of Matrices

Certain types of matrices have unique properties that make them especially useful.

Identity Matrix

noun

The matrix equivalent of the number 1. Multiplying any matrix by an identity matrix of the appropriate size leaves it unchanged. It has 1s on the main diagonal and 0s everywhere else.

I_3 = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}

Diagonal Matrix: A matrix where all the non-diagonal elements are zero. These are computationally easy to work with.

D = \begin{bmatrix} 3 & 0 & 0 \\ 0 & -2 & 0 \\ 0 & 0 & 5 \end{bmatrix}

Symmetric Matrix: A square matrix that is equal to its own transpose ( $A = A^T$ ). These matrices appear frequently in statistics and machine learning, often representing things like the covariance between different features in a dataset.

S = \begin{bmatrix} 1 & 7 & 3 \\ 7 & 4 & -5 \\ 3 & -5 & 6 \end{bmatrix}

With these basic definitions for vectors and matrices, you have the foundational language for much of modern data science.

Quiz Questions 1/5

In mathematics, what does a vector fundamentally represent?

Quiz Questions 2/5

What is the result of adding the vectors $\begin{bmatrix} 4 \\ -1 \end{bmatrix}$ and $\begin{bmatrix} -2 \\ 5 \end{bmatrix}$ ?

Math Foundations for Vision and AI