In math, we often work with single numbers, called scalars. Your age is a scalar. The price of a coffee is a scalar. But what if you need to describe something with both a size and a direction? For that, we use vectors.

A vector is like a set of instructions. "Walk 3 miles north" is a vector. It has a magnitude (3 miles) and a direction (north). In machine learning and physics, vectors are used everywhere to represent things like forces, velocities, or even data points in a high-dimensional space.

We can represent vectors using coordinates. A vector in a 2D plane might be written as $\vec{v} = (3, 4)$, meaning it points 3 units along the x-axis and 4 units up the y-axis. The numbers 3 and 4 are its components. In three dimensions, a vector would have three components, like $(x, y, z)$.

Just like with numbers, we can perform operations on vectors. The two most basic are addition and scalar multiplication.

Geometrically, this is like following one set of instructions and then another. If you walk 3 miles north (vector $\vec{a}$) and then 4 miles east (vector $\vec{b}$), your final position relative to your start is the sum of the two vectors. This is often called the "head-to-tail" rule.

Multiplying a vector by 2 doubles its length but keeps its direction. Multiplying by -1 reverses its direction but keeps its length. Multiplying by 0.5 halves its length.

How do you multiply two vectors? It's not as straightforward as adding them. There are two main ways: the dot product and the cross product.

The dot product of vectors $\vec{a}$ and $\vec{b}$ is written as $\vec{a} \cdot \vec{b}$. You calculate it by multiplying corresponding components and summing the results.

For example, if $\vec{a} = (1, 2, 3)$ and $\vec{b} = (4, -5, 6)$, their dot product is $(1)(4) + (2)(-5) + (3)(6) = 4 - 10 + 18 = 12$.

The result's sign tells you about the angle between the vectors. If it's positive, the angle is less than 90°. If it's negative, the angle is greater than 90°.

Next is the cross product, which is only defined for vectors in 3D space.

The cross product of $\vec{a}$ and $\vec{b}$, written $\vec{a} \times \vec{b}$, creates a new vector. The direction of this new vector is found using the "right-hand rule": if you point your index finger in the direction of $\vec{a}$ and your middle finger in the direction of $\vec{b}$, your thumb points in the direction of $\vec{a} \times \vec{b}$.

The magnitude (length) of the resulting vector is equal to the area of the parallelogram formed by $\vec{a}$ and $\vec{b}$.

Vectors are perfect for describing lines and planes. A line in space can be defined by a point on the line and a direction vector that is parallel to the line.

Planes work similarly, but they are defined by a point and a normal vector—a vector that is perpendicular (orthogonal) to the plane. The equation of a plane is derived from the dot product.

If $\vec{n} = (A, B, C)$ is the normal vector and $\vec{p_0} = (x_0, y_0, z_0)$ is a known point on the plane, then for any other point $\vec{p} = (x, y, z)$ on the plane, the vector from $\vec{p_0}$ to $\vec{p}$ must be perpendicular to $\vec{n}$. This means their dot product is zero:

This expands to the standard equation of a plane: $A(x-x_0) + B(y-y_0) + C(z-z_0) = 0$. Understanding this geometric relationship is key to working with planes in higher dimensions, a common task in data analysis and machine learning.

Let's check your understanding of these core vector concepts.