Imagine you're giving someone directions. You might say, "Walk three blocks east and four blocks north." That instruction is a vector. It has a magnitude (the total distance) and a direction (northeast-ish). In mathematics and computer science, vectors are simply lists of numbers that represent a point in space, holding both size and direction.

The directions "three blocks east and four blocks north" can be written as a vector $\vec{v} = [3, 4]$. Each number in the vector is called a component. This vector lives in a 2-dimensional space because it has two components. A vector like $[5, 2, -1]$ would live in a 3-dimensional space. In machine learning, vectors can have thousands or even millions of dimensions, representing complex data like images or text.

There are two fundamental operations you can perform with vectors: addition and scalar multiplication.

Vector addition is like following one set of directions after another. If you first walk [3, 4] and then walk [1, -2] (one block east, two blocks south), where do you end up? You just add the corresponding components:

Your final position is four blocks east and two blocks north of your starting point.

Scalar multiplication involves scaling a vector by a plain number (a "scalar"). If you wanted to walk twice as far in the same direction as our original vector, you would multiply it by the scalar 2:

The new vector points in the same direction but is twice as long. Multiplying by a negative scalar would flip the vector's direction.

When we combine scalar multiplication and vector addition, we get a linear combination. For example, we can take two vectors $\vec{u} = [1, 2]$ and $\vec{v} = [3, 1]$ and combine them like this: $2\vec{u} + 3\vec{v} = 2[1, 2] + 3[3, 1] = [2, 4] + [9, 3] = [11, 7]$.

This simple idea is incredibly powerful. The set of all possible vectors you can create by taking linear combinations of a set of vectors is called their span.

For example, the span of a single vector like $[2, 1]$ is an infinite line passing through the origin and the point (2,1). You can get to any point on that line by multiplying the vector by some scalar.

If you have two vectors that don't point along the same line, like $[1, 0]$ (east) and $[0, 1]$ (north), their span is the entire 2D plane. You can reach any point $[x, y]$ by finding the right scalars $c_1$ and $c_2$ to form the combination $c_1[1, 0] + c_2[0, 1]$.

This leads to the ideas of basis and dimension. A vector space is the set of all possible vectors of a certain length (e.g., all 2D vectors, which form the 2D plane). A basis for a vector space is a set of vectors that is just large enough to span the entire space, without any redundancy.

For a basis, two conditions must be met:

1. The vectors must span the space (you can reach every point).
2. The vectors must be linearly independent (none of the vectors can be made from a combination of the others).

This second point is about efficiency. The set of vectors {[1, 0], [0, 1], [2, 2]} spans the 2D plane, but it's not a basis because [2, 2] is redundant, it can be made from the other two (2*[1,0] + 2*[0,1]). The set {[1, 0], [0, 1]} is a basis because both vectors are needed and they're sufficient.

The number of vectors in any basis for a space is called the dimension of that space. For the 2D plane, the dimension is 2. For 3D space, it's 3. In machine learning, the dimension of a vector space can be very high, reflecting the complexity of the data.

Let's check your understanding of these core concepts.

Understanding these building blocks—vectors, operations, span, and basis—is the first step into the world of linear algebra that powers so much of modern AI.