At its core, a vector is just a list of numbers. For example, [3, 4] is a vector. So is [10, -2, 5.5]. Each number in the list is called a component or an element.

In machine learning, vectors are used to represent data. Imagine you're describing houses. You might create a vector for each house with features like [number of bedrooms, square footage, age in years]. A three-bedroom, 2000-square-foot house that's 10 years old could be represented as the vector [3, 2000, 10].

Geometrically, you can think of a vector as an arrow in space. It has two key properties: magnitude (how long is the arrow?) and direction (which way does it point?). A vector like [3, 4] can be visualized as an arrow starting at the origin (0,0) and ending at the point (3,4) on a graph.

The number of components in a vector determines its dimensionality. The vector [3, 4] is two-dimensional (2D), while [3, 2000, 10] is three-dimensional (3D). Machine learning models often work with vectors in hundreds or even thousands of dimensions.

To work with vectors, we need a few basic operations. Let’s say we have two 2D vectors, $\vec{v} = [v_1, v_2]$ and $\vec{w} = [w_1, w_2]$.

For example, if $\vec{v} = [2, 5]$ and $\vec{w} = [3, 1]$, their sum is $[2+3, 5+1] = [5, 6]$. Geometrically, adding vectors is like placing them tip-to-tail. The new vector goes from the start of the first vector to the tip of the second.

Scalar Multiplication: A scalar is just a regular number, not a vector. Multiplying a vector by a scalar means you multiply every component of the vector by that number. This scales the vector, making it longer or shorter.

If we multiply our vector $\vec{v} = [2, 5]$ by the scalar 3, we get $[3 \cdot 2, 3 \cdot 5] = [6, 15]$. This new vector points in the same direction but is three times as long. Multiplying by a negative scalar reverses the vector's direction.

Vectors don't exist in a vacuum. They live in a vector space. A vector space is the set of all possible vectors of a certain dimensionality. For instance, the 2D vector space, often written as $\mathbb{R}^2$, includes every possible 2D vector you can imagine, like [1, 2], [-5, 8.3], and [0, 0]. It's essentially the entire flat plane of an x-y graph.

Similarly, $\mathbb{R}^3$ is the 3D space we live in, and it contains all 3D vectors. The rules of a vector space ensure that if you add any two vectors within it, or multiply any vector by a scalar, the result is still a vector in that same space.

Finally, we often need to measure the size or length of a vector. This is called the norm. The most common type is the Euclidean norm (or L2 norm), which corresponds to the straight-line length of the vector's arrow. For a 2D vector $\vec{v} = [v_1, v_2]$, the formula is based on the Pythagorean theorem.

For our vector [3, 4], the norm would be $\sqrt{3^2 + 4^2} = \sqrt{9 + 16} = \sqrt{25} = 5$. The length of the arrow from (0,0) to (3,4) is 5 units. This idea of measuring vector length is crucial for many machine learning algorithms, which often rely on calculating the "distance" between different data points.

Ready to check your understanding?

These basic concepts, from simple lists of numbers to the spaces they inhabit, are the foundation upon which complex machine learning models are built.