Vectors and Scalars

Beyond Single Numbers

In our daily lives, we're constantly dealing with numbers. The temperature is 18 degrees Celsius. A car is travelling at 50 kilometres per hour. These single values are called scalars. They tell you 'how much' of something there is.

scalar

noun

A quantity that is fully described by a single numerical value, representing its magnitude.

But what if 'how much' isn't the whole story? If you're navigating, knowing just the distance isn't enough. You also need to know the direction. This combination of magnitude and direction is a vector.

A vector has both a magnitude and a direction. A scalar has only magnitude.

vector

noun

A quantity that has both magnitude and direction. It is often represented as an arrow or a list of numbers.

Representing Vectors

Geometrically, we can think of a vector as an arrow in space. The length of the arrow is its magnitude, and where it points is its direction.

Mathematically, we represent vectors as an ordered list of numbers, called components. For a two-dimensional vector (like on a map), we have two components: one for the horizontal movement (x-axis) and one for the vertical movement (y-axis). We usually write this in a column.

\vec{v} = \begin{pmatrix} 3 \\ 4 \end{pmatrix}

In machine learning, vectors are incredibly useful. Imagine you want to represent a user's movie preferences. You could create a vector where each component corresponds to a genre. A high number means they like that genre a lot, while a low number means they don't.

For example, the vector below could represent someone who loves science fiction, is neutral about comedy, and dislikes romance.

\text{user_prefs} = \begin{pmatrix} 9.5 \\ 5.0 \\ 1.2 \end{pmatrix} \begin{matrix} \leftarrow \text{Sci-Fi} \\ \leftarrow \text{Comedy} \\ \leftarrow \text{Romance} \end{matrix}

Working with Vectors

We can perform mathematical operations on vectors. The three basic operations are addition, subtraction, and scalar multiplication.

Vector operations are performed component-wise. This means we work with each corresponding pair of numbers separately.

Let's say we have two vectors, $\vec{a}$ and $\vec{b}$ .

\vec{a} = \begin{pmatrix} 2 \\ 5 \end{pmatrix}, \quad \vec{b} = \begin{pmatrix} 4 \\ 1 \end{pmatrix}

To add them, we add the corresponding components. Geometrically, this is like placing the tail of the second vector at the tip of the first. The new vector, called the resultant vector, goes from the start of the first to the tip of the second.

\vec{a} + \vec{b} = \begin{pmatrix} 2+4 \\ 5+1 \end{pmatrix} = \begin{pmatrix} 6 \\ 6 \end{pmatrix}

Subtraction works the same way, but we subtract the components.

\vec{a} - \vec{b} = \begin{pmatrix} 2-4 \\ 5-1 \end{pmatrix} = \begin{pmatrix} -2 \\ 4 \end{pmatrix}

Finally, we can multiply a vector by a scalar. This operation scales the vector, changing its magnitude but not its direction (unless the scalar is negative, which reverses the direction). We simply multiply each component by the scalar.

3\vec{a} = 3 \begin{pmatrix} 2 \\ 5 \end{pmatrix} = \begin{pmatrix} 3 \times 2 \\ 3 \times 5 \end{pmatrix} = \begin{pmatrix} 6 \\ 15 \end{pmatrix}

These basic building blocks, scalars and vectors, are fundamental to linear algebra and provide the language for describing data and transformations in machine learning.

Quiz Questions 1/5

What is the fundamental difference between a scalar and a vector?

Quiz Questions 2/5

Which of the following is an example of a scalar quantity?

Understanding these core concepts is the first step toward working with more complex models.

Linear Algebra for Machine Learning Beginners