Essential Linear Algebra for Data Science and Engineering
Vector Spaces
The Playground for Vectors
Imagine a playground with specific rules for how you can move around. You can walk from one point to another, and you can also walk a certain distance in a specific direction. Vectors are like these movements, and a vector space is the playground itself. It's a collection of vectors where two main operations are allowed: adding vectors together and multiplying them by numbers (scalars).
A vector space is a set of vectors, along with rules for vector addition and scalar multiplication, that must satisfy ten specific axioms.
These axioms ensure that the space behaves in a consistent, predictable way. They aren't arbitrary rules; they're the properties that make vector spaces so useful. For example, the familiar 2D plane (called ) and 3D space () are both vector spaces. Any arrow you can draw in them follows the rules.
Subspaces
Within a large playground, you might find smaller, self-contained areas. A subspace is like that: it's a part of a larger vector space that is also a vector space in its own right. For a set of vectors to be a subspace, it must pass three simple tests.
A subset of a vector space is a subspace if:
- It contains the zero vector ().
- It's closed under addition (if , then ).
- It's closed under scalar multiplication (if and is a scalar, then ).
Think about 3D space (). A flat plane that passes through the origin is a subspace. Why? It contains the origin (the zero vector), adding any two vectors on the plane gives you another vector on that same plane, and scaling any vector on the plane keeps it on the plane. However, a plane that doesn't pass through the origin is not a subspace because it fails the first test.
Building with Vectors
The most powerful idea in linear algebra is combining vectors to create other vectors. A linear combination is just a scaled sum of vectors. For example, given vectors and , a linear combination would be an expression like , where and are scalars.
Now, imagine all the possible vectors you can create by taking every single linear combination of a set of vectors. This collection of all possible results is called the span.
Span
noun
The set of all possible linear combinations of a collection of vectors.
In , the span of a single non-zero vector is a line through the origin. The span of two vectors that don't point in the same or opposite direction is a plane through the origin. And the span of three well-chosen vectors can be all of .
The Bare Essentials
What if we want to describe an entire vector space using the smallest possible set of building blocks? This brings us to the idea of a basis.
A basis is a set of vectors that is both linearly independent and spans the space.
Spans the space means you can reach any vector in the space with a linear combination of the basis vectors. Linearly independent means that none of the vectors in the basis can be created by a combination of the others. In other words, there's no redundancy. Every vector in the basis contributes something new and unique.
For the 2D plane , a common basis is the set of vectors and . They are linearly independent (you can't make by scaling ), and they span all of (any vector can be written as ). But this isn't the only basis; any two non-collinear vectors will do.
The number of vectors in a basis is a fundamental property of the vector space, called its dimension. The dimension of is 2, and the dimension of is 3. This matches our intuition. Every basis for a given vector space will have the same number of vectors.
Understanding the basis and dimension of a space is crucial. It tells us the minimum amount of information we need to describe any point within that space, a concept that is fundamental to everything from computer graphics to machine learning.
Which of the following best describes a vector space?
In the vector space , which of the following sets of vectors constitutes a subspace?
Vector spaces provide the stage on which linear algebra happens. By understanding these core concepts, you have the foundation to explore more advanced topics.