So far, we've treated row operations—scaling, swapping, and adding—as manual steps you perform on a matrix. But in linear algebra, we can translate these actions into the language of matrix multiplication. Instead of just doing an operation to a matrix, we can multiply it by a special matrix to achieve the exact same result.

These special matrices are called elementary matrices. The trick is simple: to create an elementary matrix that performs a specific row operation, you just perform that same operation on an identity matrix. The result is a new matrix that acts like a machine, built to execute that single task.

There are three types of elementary matrices, one for each row operation. Let's see how to build each one.

Row Scaling

To create a matrix that scales a row by a constant $c$, we take an identity matrix and multiply the corresponding row by $c$. For example, to scale the second row of a 3x3 matrix by $c$, we start with the 3x3 identity matrix:

Then, we multiply its second row by $c$ to get our elementary matrix, which we'll call $E_{scale}$.

Multiplying any 3x3 matrix $A$ by $E_{scale}$ on the left will scale the second row of $A$ by $c$.

Row Swapping

To swap two rows, we swap the corresponding rows in an identity matrix. These are often called Permutation matrices because they permute, or reorder, the rows. To swap rows 1 and 3 of a 3x3 matrix, we create $E_{swap}$ by swapping the first and third rows of the identity matrix.

Row Addition

This is the workhorse of Gaussian elimination. To create a matrix that adds a multiple of one row to another, we perform that operation on the identity matrix. For instance, to create a matrix that adds $k$ times row 1 to row 3, we start with $I$ and add $k$ times its first row to its third row.

You might have noticed we always say "multiply on the left." This is critical. Multiplying a matrix $A$ by an elementary matrix $E$ on the left ($EA$) performs the operation on the rows of $A$.

What happens if you multiply on the right ($AE$)? It performs the equivalent column operation. For example, if $E$ is a row-swapping matrix, $AE$ will swap the columns of $A$ instead of the rows.

This framework turns the procedural steps of row reduction into a sequence of matrix multiplications. It's a powerful idea that allows us to analyze and automate the process of solving linear systems in a more abstract, algebraic way.