Python for AI Data Structures
Understanding Tensors
What's a Tensor?
At the heart of modern AI are special containers for numbers called tensors. Think of them as a way to organize data. You're already familiar with the basic forms of tensors, even if you don't use that name.
A single number, like 5, is a 0-dimensional tensor, also called a scalar. A list of numbers, like [1, 2, 3], is a 1-dimensional tensor, or a vector. A grid of numbers, like a spreadsheet, is a 2-dimensional tensor, which you probably know as a matrix.
Tensors are simply the generalization of these concepts to any number of dimensions. They are multi-dimensional arrays of numbers.
So, a 3D tensor is like a cube of numbers, a 4D tensor is like a list of cubes, and so on. This ability to store data in many dimensions makes tensors incredibly powerful for representing complex information, which is exactly what AI models need to do.
tensor
noun
A multi-dimensional array of numbers. Tensors are a generalization of scalars (0D), vectors (1D), and matrices (2D) to any number of dimensions.
Tensors in Action
Let's make this more concrete. How does a computer see an image? As a tensor.
A simple grayscale image can be represented as a 2D tensor (a matrix). Each number in the matrix corresponds to the brightness of a pixel. For example, 0 could be black, 255 could be white, and numbers in between would be shades of gray.
A color image is a 3D tensor. It's typically represented by three 2D matrices stacked together: one for the red color values, one for green, and one for blue (the RGB color model). So the dimensions would be [height, width, 3].
What if you have a collection of images, like a batch of 10 color pictures you want to feed into a model? That would be a 4D tensor with dimensions [10, height, width, 3]. Tensors provide a flexible and consistent way to handle all this data.
Tensors in Code
Frameworks like PyTorch and TensorFlow are built around tensors. They provide powerful tools for creating and manipulating them, especially for the kinds of calculations needed in AI.
Let's see how this works in Python using PyTorch. First, you import the library and create a couple of tensors. In this case, we'll make two 2x2 matrices.
A tensor is a multi-dimensional array that is the fundamental data structure used in PyTorch (and many other machine learning frameworks).
import torch
# Create a tensor from a list of lists
tensor_a = torch.tensor([[1, 2], [3, 4]])
tensor_b = torch.tensor([[5, 6], [7, 8]])
print(tensor_a)
print(tensor_b)
Now, we can perform mathematical operations on them, just like with regular numbers. For instance, we can add them together. This performs an element-wise addition, meaning it adds the numbers in the same position in each tensor.
# Add the two tensors
sum_tensor = tensor_a + tensor_b
print(sum_tensor)
# tensor([[ 6, 8],
# [10, 12]])
We can do the same with multiplication.
# Multiply the two tensors element-wise
product_tensor = tensor_a * tensor_b
print(product_tensor)
# tensor([[ 5, 12],
# [21, 32]])
These simple operations are the building blocks. AI models perform millions of these calculations on massive tensors to learn patterns from data.
In the context of modern AI, what is a tensor?
A standard spreadsheet with rows and columns of numbers is conceptually equivalent to what kind of tensor?
Understanding tensors is the first step into the world of AI data representation. With this foundation, you can start to see how complex data like images, text, and sound are prepared for machine learning models.