Convolutional Neural Networks

Seeing with Math

How does a computer see a picture? To us, a photo of a cat is just... a cat. To a computer, it's a massive grid of numbers, where each number represents the color of a single pixel. A Convolutional Neural Network, or CNN, is a special type of neural network designed to find patterns in these grids of numbers. They are the workhorses behind image recognition, self-driving cars, and medical image analysis.

Convolutional Neural Networks (CNNs) are specifically designed for handling image data.

The core operation that gives CNNs their name is the convolution. Think of it like shining a small flashlight over a large painting in a dark room. The flashlight is called a kernel or filter, and the painting is the input image. As you slide the flashlight across the painting, you only see a small patch at a time. The CNN does the same thing. It slides a small kernel over the image's grid of pixels, one patch at a time, to look for simple patterns like edges, corners, or colors.

Mathematically, this sliding and computing operation is a discrete convolution. Each position in the output, called a feature map, is the sum of the element-wise product of the kernel and the corresponding patch of the input image.

S(i, j) = \sum_{m}\sum_{n} I(i-m, j-n) K(m, n)

By using different kernels, a CNN can create different feature maps, each one highlighting a specific feature. One kernel might detect vertical edges, another might detect horizontal edges, and another might detect a certain color. Together, these simple features form the building blocks for recognizing more complex objects.

The Layers of a CNN

A modern CNN is composed of a series of layers stacked on top of each other. The three main types are the convolutional layer, the pooling layer, and the fully connected layer.

Convolutional Layer: This is the core building block we've been discussing. It applies a set of kernels to the input to create a stack of feature maps. Early layers learn simple features like edges and corners. Deeper layers combine these simple features to learn more complex ones, like eyes, noses, or wheels.

Pooling Layer: After a convolutional layer, it's common to add a pooling layer. Its job is to shrink the size of the feature maps, which reduces the number of calculations and helps the network focus on the most important information. The most common type is max pooling, which takes a small window of pixels (say, 2x2) and keeps only the maximum value, discarding the rest. This makes the network more robust to small shifts or rotations in the image.

Pooling helps the network generalize, so it can recognize an object even if it's slightly off-center or tilted.

Fully Connected Layer: After several convolutional and pooling layers have extracted features from the image, the final feature maps are flattened into a single, long vector of numbers. This vector is then fed into a standard, fully connected neural network. This part of the network acts as a classifier; it takes the high-level features and decides what the original image actually contains. For example, it might output probabilities that the image is a 'cat', 'dog', or 'bird'.

Building a CNN in Code

Frameworks like PyTorch and TensorFlow make building CNNs straightforward. Here’s what a simple CNN architecture looks like in PyTorch. It defines two convolutional layers, each followed by a pooling layer, and then two fully connected layers for classification.

import torch
import torch.nn as nn

class SimpleCNN(nn.Module):
    def __init__(self):
        super(SimpleCNN, self).__init__()
        # Input: 3 color channels, Output: 16 feature maps
        self.conv1 = nn.Conv2d(3, 16, kernel_size=3, padding=1)
        self.pool = nn.MaxPool2d(kernel_size=2, stride=2)
        # Input: 16 feature maps, Output: 32 feature maps
        self.conv2 = nn.Conv2d(16, 32, kernel_size=3, padding=1)
        # Fully connected layers
        # The size depends on the input image dimensions
        self.fc1 = nn.Linear(32 * 8 * 8, 120) # Assuming 32x32 input image
        self.fc2 = nn.Linear(120, 10) # 10 output classes

    def forward(self, x):
        x = self.pool(torch.relu(self.conv1(x)))
        x = self.pool(torch.relu(self.conv2(x)))
        x = x.view(-1, 32 * 8 * 8) # Flatten the feature map
        x = torch.relu(self.fc1(x))
        x = self.fc2(x)
        return x

And here is the equivalent model using TensorFlow's Keras API. The structure is identical, just with slightly different syntax.

import tensorflow as tf
from tensorflow.keras import layers, models

def create_simple_cnn():
    model = models.Sequential()
    # Input shape: 32x32 image with 3 color channels
    model.add(layers.Conv2D(16, (3, 3), activation='relu', input_shape=(32, 32, 3)))
    model.add(layers.MaxPooling2D((2, 2)))
    model.add(layers.Conv2D(32, (3, 3), activation='relu'))
    model.add(layers.MaxPooling2D((2, 2)))
    
    # Flatten and add fully connected layers
    model.add(layers.Flatten())
    model.add(layers.Dense(120, activation='relu'))
    model.add(layers.Dense(10)) # 10 output classes
    
    return model

model = create_simple_cnn()
model.summary()

By stacking these simple layers, CNNs learn a hierarchy of features. They start with basic lines and curves and build up to complex concepts, allowing them to make remarkably accurate predictions about the visual world.

Ready to test your knowledge?

Quiz Questions 1/5

How does a computer perceive a digital image, such as a photo of a cat?

Quiz Questions 2/5

What is the primary role of a 'kernel' or 'filter' in a convolutional layer?

With these foundational concepts, you can begin to explore the powerful capabilities of Convolutional Neural Networks.

Advanced Deep Learning Architectures Beyond NLP

Advanced Deep Learning Architectures Beyond NLP

Convolutional Neural Networks

Seeing with Math

The Layers of a CNN

Building a CNN in Code