Generative models are a fascinating area of machine learning. Instead of just classifying data or making predictions, these models learn the underlying patterns of a dataset so well that they can create entirely new data that fits those patterns. Think of an artist who studies thousands of portraits from a specific historical period. Eventually, they can paint a new portrait that looks like it belongs to that era, even though the person in the painting never existed. That's the essence of a generative model.

These models can be used to generate images, music, text, and even code. Let's look at a few key types of generative models that paved the way for today's advanced AI.

A Variational Autoencoder, or VAE, has two main parts: an encoder and a decoder. The encoder's job is to take a piece of data, like an image, and compress it into a much simpler representation. This compressed version, called the latent space, captures the most essential features of the original data. You can think of it like creating a detailed sketch that highlights the key characteristics of a photograph.

The decoder then takes that simplified representation from the latent space and tries to reconstruct the original image. By training the encoder and decoder together, the VAE gets very good at identifying and rebuilding the fundamental properties of the training data. To generate a new image, we can simply pick a random point in the latent space and feed it to the decoder. The decoder will then create a novel image that shares the characteristics of the original dataset.

Generative Adversarial Networks, or GANs, take a different approach. A GAN consists of two neural networks that compete with each other in a game. The first network is the Generator. Its job is to create fake data. The second network is the Discriminator. Its job is to look at data—some real, some fake from the generator—and decide which is which.

Imagine a forger trying to create a perfect replica of a famous painting, and an art critic trying to spot the fakes. At first, the forger is clumsy, and the critic easily spots the fakes. But with each attempt, the forger gets feedback and improves. In response, the critic has to get better at finding more subtle flaws. This cycle continues until the forger becomes so skilled that the critic can no longer reliably tell the difference. At that point, the Generator is producing high-quality, realistic data.

More recently, a new type of generative model has gained popularity: the diffusion model. These models have become known for creating stunningly detailed and coherent images. The core idea is inspired by thermodynamics.

The process starts with a clear image and systematically adds a small amount of noise over many steps, until the image becomes pure, unrecognizable static. The model is then trained to do the reverse: starting with just noise, it learns how to gradually remove the noise, step by step, to reconstruct a clean image. To generate a new image, the model simply starts with a random field of noise and applies this learned denoising process.

VAEs, GANs, and diffusion models are all powerful tools for generating new content. Each has its own strengths and weaknesses, but they all share the common goal of learning the essence of a dataset to create novel, realistic examples.

Ready to check your understanding?

Now that you have a foundational understanding of what generative models are and the different types that exist, you're ready to dive deeper into how diffusion models work their magic.