To understand how computers get things done, let's think about making deliveries. If you need to deliver one large, fragile, and complicated piece of equipment across town, you'd use a single, powerful van. It’s fast, specialised, and handles the complex job efficiently. This is like a Central Processing Unit, or CPU.

A CPU has a few very powerful cores, designed to tackle complex tasks one after another in a sequence. This is called serial execution. It excels at jobs where each step depends on the one before it, like loading an operating system or running a web browser.

Now, imagine you need to deliver 10,000 identical letters to 10,000 different houses in the same neighbourhood. Sending one van to each house, one by one, would be incredibly slow. It would be much smarter to hire a fleet of 10,000 bicycle couriers. Each courier takes one letter and they all pedal off at the same time. This is the core idea of parallelism—doing many similar things at once.

This is where the Graphics Processing Unit, or GPU, shines. A GPU is like that fleet of bicycles. It has thousands of smaller, simpler cores. This is often called a Many-Core architecture. While each core isn't as fast or complex as a CPU core, their combined power for parallel tasks is enormous. This makes GPUs perfect for jobs that can be broken down into thousands of identical, independent pieces, like rendering graphics in a video game (calculating the colour of millions of pixels at once) or training an artificial intelligence model.

This difference in design leads to two important performance metrics: latency and throughput.

CPUs are latency-oriented. They are built to minimise the delay for a single job. The delivery van has low latency: it gets that one package to its destination as fast as possible.

GPUs are throughput-oriented. They are built to maximise the total amount of work done. The fleet of bicycles has high throughput: while any single letter delivery might be slower than the van, the total number of letters delivered per hour is vastly higher.

So, why are we talking about vans and bicycles? Because understanding this fundamental difference is the key to unlocking the power of GPU acceleration with platforms like CUDA. CUDA is a programming model created by NVIDIA that lets us tell the GPU how to perform general-purpose calculations, not just graphics.

By learning CUDA, you are learning how to be the dispatcher for that fleet of thousands of bicycles. You'll learn how to take a big computational problem, break it into tiny, identical pieces, and assign each piece to a GPU core to be solved in parallel. This is why GPUs have become essential for today's most demanding tasks, from developing deep learning models to simulating complex weather systems and discovering new medicines.

This core concept of serial versus parallel processing is the foundation for everything we'll build upon.