Traditional computers build a wall between their main processor (CPU) and graphics processor (GPU). The CPU uses system RAM for its tasks, while the GPU has its own dedicated pool of video RAM, or VRAM. When you want to run a large AI model, the model's data must first be loaded into RAM and then copied over to the GPU's VRAM. This copy operation happens over a connection called the PCIe bus, which acts as a bottleneck, slowing everything down. If a model is too big for the GPU's VRAM, it simply won't run, or will run painfully slowly by swapping data back and forth.

Apple Silicon chips tear down this wall. Their Unified Memory Architecture (UMA) creates a single, high-bandwidth pool of memory that both the CPU and GPU can access directly. There is no separate VRAM and no slow bus to cross. Data doesn't need to be copied; both processors work from the same source. This design is a core part of Apple's System on a Chip (SoC) philosophy, which integrates all essential components onto a single piece of silicon.

This has a massive impact on local AI. A MacBook with 32 GB of unified memory can dedicate nearly all of that to a model, treating it as VRAM. A comparable PC laptop might have a powerful GPU, but be limited by its 8 GB or 12 GB of VRAM, making it impossible to run the same large models locally.

Beyond UMA, newer Apple Silicon chips like the M-series family include another powerful tool for AI: specialized accelerators within the GPU cores themselves. These aren't the same as the main Neural Engine (NPU), which handles a variety of machine learning tasks. Instead, these GPU accelerators are fine-tuned for the types of calculations common in large language models, leading to significant speed-ups. In some cases, they can accelerate prompt processing by up to four times compared to previous generations.

Memory bandwidth is the other crucial part of the equation. This measures how quickly data can move between the processors and memory. Higher bandwidth allows the model's parameters to be fed to the GPU faster, which in turn allows the GPU to generate tokens—the words and phrases of the AI's response—more quickly. This is why a chip with higher memory bandwidth can often feel much more responsive when interacting with a local LLM.

How much memory do you actually need? You can estimate the VRAM requirement for a model with a simple calculation. The key factors are the number of parameters (measured in billions) and the model's quantization level.

A model's parameters are typically stored as 16-bit floating-point numbers (FP16), which take up 2 bytes each. So, a 7-billion parameter model would initially require:

$7 \text{ billion parameters} \times 2 \frac{\text{bytes}}{\text{parameter}} = 14 \text{ GB}$

This is where quantization comes in. An 8-bit quantized model (INT8) uses 1 byte per parameter, cutting the size in half. A 4-bit model (INT4) uses just half a byte per parameter, reducing the original size by 75%.

This table gives you a good starting point. Remember to leave some memory for the operating system and other applications—a good rule of thumb is to keep at least 4-6 GB free. Thanks to the Unified Memory Architecture, you can run models that would be out of reach for most dedicated GPUs, all on a slim laptop.

Now, let's review the key concepts we've covered.