At the heart of nearly every modern Large Language Model (LLM) lies an architecture called the Transformer. Introduced in a 2017 paper titled "Attention Is All You Need," it marked a fundamental shift from previous designs. Before the Transformer, models processed text sequentially, like reading a sentence one word at a time. This created a bottleneck, making it difficult to capture long-range dependencies in text.

This ability to handle data in parallel, combined with its sophisticated context-awareness, is what allows models like GPT and Claude to generate such coherent and relevant text. It's the engine that powers the generative AI tools we use today.

Before a model can process text, it must first break it down into manageable pieces called tokens and convert them into numbers. A token can be a word, a part of a word (like 'ing'), or even just a single character. This process, called tokenization, allows the model to handle a vast vocabulary with a finite set of building blocks.

Once tokenized, each token is mapped to a high-dimensional vector known as an embedding. This isn't just a random ID; the embedding is a rich numerical representation that captures the token's semantic meaning. Words with similar meanings will have similar embedding vectors, allowing the model to understand relationships like "king" is to "queen" as "man" is to "woman".

However, since the Transformer processes all tokens at once, it has no inherent sense of their order. To solve this, it uses positional encoding. A unique vector, based on the token's position in the sequence, is added to each token's embedding. This gives the model crucial information about word order without slowing it down with sequential processing.

The core mechanism of the Transformer is self-attention. Think of it as a way for the model to create a 'cheat sheet' for each word in a sentence. For every single word, self-attention looks at all the other words in the sentence and decides how important each one is for understanding that specific word's context. When processing the word "it" in the sentence "The dog chased the ball because it was fast," attention helps the model figure out that "it" refers to the "dog," not the "ball."

To do this, the model creates three vectors for each token embedding: a Query (Q), a Key (K), and a Value (V).

• Query: Represents the current word's question, asking "Who am I, and what should I pay attention to?"
• Key: Represents the other words' labels, announcing "This is what I am."
• Value: Represents the actual meaning or substance of the other words.

The model calculates an attention score by matching the Query of the current word with the Keys of all other words. A high score means a strong relevance. These scores are then used to create a weighted sum of all the Value vectors, producing a new vector for the current word that is rich with contextual information from the entire sentence.

The original Transformer had two main parts: an encoder and a decoder. However, modern LLMs often use just one of these components, depending on their primary goal.

Understanding these distinctions is key to selecting the right tool. If your task is to understand and categorise existing text, an encoder-based model is likely the best fit. If you need to generate new text, a decoder-based model is the way to go.

One of the most discussed topics around LLMs is their size, often measured in billions of parameters. A parameter is essentially a value that the model learns during training. Think of them as the knobs and dials the model tunes to minimise its error.

The relationship between size, data, and performance is described by scaling laws. In general, as you increase the number of parameters and the amount of training data, the model's performance improves predictably. This discovery is what fuelled the race to build ever-larger models.

However, bigger isn't always better. Extremely large models are expensive to train and run, consume vast amounts of energy, and can sometimes be outperformed by smaller, more specialised models that have been fine-tuned on high-quality, domain-specific data. The trend is now shifting towards finding the optimal balance between size, efficiency, and capability.

By understanding these core components—from tokens and embeddings to attention and architectural choices—you can move beyond simply using LLMs and start to appreciate how they work, enabling you to select and apply them more effectively.