A Large Language Model doesn't read text; it performs a mathematical analysis of it. The first step is to break down a sentence into manageable pieces. This process, called tokenization, converts strings of text into a sequence of tokens. These tokens can be words, parts of words, or even just punctuation.

Once the text is tokenized, each token is converted into a high-dimensional vector of numbers known as an embedding. Think of these embeddings as coordinates on a vast, multidimensional map of meaning. Words with similar meanings, like "happy" and "joyful," are located close to each other in this space. This numerical representation is what allows the model to work with language mathematically.

With text converted into a list of meaningful vectors, the model needs a way to figure out which words are important to which other words. This is the job of the self-attention mechanism, the core innovation of the Transformer architecture.

Consider the sentence: "The delivery truck blocked the driveway because it was too wide." For a human, it's obvious that "it" refers to the "truck." Self-attention allows the model to make the same connection. When processing the word "it," the model assigns a high "attention score" to "truck," effectively linking them and understanding the context.

This is accomplished through the Query-Key-Value (QKV) framework. For every token in the input, the model generates three separate vectors:

To calculate the attention for a given token, its Query vector is compared with the Key vector of every other token in the sequence. This comparison produces a score that determines how much focus to place on each of those other tokens. These scores are then used to create a weighted average of all the Value vectors. The result is a new vector for the original token that is now enriched with contextual information from the entire sentence.

Transformer models generally come in two flavors: encoder-only and decoder-only. Encoder-only models, like BERT, are designed for tasks that require understanding the full context of a sentence, such as sentiment analysis or text classification. They look at the text from both directions (left-to-right and right-to-left) at once.

Decoder-only models, like the GPT series, are built for generation. They process text sequentially from left to right, predicting the next token based on the ones that came before. This makes them ideal for tasks like writing emails, summarizing articles, or creating chatbot responses.

Both architectures are constrained by a critical limitation: the context window. This is the maximum number of tokens the model can consider at one time. If a conversation or document exceeds this limit, the model starts to forget what was said at the beginning. Information from the earliest parts of the text is dropped to make room for new tokens.

This is why a chatbot might lose the thread of a long conversation or a summarization tool might ignore points made in the first few paragraphs of a lengthy report. Managing and working within this fixed window is a key challenge in applying LLMs effectively.

Ready to test your knowledge? Let's see how well you've grasped these core mechanisms.

Understanding these internal mechanics helps you move from being a casual user to a strategic operator, capable of predicting a model's behavior and recognizing its architectural limits.