A large language model, or LLM, is a type of artificial intelligence trained on vast amounts of text data. The "large" in its name refers to two things: the enormous size of the dataset it learns from and the huge number of parameters within the model itself. Parameters are like the knobs and dials the model tunes during training to make its predictions better. The more parameters, the more nuanced its understanding of language can become.

Think of it like the autocomplete on your phone, but supercharged. Instead of just suggesting the next word based on a few previous ones, an LLM understands grammar, context, facts, and even writing style from its training. This allows it to do much more than complete a sentence. Its fundamental purpose is to understand, interpret, and generate human-like text in a way that is coherent and relevant to a given prompt.

Most modern LLMs are built on an architecture called the Transformer, first introduced in a 2017 paper called "Attention Is All You Need." This design revolutionized how machines process language. Before the Transformer, models struggled to keep track of context in long sentences. They might forget what a pronoun referred to if it was too far from the noun.

The Transformer solved this with a powerful mechanism called attention. Attention allows the model to weigh the importance of different words in the input text when it's processing a particular word. It can look back or forward across the entire sentence to find relevant clues.

The Transformer architecture consists of two main parts: an encoder and a decoder. The encoder's job is to read and understand the input text, building a rich numerical representation of its meaning. The decoder then takes that representation and generates the output text, word by word, paying close attention to both the original input and the words it has already generated.

LLMs learn through a process called pre-training. During this phase, the model is fed a massive corpus of text from the internet, books, and other sources. It's not given explicit instructions on grammar or facts. Instead, it learns through a technique called self-supervised learning.

A common pre-training task is called masked language modeling. The model is given a sentence with a word missing, and its goal is to predict the correct word that fills the blank. For example, it might see: "The cat sat on the [MASK]."

By trying to predict the masked word ("mat," "couch," "floor"), the model begins to learn the relationships between words, sentence structures, and general knowledge about the world. When this process is repeated billions of times across a diverse dataset, the LLM develops a sophisticated internal representation of language.

After pre-training, an LLM is a powerful general-purpose tool for a wide range of natural language processing (NLP) tasks. Because it has learned the underlying patterns of language, it can be prompted to perform many different functions without needing to be retrained from scratch for each one.

These capabilities make LLMs a foundational technology in modern AI. They act as a base model that understands language broadly. This general understanding is the starting point for creating more specialized applications through a process called fine-tuning.