Have you ever started a sentence and had a friend finish it for you? In a way, that's what large language models are doing all the time. At their core, they are built to master one fundamental task: predicting the next piece of text in a sequence.

In machine learning, we don't just work with words. We break text down into smaller units called tokens. A token can be a full word like “apple,” a part of a word like “un” or “-able,” or even just a punctuation mark like a comma. This allows the model to handle a vast vocabulary and understand the structure of words themselves.

Next-token prediction is the process of figuring out which token is most likely to come next, given all the tokens that came before it. It’s a game of probabilities. If a model sees the sequence, “The sun is shining and the sky is,” it analyzes the context and calculates the probability for every possible token that could follow.

The token “blue” would get a very high probability score. “Clear” would also score highly. Words like “falling” or “green” would receive extremely low scores because they don’t make sense in this context. The model then picks the most probable token to continue the sentence.

This simple-sounding task is the engine that drives nearly all modern language AI. By repeatedly predicting the next token, a model can generate text one piece at a time. It predicts a token, adds it to the sequence, and then uses this new, longer sequence to predict the next token. This loop is what allows a model to write an email, compose a poem, or answer a question.

This process is called inference. It's how a trained model puts its knowledge to use to create new text. Each token it generates becomes part of the context for the next one, creating a chain of predictions that forms a coherent response.

Next-token prediction isn't just an academic exercise. It powers many of the tools we use daily.

• Autocomplete: When your phone suggests the next word as you type a text message, it's using a simple form of next-token prediction.
• Translation: Translation models predict the next token in the target language based on the source text and the translated text so far.
• Chatbots and Assistants: When you ask a digital assistant a question, it generates its answer one token at a time, always trying to predict the most helpful and relevant next piece of the response.
• Code Generation: AI coding assistants predict the next token of code, helping developers write functions, fix bugs, and complete lines more quickly.

Understanding this core concept is the first step to seeing how complex AI systems can generate text that is often indistinguishable from what a human would write.

Ready to check your understanding?