Computers don't understand language the way we do. To a machine, words like "apple" and "orange" are just sequences of characters. They have no inherent concept of fruit, sweetness, or color. To work with language, a Large Language Model (LLM) first needs to translate text into a format it understands: numbers.

This translation process creates something called an embedding. An embedding is a list of numbers—a vector—that represents a piece of text. It could be a single word, a sentence, or an entire document. Think of it like giving every word or phrase its own unique set of coordinates on a giant map.

These numerical coordinates aren't random. They're carefully calculated to capture the meaning and context of the words. On this conceptual map, words with similar meanings are placed close together. The coordinate for "cat" would be near "kitten," and "running" would be near "jogging."

More impressively, embeddings capture the relationships between words. The distance and direction from "man" to "woman" on the map would be very similar to the distance and direction from "king" to "queen." The model learns these analogies mathematically.

This concept extends beyond single words. The embedding for the sentence "The dog is chasing the ball" would be located in a similar region of the map as "A puppy ran after its toy." The LLM understands that these sentences, despite using different words, describe a very similar idea.

LLMs aren't explicitly taught these relationships. Instead, they learn them by analyzing enormous amounts of text from the internet, books, and other sources during a process called pre-training.

By observing which words frequently appear together and in what contexts, the model starts to build its map. It learns that words like "coffee" and "tea" often show up near words like "cup," "hot," and "drink." Over billions of examples, it creates a complex, multi-dimensional space where the position of each word or phrase reflects its learned meaning.

Once this map is built, it can be used for a wide range of tasks. For example, a search engine can use embeddings to find documents that are conceptually related to your query, not just ones that contain the exact keywords. A recommendation system can suggest articles based on the similarity of their embeddings to articles you've already read. Embeddings are the foundation that allows LLMs to work with language in a meaningful way.

That's the core idea behind embeddings. They are the bridge between human language and the mathematical world of computers.