Computers don't understand words. They understand numbers. So how can a large language model know that "cat" is more similar to "kitten" than it is to "car"? The answer is embeddings.

An embedding is a way to translate text into a list of numbers, called a vector. This isn't a random translation. The vector captures the semantic meaning, or the underlying concept, of the original text. Think of it like a coordinate system for ideas. Just as a GPS coordinate pinpoints a location on a map, an embedding vector pinpoints a concept in a vast "meaning space."

Each number in the vector represents a different abstract feature of the text's meaning. We don't know exactly what each feature corresponds to—it could be anything from tone to topic to grammatical role—but a model learns these features by analyzing huge amounts of text. A piece of text might be represented by a vector with hundreds or thousands of dimensions.

For example, a short phrase might be converted into a vector like this:

Because these vectors represent meaning, the distance between them is significant. Vectors for concepts that are closely related will be close together in this meaning space. The vectors for "king" and "queen" will be much closer to each other than the vector for "bicycle."

OpenAI provides models specifically designed to create high-quality embeddings for text and code. Instead of generating language like a chat model, these models output a vector for any given input string.

One of the most widely used is text-embedding-ada-002. It's powerful because it can handle different types of text, from single words to entire documents, and it performs well across a wide range of tasks. It balances cost, speed, and capability, making it a go-to choice for many applications.

These models are designed to be general-purpose. They weren't trained on one specific task, but on the general structure and meaning of language. This makes their output useful for many different downstream applications without needing to be retrained.

Once you have a way to turn words into numbers that represent meaning, you can do some amazing things. Since similar concepts have similar vectors, we can use math to measure how related two pieces of text are.

Here are some common applications:

In each case, the core idea is the same: by converting text to embeddings, we can use mathematical distance to approximate semantic similarity. This simple but powerful technique is a building block for many of the sophisticated AI tools we use today.

Now let's test your understanding of embeddings.

Embeddings are a foundational concept in modern NLP. By representing text as numerical vectors, they allow machines to work with the meaning of language, enabling a new class of intelligent applications.