When comparing document embeddings, the choice of similarity metric is crucial. It’s not just about finding the “closest” vectors; it’s about defining what “close” means in the context of your task. The two most common metrics, dot product and cosine similarity, seem similar on the surface but measure different things. Their distinction lies in how they handle vector magnitude and direction.

The dot product measures how much one vector goes in the direction of another. It considers both the alignment (direction) and the lengths (magnitudes) of the two vectors. If you have two vectors, $\vec{a}$ and $\vec{b}$, their dot product is defined as:

A larger dot product means the vectors are not only pointing in a similar direction but are also long. In NLP, this means a longer document, which often has a larger vector magnitude, could be considered more similar to another long document, even if their topics are less aligned than a shorter, more focused document.

Cosine similarity, on the other hand, isolates the direction. It deliberately ignores the magnitude of the vectors by dividing the dot product by the product of their magnitudes. This process is called normalization and it forces every vector to a length of 1.

This focus on direction is why cosine similarity is the standard for semantic search. When comparing text, you usually care about the topic, not the word count. A short, concise article and a long, detailed dissertation on the same topic should be considered highly similar. Cosine similarity achieves this by ensuring that document length doesn't bias the relevance score.

So, when does magnitude matter? The dot product is useful when the magnitude of the vector carries meaningful information. For example, in a recommendation system, the magnitude of a user's vector might represent their level of engagement. A user who has rated many movies (a high-magnitude vector) might be a better match for another highly engaged user, even if their tastes are only moderately aligned.

In high-dimensional spaces where text embeddings live, our geometric intuition can be misleading. As the number of dimensions increases, vectors tend to become almost orthogonal (at 90-degree angles) to each other. The angle between vectors becomes a more reliable measure of similarity than Euclidean distance, as the concept of “distance” becomes less meaningful. This is another reason why angular-based metrics like cosine similarity are so effective for NLP tasks.

For practical purposes, if your vectors are already normalized to unit length, the dot product and cosine similarity become equivalent and computationally faster. The denominator in the cosine similarity formula becomes 1, leaving just the dot product.

Let's review the key terms we've covered.

Now, check your understanding of these concepts.

Ultimately, the choice depends on what you want similarity to mean for your specific problem. For most text analysis tasks, that meaning is purely semantic, making cosine similarity the default choice.