Query Transformation Strategies

Rethinking the Query

In a basic RAG system, a user's prompt is taken at face value. It's converted into an embedding and used to find similar document chunks. This works, but it's brittle. A user's phrasing might not perfectly match the language in your knowledge base, leading to poor retrieval.

Advanced RAG systems don't just accept the initial query. They transform it. By refining, expanding, or decomposing the user's prompt before it ever hits the vector database, you can dramatically improve the quality of the retrieved context. This isn't about simple prompt engineering; it's about systematically restructuring the query for optimal retrieval.

One Question, Many Angles

A single query often has multiple facets. A user asking, "What were the key challenges in the Apollo 11 mission?" could be interested in technical hurdles, political pressures, or logistical nightmares. A simple vector search might only catch documents that use the exact phrase "key challenges."

This is where Multi-Query Retrieval comes in. Instead of using the original query alone, you use an LLM to generate several variations of it. These aren't just rephrasings; they are different perspectives on the same core question.

# Pseudocode for Multi-Query Generation

original_query = "What were the key challenges in the Apollo 11 mission?"

# Use an LLM to generate related queries
generated_queries = llm.generate(
  prompt=f"Generate 3 alternative queries for: {original_query}",
  perspectives=["technical", "political", "logistical"]
)

# Example output from the LLM:
# 1. "Technical failures and engineering problems during Apollo 11."
# 2. "Political climate and Cold War pressures on the Apollo 11 mission."
# 3. "Supply chain and logistical planning for the moon landing."

Now you have multiple sets of search results, one for each query. How do you combine them effectively? Simply concatenating them can be messy. A better approach is RAG-Fusion, which merges the results using a method called Reciprocal Rank Fusion (RRF).

RRF looks at the rank of each document across all result sets. It gives more weight to documents that appear consistently high in the rankings, regardless of their raw similarity scores. This approach is powerful because a document relevant to multiple perspectives of a query is likely very important.

RRFScore(d) = \sum_{q \in Q} \frac{1}{k + \text{rank}_q(d)}

Breaking Down Complexity

Some questions can't be answered in a single step. These are often called "multi-hop" questions because they require synthesizing information from different sources. For example, consider the query: "What company acquired the developer of the game Chrono Trigger, and what was that developer's most recent release?"

Asking this question directly to a simple RAG system will likely fail. It needs to find two distinct pieces of information. Sub-query Decomposition solves this by using an LLM to break the complex query into simpler, standalone questions.

Who developed the game Chrono Trigger?
What company acquired that developer?
What was the most recent release from that developer?

Each sub-query is run against the retriever independently. The collected contexts are then passed to a final LLM call that synthesizes them into a single, coherent answer.

Another powerful technique is Step-Back Prompting. Instead of breaking a query down, you take a step back to ask a broader, more conceptual question. If a user asks, "Could a C++ std::vector be used to implement a priority queue?", the retrieval might pull up specific implementation details that miss the bigger picture.

A step-back prompt generated by an LLM might be, "What are the underlying data structures and time complexities for priority queues?" Retrieving documents that answer this higher-level question provides crucial context. When this general context is combined with the context from the original, specific query, the LLM can generate a much more insightful and complete answer.

Answering Your Own Question

Sometimes the biggest challenge is the semantic gap between a concise user query and a verbose, detailed document. A user might search for "benefits of RRF," while the relevant document explains the algorithm in detail without ever using that exact phrase.

Hypothetical Document Embeddings (HyDE) offers a clever solution. It flips the retrieval process on its head. First, it uses an LLM to generate a hypothetical, ideal answer to the user's query. This generated document is a fiction, but it's rich with the kind of language, terminology, and concepts you'd expect to find in a real answer.

Then, instead of creating an embedding of the user's query, it creates an embedding of this hypothetical document. This new embedding is then used to search the vector database. The system finds actual documents that are semantically similar to the ideal answer it just imagined. This method effectively translates the user's intent into the language of the knowledge base, bridging the semantic gap.

These query transformation strategies are not mutually exclusive. A production-grade system might decompose a query, generate multiple versions of each sub-query, and then use HyDE to improve retrieval for each one. By treating the user's prompt as a starting point rather than an endpoint, you can build RAG systems that are far more robust, accurate, and capable of handling true complexity.

Quiz Questions 1/6

What is the primary purpose of the Multi-Query Retrieval technique in an advanced RAG system?

Quiz Questions 2/6

The Hypothetical Document Embeddings (HyDE) technique creates an embedding of the user's initial query to search the vector database.

Mastering Advanced RAG Architectures