Advanced Query Transformation: HyDE (Hypothetical Document Embeddings) & Multi-Query Decomposition

1. The Query-Document Asymmetry Problem

In standard RAG, short, informal user queries (e.g., "Why did my pod crash with 137?") are compared against lengthy, formal documentation chunks (e.g., "Kubernetes OOMKilled SIGKILL Exit Code 137 occurs when cgroup memory limits are breached..."). Because the user query and document chunk have completely different lengths, linguistic structures, and vocabulary distributions, standard vector cosine similarity often performs poorly.

User Query: "Why did my pod crash with 137?"
    │
    ▼ (LLM generates hypothetical answer)
Hypothetical Document (HyDE): 
"Exit code 137 indicates that a Linux process received SIGKILL, typically triggered 
by the Kubernetes OOM (Out Of Memory) killer when the container exceeded memory limit..."
    │
    ▼ (Embed the hypothetical answer instead of the query!)
Dense Vector ──► Vector Database Search ──► Accurate Production Docs Retrieved

2. Multi-Query Expansion & Sub-Question Decomposition

For complex queries requiring multi-step reasoning (e.g., "Compare Stripe and PayPal transaction fees for cross-border SaaS companies"):

1. Multi-Query Expansion: An LLM generates 3–5 linguistic variations of the query to broaden search recall.
2. Sub-Question Decomposition: Breaks multi-part queries into atomic single-intent questions:
   • Query 1: "Stripe cross-border international transaction fees"
   • Query 2: "PayPal international merchant payment processing rates"
   • Query 3: "SaaS recurring billing payment gateway fee comparison"

3. Production Query Decomposition Implementation

from typing import List

async def decompose_complex_query(llm_client, user_query: str) -> List[str]:
    prompt = f"""Break down the following complex user query into 2 to 4 atomic sub-questions 
suitable for vector search retrieval. Output one question per line without numbering.

User Query: {user_query}"""

    response = await llm_client.generate(prompt)
    sub_queries = [line.strip() for line in response.split("\n") if line.strip()]
    return sub_queries or [user_query]