Hybrid Search (Dense + Sparse) & Reciprocal Rank Fusion (RRF)

1. Why Pure Vector Search Fails in Production

Pure dense vector embeddings excel at semantic similarity (e.g., matching "automobile malfunction" with "car broken down"), but frequently fail on:

1. Exact keyword queries, part numbers, SKUs (e.g., CVE-2024-38077, ERR_CONN_REFUSED).
2. Rare domain terms and acronyms.
3. Specific identifiers (e.g., user IDs, function names).

Hybrid Search combines Sparse Lexical Search (BM25 / SPLADE) with Dense Vector Retrieval (HNSW).

2. Reciprocal Rank Fusion (RRF) Algorithm

Rather than trying to normalize and blend fundamentally incomparable score scales (e.g., cosine similarity $[0, 1]$ vs. unbounded BM25 scores $[0, \infty)$), Reciprocal Rank Fusion (RRF) scores documents strictly by their ordinal ranking position in each result list:

$$\text{RRF Score}(d) = \sum_{m \in M} \frac{1}{k + r_m(d)}$$

Where:

• $M$ is the set of retrieval systems (Dense and Sparse).
• $r_m(d)$ is the 1-based rank position of document $d$ in system $m$.
• $k$ is a smoothing constant (standard production default: $k = 60$).

3. Python Implementation of Hybrid Search with RRF

from typing import List, Dict

def reciprocal_rank_fusion(dense_results: List[str], sparse_results: List[str], k: int = 60, top_n: int = 5) -> List[Dict]:
    rrf_scores: Dict[str, float] = {}

    # Accumulate dense ranks
    for rank, doc_id in enumerate(dense_results, start=1):
        rrf_scores[doc_id] = rrf_scores.get(doc_id, 0.0) + (1.0 / (k + rank))

    # Accumulate sparse ranks
    for rank, doc_id in enumerate(sparse_results, start=1):
        rrf_scores[doc_id] = rrf_scores.get(doc_id, 0.0) + (1.0 / (k + rank))

    # Sort documents by accumulated RRF score
    sorted_docs = sorted(rrf_scores.items(), key=lambda item: item[1], reverse=True)
    return [{"doc_id": doc_id, "score": score} for doc_id, score in sorted_docs[:top_n]]

# Example test
dense_hits = ["doc_A", "doc_B", "doc_C"]
sparse_hits = ["doc_C", "doc_A", "doc_D"]

final_top = reciprocal_rank_fusion(dense_hits, sparse_hits, k=60, top_n=3)
# doc_A and doc_C appear in both lists and receive elevated fused ranking