How to build a hybrid retriever in LlamaIndex

Embedding search can miss an exact identifier such as an error code, while keyword search misses a passage that answers the question in different words. A hybrid retriever in LlamaIndex runs both methods over the same nodes and merges their ranked lists, so one result set keeps the hits that each method would miss alone.

The QueryFusionRetriever class merges those lists with reciprocal rank fusion, which scores a node by its position in each list instead of by its raw score. BM25 and cosine scores sit on different scales, so position is the part they can share, and a node found by both retrievers rises above a node found by only one.

Query generation is the only part of QueryFusionRetriever that calls a language model, so turning it off keeps the retriever local and free of API keys. BM25 runs in the Python process and a Sentence Transformers model supplies the embeddings, although the first run needs network access to download that model.

1. Install the BM25 and Hugging Face integrations in the active virtual environment.

   $ python -m pip install \
     llama-index-core \
     llama-index-retrievers-bm25 \
     llama-index-embeddings-huggingface

   These packages pull in PyTorch, and the finished environment took about 6 GB of disk space on Linux arm64. The BM25 package keeps PyStemmer below version 3, and those releases have no wheel for Python 3.14 or for Linux arm64, so pip compiles it on either one. If that build stops on a missing compiler or Python.h, install a C compiler and the Python headers, which are the build-essential and python3-dev packages on Ubuntu, then run the command again. Tested with llama-index-core 0.14.25 and llama-index-retrievers-bm25 0.8.0 on Python 3.13 and Python 3.14.
   Related: How to install LlamaIndex with pip

2. Create a file named hybrid_search.py in your text editor with the following imports and nodes.

   hybrid_search.py

   from llama_index.core import VectorStoreIndex
   from llama_index.core.llms import MockLLM
   from llama_index.core.retrievers import QueryFusionRetriever
   from llama_index.core.schema import TextNode
   from llama_index.embeddings.huggingface import (
       HuggingFaceEmbedding,
   )
   from llama_index.retrievers.bm25 import BM25Retriever
    
   nodes = [
       TextNode(
           id_="chargebacks",
           text=(
               "Support refunds a disputed E1043 payment "
               "by filing a chargeback."
           ),
       ),
       TextNode(
           id_="release-notes",
           text=(
               "Version 4.2 adds dark mode, faster CSV exports, "
               "and the new E1043 timeout code."
           ),
       ),
       TextNode(
           id_="reimbursements",
           text=(
               "Reimbursements reach the customer's original card "
               "within five business days."
           ),
       ),
       TextNode(
           id_="returns-window",
           text=(
               "Unused goods can be sent back "
               "within 30 days of delivery."
           ),
       ),
       TextNode(
           id_="customs-delays",
           text=(
               "Parcels held at customs usually clear "
               "within three working days."
           ),
       ),
   ]

   Both retrievers read this one list. The fusion step merges hits that have identical content, so a node returned by both retrievers counts as one result with two ranks. To use your own files, load them into documents, split those with a node parser, and pass the returned nodes instead of this list.
   Related: How to load local files with SimpleDirectoryReader in LlamaIndex
   Related: How to configure a text splitter in LlamaIndex

3. Add the following vector index and retriever at the end of hybrid_search.py.

   embed_model = HuggingFaceEmbedding(
       model_name="sentence-transformers/all-MiniLM-L6-v2"
   )
   index = VectorStoreIndex(nodes, embed_model=embed_model)
   vector_retriever = index.as_retriever(similarity_top_k=2)

   Each retriever returns its two best nodes here, which is enough for the two lists to differ on this small set of nodes.

4. Add the following BM25 retriever at the end of hybrid_search.py.

   bm25_retriever = BM25Retriever.from_defaults(
       nodes=nodes,
       similarity_top_k=2,
   )

   BM25 fills its list even when a node shares no word with the query, returning that node with a score of 0.0, and fusion still gives it credit for its rank.

5. Add the following fusion retriever at the end of hybrid_search.py.

   fusion_retriever = QueryFusionRetriever(
       [vector_retriever, bm25_retriever],
       llm=MockLLM(),
       mode="reciprocal_rerank",
       similarity_top_k=3,
       num_queries=1,
   )

   The mode value selects reciprocal rank fusion, because the default simple mode keeps each retriever's raw score, and similarity_top_k caps the merged list. Setting num_queries to 1 searches with the original query alone, while the default of 4 asks a language model for three more. MockLLM fills the llm argument and is never called, and leaving it out makes the constructor load Settings.llm, which defaults to OpenAI and fails without that package or an API key.

6. Add the following comparison at the end of hybrid_search.py.

   query = "How do I refund an E1043 payment?"
    
    
   def show(label, retriever):
       print(label)
       hits = retriever.retrieve(query)
       for rank, hit in enumerate(hits, start=1):
           print(f"{rank}. {hit.node_id:<15} {hit.score:.5f}")
    
    
   show("Vector", vector_retriever)
   show("BM25", bm25_retriever)
   show("Fused", fusion_retriever)

7. Save hybrid_search.py.
8. Run the script to print all three result lists.

   $ python hybrid_search.py
   ##### snipped #####
   Vector
   1. chargebacks     0.79589
   2. reimbursements  0.39964
   BM25
   1. chargebacks     1.62136
   2. release-notes   0.31835
   Fused
   1. chargebacks     0.03333
   2. reimbursements  0.01639
   3. release-notes   0.01639

   The snipped lines can include a Hugging Face warning about unauthenticated requests, which is harmless for this public model.

9. Confirm that the Fused list contains the keyword-only release-notes node and the vector-only reimbursements node.

   Each list gives a node 1/(60 + rank), and the top place counts as rank 0. The chargebacks node earns 1/60 from both lists for 0.03333, while each single-list hit earns 1/61, which is 0.01639.