How to build a hybrid retriever in Haystack

Keyword search rewards exact terms, while embedding search can recover documents that express the same idea with different words. A hybrid retriever runs both searches for one query and fuses their rankings so neither signal has to carry the result alone.

Four short passages are embedded with sentence-transformers/all-MiniLM-L6-v2 and written to one in-memory document store. The query uses the same model, which keeps the stored vectors and query vector in the same embedding space.

The InMemoryBM25Retriever and InMemoryEmbeddingRetriever components feed separate candidate lists to DocumentJoiner. Its reciprocal_rank_fusion mode uses rank positions rather than incomparable raw BM25 and cosine scores, and the pipeline retains both branch outputs so the fusion can be checked directly.

1. Install the Haystack core and sentence-transformer integration packages in the active Python environment.

   $ python3 -m pip install haystack-ai sentence-transformers-haystack

   The first model load downloads sentence-transformers/all-MiniLM-L6-v2 from Hugging Face.
   Related: How to install Haystack with pip

2. Create haystack_hybrid_retriever.py with the imports and four source records.

   haystack_hybrid_retriever.py

   from haystack import Document, Pipeline
   from haystack.components.joiners import DocumentJoiner
   from haystack.components.retrievers.in_memory import (
       InMemoryBM25Retriever,
       InMemoryEmbeddingRetriever,
   )
   from haystack.document_stores.in_memory import InMemoryDocumentStore
   from haystack_integrations.components.embedders.sentence_transformers import (
       SentenceTransformersDocumentEmbedder,
       SentenceTransformersTextEmbedder,
   )
    
    
   documents = [
       Document(
           id="hybrid-search",
           content="Hybrid retrieval combines keyword matching with semantic search.",
       ),
       Document(
           id="bm25-keywords",
           content="BM25 finds exact terms such as product names and error codes.",
       ),
       Document(
           id="vector-meaning",
           content="Embedding search finds related meaning despite different wording.",
       ),
       Document(
           id="coffee-notes",
           content="Coffee brewing depends on water temperature and grind size.",
       ),
   ]

   The stable IDs allow the joiner to recognize the same document when both retrievers return it.

3. Append the document embedding and indexing stage to haystack_hybrid_retriever.py.

   model = "sentence-transformers/all-MiniLM-L6-v2"
   document_embedder = SentenceTransformersDocumentEmbedder(model=model)
   document_embedder.warm_up()
   embedded_documents = document_embedder.run(documents=documents)["documents"]
    
   document_store = InMemoryDocumentStore(
       embedding_similarity_function="cosine"
   )
   document_store.write_documents(embedded_documents)

   The document embedder and later query embedder must use the same model so cosine similarity compares compatible vectors.

4. Append the query components and reciprocal-rank-fusion joiner to haystack_hybrid_retriever.py.

   text_embedder = SentenceTransformersTextEmbedder(model=model)
   bm25_retriever = InMemoryBM25Retriever(
       document_store=document_store,
       top_k=2,
   )
   embedding_retriever = InMemoryEmbeddingRetriever(
       document_store=document_store,
       top_k=2,
   )
   joiner = DocumentJoiner(
       join_mode="reciprocal_rank_fusion",
       top_k=3,
   )

   Each retriever contributes at most two candidates, while the joiner keeps three unique documents after fusion.

5. Connect the text embedder and both retrieval branches in haystack_hybrid_retriever.py.

   pipeline = Pipeline()
   pipeline.add_component("text_embedder", text_embedder)
   pipeline.add_component("bm25_retriever", bm25_retriever)
   pipeline.add_component("embedding_retriever", embedding_retriever)
   pipeline.add_component("joiner", joiner)
   pipeline.connect(
       "text_embedder.embedding",
       "embedding_retriever.query_embedding",
   )
   pipeline.connect("bm25_retriever.documents", "joiner.documents")
   pipeline.connect("embedding_retriever.documents", "joiner.documents")

   The joiner's documents socket accepts both lists, while the embedding retriever receives the vector produced for the query.

6. Add the shared query and pipeline execution to haystack_hybrid_retriever.py.

   query = "hybrid retrieval keyword semantic"
   result = pipeline.run(
       data={
           "text_embedder": {"text": query},
           "bm25_retriever": {"query": query},
       },
       include_outputs_from={
           "bm25_retriever",
           "embedding_retriever",
       },
   )

   include_outputs_from retains the connected retrievers' candidate lists in addition to the joiner's leaf output.

7. Append branch, fusion, and printed-output checks to haystack_hybrid_retriever.py.

   bm25_ids = [
       doc.id for doc in result["bm25_retriever"]["documents"]
   ]
   embedding_ids = [
       doc.id for doc in result["embedding_retriever"]["documents"]
   ]
   joined_documents = result["joiner"]["documents"]
   joined_ids = [doc.id for doc in joined_documents]
    
   assert "hybrid-search" in bm25_ids
   assert "hybrid-search" in embedding_ids
   assert joined_ids[0] == "hybrid-search"
   assert "coffee-notes" not in joined_ids
    
   print(f"Indexed documents: {document_store.count_documents()}")
   print(f"BM25 candidates: {', '.join(bm25_ids)}")
   print(f"Embedding candidates: {', '.join(embedding_ids)}")
   print("Fused ranking:")
   for position, document in enumerate(joined_documents, start=1):
       print(f"{position}. {document.id} | {document.score:.4f}")

   The assertions fail if either branch misses the shared document, fusion moves it below first place, or the unrelated document enters the fused top three.

8. Compare the completed haystack_hybrid_retriever.py file with the assembled source.

   haystack_hybrid_retriever.py

   from haystack import Document, Pipeline
   from haystack.components.joiners import DocumentJoiner
   from haystack.components.retrievers.in_memory import (
       InMemoryBM25Retriever,
       InMemoryEmbeddingRetriever,
   )
   from haystack.document_stores.in_memory import InMemoryDocumentStore
   from haystack_integrations.components.embedders.sentence_transformers import (
       SentenceTransformersDocumentEmbedder,
       SentenceTransformersTextEmbedder,
   )
    
    
   documents = [
       Document(
           id="hybrid-search",
           content="Hybrid retrieval combines keyword matching with semantic search.",
       ),
       Document(
           id="bm25-keywords",
           content="BM25 finds exact terms such as product names and error codes.",
       ),
       Document(
           id="vector-meaning",
           content="Embedding search finds related meaning despite different wording.",
       ),
       Document(
           id="coffee-notes",
           content="Coffee brewing depends on water temperature and grind size.",
       ),
   ]
    
   model = "sentence-transformers/all-MiniLM-L6-v2"
   document_embedder = SentenceTransformersDocumentEmbedder(model=model)
   document_embedder.warm_up()
   embedded_documents = document_embedder.run(documents=documents)["documents"]
    
   document_store = InMemoryDocumentStore(
       embedding_similarity_function="cosine"
   )
   document_store.write_documents(embedded_documents)
    
   text_embedder = SentenceTransformersTextEmbedder(model=model)
   bm25_retriever = InMemoryBM25Retriever(
       document_store=document_store,
       top_k=2,
   )
   embedding_retriever = InMemoryEmbeddingRetriever(
       document_store=document_store,
       top_k=2,
   )
   joiner = DocumentJoiner(
       join_mode="reciprocal_rank_fusion",
       top_k=3,
   )
    
   pipeline = Pipeline()
   pipeline.add_component("text_embedder", text_embedder)
   pipeline.add_component("bm25_retriever", bm25_retriever)
   pipeline.add_component("embedding_retriever", embedding_retriever)
   pipeline.add_component("joiner", joiner)
   pipeline.connect(
       "text_embedder.embedding",
       "embedding_retriever.query_embedding",
   )
   pipeline.connect("bm25_retriever.documents", "joiner.documents")
   pipeline.connect("embedding_retriever.documents", "joiner.documents")
    
   query = "hybrid retrieval keyword semantic"
   result = pipeline.run(
       data={
           "text_embedder": {"text": query},
           "bm25_retriever": {"query": query},
       },
       include_outputs_from={
           "bm25_retriever",
           "embedding_retriever",
       },
   )
    
   bm25_ids = [
       doc.id for doc in result["bm25_retriever"]["documents"]
   ]
   embedding_ids = [
       doc.id for doc in result["embedding_retriever"]["documents"]
   ]
   joined_documents = result["joiner"]["documents"]
   joined_ids = [doc.id for doc in joined_documents]
    
   assert "hybrid-search" in bm25_ids
   assert "hybrid-search" in embedding_ids
   assert joined_ids[0] == "hybrid-search"
   assert "coffee-notes" not in joined_ids
    
   print(f"Indexed documents: {document_store.count_documents()}")
   print(f"BM25 candidates: {', '.join(bm25_ids)}")
   print(f"Embedding candidates: {', '.join(embedding_ids)}")
   print("Fused ranking:")
   for position, document in enumerate(joined_documents, start=1):
       print(f"{position}. {document.id} | {document.score:.4f}")

9. Verify both retrieval branches and the fused ranking by running haystack_hybrid_retriever.py.

   $ python3 haystack_hybrid_retriever.py
   Indexed documents: 4
   BM25 candidates: hybrid-search, bm25-keywords
   Embedding candidates: hybrid-search, vector-meaning
   Fused ranking:
   1. hybrid-search | 1.0000
   2. bm25-keywords | 0.4919
   3. vector-meaning | 0.4919

   hybrid-search appears in both branch lists and ranks first after fusion, while each branch's unique candidate remains in the fused top three.