How to filter documents in a Haystack document store

Metadata can keep unrelated records out of a retrieval path before ranking or generation begins. Haystack document stores apply that boundary where the documents live, returning only the records that satisfy the requested labels.

The filter_documents() method accepts comparison dictionaries built from field, operator, and value keys. A logical dictionary places those comparisons in conditions and uses an operator such as AND when every condition must match.

An InMemoryDocumentStore keeps the demonstration local and makes the filter result easy to inspect. The records include one exact match and two near misses, while the final assertions confirm both the selected ID and its metadata; other document-store integrations may support a different operator set.

1. Create document_store_filter_demo.py with an in-memory store and three records that define the selection boundary.

   document_store_filter_demo.py

   from haystack import Document
   from haystack.document_stores.in_memory import InMemoryDocumentStore
    
    
   document_store = InMemoryDocumentStore()
    
   documents = [
       Document(
           id="apac-release-note",
           content="Publish the APAC release note.",
           meta={"region": "apac", "status": "approved"},
       ),
       Document(
           id="emea-release-note",
           content="Publish the EMEA release note.",
           meta={"region": "emea", "status": "approved"},
       ),
       Document(
           id="apac-release-draft",
           content="Review the APAC release draft.",
           meta={"region": "apac", "status": "draft"},
       ),
   ]

2. Append the write call and the two-condition AND filter after the document list.

   document_store.write_documents(documents)
    
   filters = {
       "operator": "AND",
       "conditions": [
           {"field": "meta.region", "operator": "==", "value": "apac"},
           {"field": "meta.status", "operator": "==", "value": "approved"},
       ],
   }

   The EMEA record matches only the status condition, and the APAC draft matches only the region condition.

3. Append the filtered query and fail-capable assertions after the filter dictionary.

   matched_documents = document_store.filter_documents(filters=filters)
   matched_ids = sorted(document.id for document in matched_documents)
    
   assert document_store.count_documents() == 3
   assert matched_ids == ["apac-release-note"], matched_ids
   assert all(
       document.meta["region"] == "apac"
       and document.meta["status"] == "approved"
       for document in matched_documents
   )
    
   print(f"stored documents: {document_store.count_documents()}")
   print(f"matched documents: {len(matched_documents)}")
   print(f"matched ids: {', '.join(matched_ids)}")

   The assertions stop execution if the store count, returned ID, or returned metadata no longer matches the intended boundary.

4. Run the completed document-store filter program.

   $ python3 document_store_filter_demo.py
   stored documents: 3
   matched documents: 1
   matched ids: apac-release-note