How to refresh a chatbot knowledge base in LlamaIndex

A retrieval-backed chatbot can keep serving an outdated answer after its source file changes because the stored index still contains the earlier document nodes. Refreshing the existing LlamaIndex index updates changed documents while leaving unchanged knowledge-base files alone.

Stable file-based IDs come from SimpleDirectoryReader when filename_as_id=True. On later runs, refresh_ref_docs() compares those IDs and document hashes with the persisted docstore, then returns one Boolean result for each input document.

The keyless smoke test uses MockEmbedding and one FAQ file so the retrieved text is easy to inspect. A deployed chatbot needs its existing embedding model and storage directory, while a long-running process needs a reload before it can serve the refreshed index object.

1. Create the knowledge_base directory for the chatbot source files.

   $ mkdir knowledge_base

2. Write the current support answer to /knowledge_base/support-faq.txt.

   knowledge_base/support-faq.txt

   Question: Which support email should the chatbot use?
   Answer: Use support-old@example.com for all support requests.

3. Start /refresh_chatbot_kb.py with the imports, paths, and keyless embedding model.

   refresh_chatbot_kb.py

   from pathlib import Path
    
   from llama_index.core import (
       MockEmbedding,
       Settings,
       SimpleDirectoryReader,
       StorageContext,
       VectorStoreIndex,
       load_index_from_storage,
   )
    
    
   KNOWLEDGE_BASE_DIR = Path("knowledge_base")
   STORAGE_DIR = Path("storage")
    
   Settings.embed_model = MockEmbedding(embed_dim=8)

   MockEmbedding is limited to the keyless smoke test. A deployed refresh job needs the embedding model that created its stored vectors because a different model can make old and new vectors incompatible.

4. Add stable file-based document loading below the settings block.

   def load_documents():
       return SimpleDirectoryReader(
           input_dir=str(KNOWLEDGE_BASE_DIR),
           filename_as_id=True,
       ).load_data()

   filename_as_id=True keeps the document ID tied to its file path so a later load identifies the edited FAQ as the same source document.

5. Define load_or_refresh_index() below load_documents().

   def load_or_refresh_index():
       documents = load_documents()
    
       if not STORAGE_DIR.exists():
           index = VectorStoreIndex.from_documents(documents)
           changed = [True] * len(documents)
           action = "created"
       else:
           storage_context = StorageContext.from_defaults(
               persist_dir=str(STORAGE_DIR)
           )
           index = load_index_from_storage(storage_context)
           changed = index.refresh_ref_docs(documents)
           action = "refreshed"
    
       index.storage_context.persist(persist_dir=str(STORAGE_DIR))
       return index, action, changed

6. Define retrieve_answer() below load_or_refresh_index().

   def retrieve_answer(index):
       retriever = index.as_retriever(similarity_top_k=1)
       nodes = retriever.retrieve(
           "Which support email should the chatbot use?"
       )
       return nodes[0].node.get_content().strip()

7. Add the script entry point below retrieve_answer().

   index, action, changed = load_or_refresh_index()
   print(f"Index action: {action}")
   print(f"Changed documents: {changed}")
   print(retrieve_answer(index))

8. Run /refresh_chatbot_kb.py to create the first persisted index.

   $ python3 refresh_chatbot_kb.py
   Index action: created
   Changed documents: [True]
   Question: Which support email should the chatbot use?
   Answer: Use support-old@example.com for all support requests.

9. Replace the support answer in /knowledge_base/support-faq.txt.

   knowledge_base/support-faq.txt

   Question: Which support email should the chatbot use?
   Answer: Use support-new@example.com for all support requests.

10. Execute /refresh_chatbot_kb.py again to confirm the persisted index contains the changed answer.

    $ python3 refresh_chatbot_kb.py
    Index action: refreshed
    Changed documents: [True]
    Question: Which support email should the chatbot use?
    Answer: Use support-new@example.com for all support requests.

    [True] means the input document was inserted or refreshed. An unchanged document returns False on its next run.