How to build a FAQ knowledge base for a chatbot in LlamaIndex

Support answers become easier to audit when each approved question and response is stored as a separate retrieval unit. A small LlamaIndex knowledge base can keep refund, account, and billing replies tied to the FAQ entry that supplied their context.

This local smoke test uses three Document objects with stable faq_id metadata. A deterministic keyword embedding chooses the matching entry, while a local completion model returns its answer without an API key or hosted model account.

The in-memory index is suitable for proving the FAQ structure and chatbot handoff before selecting production embedding, language-model, and vector-store integrations. The final transcript includes the answer, matched FAQ ID, and retrieved source text so a wrong match remains visible.

1. Install the LlamaIndex core package in the active Python environment.

   $ python3 -m pip install --upgrade llama-index-core

   A project virtual environment keeps this package separate from the shared system Python environment.
   Related: How to install LlamaIndex with pip

2. Start faq_chatbot.py with the imports and approved FAQ documents.

   faq_chatbot.py

   import re
   from typing import Any, List
    
   from pydantic import Field
    
   from llama_index.core import Document, Settings, VectorStoreIndex
   from llama_index.core.embeddings import BaseEmbedding
   from llama_index.core.llms import (
       CompletionResponse,
       CompletionResponseGen,
       CustomLLM,
       LLMMetadata,
   )
   from llama_index.core.llms.callbacks import llm_completion_callback
   from llama_index.core.node_parser import SentenceSplitter
    
    
   faq_documents = [
       Document(
           text=(
               "Question: How do I reset my password?\n"
               "Answer: Use the account recovery page to send a reset link to the "
               "verified email address on the account."
           ),
           metadata={"faq_id": "password-reset"},
       ),
       Document(
           text=(
               "Question: What is the refund window?\n"
               "Answer: Customers can request a refund within 30 days of cancellation "
               "from the billing portal."
           ),
           metadata={"faq_id": "refund-window"},
       ),
       Document(
           text=(
               "Question: How do I change my billing email?\n"
               "Answer: Open Billing, choose Payment details, and update the billing "
               "contact email before the next invoice."
           ),
           metadata={"faq_id": "billing-email"},
       ),
   ]

   Each Document contains one question-and-answer pair, so the retrieved faq_id identifies one approved support response.

3. Add the deterministic FAQ embedding class below faq_documents.

   class FAQKeywordEmbedding(BaseEmbedding):
       """Small deterministic embedding for a local FAQ smoke test."""
    
       keywords: List[str] = Field(
           default_factory=lambda: [
               "refund",
               "cancel",
               "password",
               "billing",
               "email",
           ]
       )
    
       def _embed(self, text: str) -> List[float]:
           text_lower = text.lower()
           return [float(text_lower.count(keyword)) for keyword in self.keywords]
    
       def _get_query_embedding(self, query: str) -> List[float]:
           return self._embed(query)
    
       async def _aget_query_embedding(self, query: str) -> List[float]:
           return self._get_query_embedding(query)
    
       def _get_text_embedding(self, text: str) -> List[float]:
           return self._embed(text)
    
       def _get_text_embeddings(self, texts: List[str]) -> List[List[float]]:
           return [self._embed(text) for text in texts]

   The keyword vector makes the smoke test repeatable; replace it with the application's embedding integration after the FAQ flow works.

4. Add the local answer model below FAQKeywordEmbedding.

   class FAQAnswerLLM(CustomLLM):
       """Completion test double that returns the retrieved FAQ answer."""
    
       context_window: int = 2048
       num_output: int = 128
       model_name: str = "faq-answer-test-double"
    
       @property
       def metadata(self) -> LLMMetadata:
           return LLMMetadata(
               context_window=self.context_window,
               num_output=self.num_output,
               model_name=self.model_name,
           )
    
       @llm_completion_callback()
       def complete(self, prompt: str, **kwargs: Any) -> CompletionResponse:
           match = re.search(r"Answer:\s*(.+)", prompt)
           answer = match.group(1).strip() if match else "No matching FAQ answer found."
           return CompletionResponse(text=answer)
    
       @llm_completion_callback()
       def stream_complete(self, prompt: str, **kwargs: Any) -> CompletionResponseGen:
           response = self.complete(prompt).text
           current = ""
           for token in response.split():
               current = f"{current} {token}".strip()
               yield CompletionResponse(text=current, delta=f"{token} ")

   FAQAnswerLLM extracts the retrieved Answer: line for the credential-free test; use the chatbot's production LLM integration for generated answers.

5. Append the LlamaIndex settings and FAQ query path below FAQAnswerLLM.

   Settings.llm = FAQAnswerLLM()
   Settings.embed_model = FAQKeywordEmbedding()
   Settings.transformations = [SentenceSplitter(chunk_size=256, chunk_overlap=0)]
    
   index = VectorStoreIndex.from_documents(faq_documents)
   question = "Can I get a refund after cancelling?"
   retriever = index.as_retriever(similarity_top_k=1)
   matched_node = retriever.retrieve(question)[0].node
    
   chat_engine = index.as_chat_engine(chat_mode="context", similarity_top_k=1)
   response = chat_engine.chat(question)
    
   print(f"question={question}")
   print(f"answer={str(response).strip()}")
   print(f"matched_faq={matched_node.metadata['faq_id']}")
   print("source=" + matched_node.text.replace("\n", " "))

   similarity_top_k=1 keeps this FAQ test limited to its best match. Broader support answers may need multiple retrieved entries and a production prompt that cites each source.

6. Run the completed FAQ chatbot.

   $ python3 faq_chatbot.py
   question=Can I get a refund after cancelling?
   answer=Customers can request a refund within 30 days of cancellation from the billing portal.
   matched_faq=refund-window
   source=Question: What is the refund window? Answer: Customers can request a refund within 30 days of cancellation from the billing portal.

   The answer and source should both belong to refund-window; a different matched_faq exposes a retrieval error before the chatbot is connected to users.