How to connect LlamaIndex to Ollama

Local model development often starts with the question of whether application code can reach the model server before any retrieval, agent, or prompt logic is added. Connecting LlamaIndex to Ollama gives a Python project a direct path to a local chat model exposed by the Ollama API.

The Ollama LLM integration lives in the llama-index-llms-ollama package and provides the Ollama class used by LlamaIndex LLM calls. The class sends requests to http://127.0.0.1:11434 in this smoke test, selects the model name from the OLLAMA_MODEL environment variable, and keeps the generated output short so a connection check finishes quickly.

Start with the Ollama server already running and at least one chat model visible in ollama list. A short script that prints both the selected model and the returned text separates endpoint or model-name problems from later LlamaIndex index, retriever, or agent code.

1. Check that Ollama has a local chat model available.

   $ ollama list
   NAME                ID              SIZE      MODIFIED
   tinyllama:latest    2644915ede35    637 MB    27 seconds ago

   Use a model name from your own ollama list output. If no model appears, start the server and pull a model before configuring LlamaIndex.
   Related: How to start the Ollama server
   Related: How to pull an Ollama model

2. Install the LlamaIndex Ollama integration in the active Python environment.

   $ python3 -m pip install --upgrade llama-index-llms-ollama
   Successfully installed llama-index-core-0.14.23 llama-index-llms-ollama-0.10.1 ollama-0.6.2

   llama-index-llms-ollama installs the Ollama LLM wrapper and the Python Ollama client used by the script. Use a project virtual environment when the system Python environment is shared.
   Related: How to install LlamaIndex with pip

3. Set the model name for the smoke test.

   $ export OLLAMA_MODEL=tinyllama:latest

   Replace tinyllama:latest with the exact NAME value from ollama list. Use a small non-reasoning model for the first connection check when a larger reasoning model spends the output budget on hidden reasoning.

4. Create a LlamaIndex script that points at the local Ollama endpoint.

   $ cat > llamaindex-ollama.py <<'PY'
   import os
    
   from llama_index.llms.ollama import Ollama
    
   model = os.environ["OLLAMA_MODEL"]
    
   llm = Ollama(
       model=model,
       base_url="http://127.0.0.1:11434",
       request_timeout=120.0,
       context_window=2048,
       temperature=0,
       keep_alive=0,
       additional_kwargs={"num_predict": 96},
   )
    
   response = llm.complete(
       "Write one short sentence confirming that LlamaIndex reached Ollama."
   )
    
   print(f"model={model}")
   print(f"response={str(response).strip()}")
   PY

   request_timeout allows slow local model startup, context_window limits memory use for the probe, and num_predict caps the response length sent to Ollama.

5. Run the script and confirm LlamaIndex receives a response from Ollama.

   $ python3 llamaindex-ollama.py
   model=tinyllama:latest
   response=LlamaIndex has successfully reached Ollama!

6. Remove the temporary smoke-test script after copying the working Ollama settings into the application.

   $ rm llamaindex-ollama.py