Applications built around OpenAI's chat interface can also send prompts to a local model. Ollama's compatibility layer lets the Python OpenAI client obtain an assistant reply without switching to Ollama's native request format.
The client chooses its destination through its base URL. Local Ollama accepts a harmless placeholder where the SDK requires an API key; that value does not authenticate requests or protect the server from other callers.
A running local Ollama server, an installed chat model, and Python 3.10 or newer with virtual-environment support are required. The Python client and server must be reachable from the same machine for a loopback address to work; compatibility covers a subset of OpenAI features rather than every hosted service capability.
Steps to use an OpenAI-compatible client with Ollama:
- Confirm that qwen2.5:0.5b is available on the local Ollama server.
$ ollama listThe model name must appear in the list. A different installed chat model needs its exact name in the Python request. Related: How to start the Ollama server
Related: How to pull an Ollama model - Create a Python virtual environment for the OpenAI client.
$ python3 -m venv .venv
The shell commands use POSIX virtual-environment paths. On Windows, the environment's Python executable is in its Scripts directory.
- Install the OpenAI SDK in the virtual environment.
$ .venv/bin/python -m pip install openai
- Create ollama-openai.py with a client pointed at local Ollama.
- ollama-openai.py
from openai import OpenAI client = OpenAI( base_url="http://127.0.0.1:11434/v1/", api_key="ollama", )
The SDK adds the chat-completions path to the base URL, so the base ends at /v1/. The literal placeholder overrides any hosted key in the environment. A local Ollama request needs no real OpenAI key.
- Append a chat-completions request to ollama-openai.py.
completion = client.chat.completions.create( model="qwen2.5:0.5b", messages=[ {"role": "user", "content": "What is 17 plus 25? Answer in one short sentence."} ], temperature=0, max_tokens=80, )
The request returns one complete response by default. The token limit leaves room for a short answer, and a temperature of zero reduces variation. The model name is an Ollama tag, rather than an OpenAI model name.
- Append a completed-reply check to ollama-openai.py.
choice = completion.choices[0] assert choice.finish_reason == "stop", completion assert choice.message.role == "assistant" and choice.message.content.strip(), completion print(choice.message.content)
The answer comes from the actual response's first choice. A missing answer or a token-limit finish stops the check instead of printing an assumed reply.
- Run ollama-openai.py to obtain the assistant's answer.
$ .venv/bin/python ollama-openai.py 17 plus 25 equals 42.
The wording can vary. A connection error points to the server address or listener; a model-not-found error points to a tag that is missing from that server. The printed answer confirms that the SDK reached Ollama and parsed a completed chat response.