Container replacement should not mean downloading every language model again. The official Ollama Docker image can run a CPU server with a named volume for model storage and an API reachable from the Docker host.

Docker Engine or Docker Desktop must already be running, and the Docker CLI must have access to it. Docker Engine 28 or newer avoids an older localhost-port exposure issue. No host Ollama installation is needed because the image supplies the server and model runner.

The named volume survives container removal; data kept only in a container's writable layer does not. Reusing the same volume makes downloaded models available to a replacement container.

Steps to run Ollama with Docker:

  1. Check whether the container name ollama is already in use.
    $ docker ps -a --filter 'name=^/ollama$'

    An empty table means the name is available. An existing container must be accounted for before starting another with that name; a stopped container still reserves its name.

  2. Create the named Docker volume ollama for persistent model storage.
    $ docker volume create ollama
    ollama

    Docker reuses a named volume that already exists. The volume is mounted at /root/.ollama inside the container and remains after the container is removed. Deleting the volume deletes its stored models.

  3. Start the official Ollama container with a loopback-only host port.

    Ollama's local API does not require authentication. Publishing to all host interfaces can let other machines send requests; the 127.0.0.1 mapping limits this host port to local clients.

    $ docker run -d \
      -v ollama:/root/.ollama \
      -p 127.0.0.1:11434:11434 \
      --name ollama \
      ollama/ollama:latest

    The image starts the server automatically. Port 11434 must be free on the Docker host; an existing native Ollama server may already occupy it. This CPU path does not request GPU devices.

  4. Verify that the API answers through the Docker host port.
    $ curl --fail --silent --show-error http://127.0.0.1:11434/api/version
    {"version":"0.40.12"}

    A version response confirms that the server is reachable. Connection refusal immediately after launch can mean the server is still starting; a stopped container or failed port mapping also prevents a response. The version number depends on the image release.

  5. Pull the small qwen2.5:0.5b model into the container's mounted volume.
    $ docker exec ollama ollama pull qwen2.5:0.5b

    The first pull downloads about 398 MB of model data and requires internet access. An existing copy can reuse its layers; larger models require more disk space and memory.

  6. Run a one-shot model prompt inside the container.
    $ docker exec ollama ollama run qwen2.5:0.5b \
      "What is 17 plus 25? Answer in one short sentence."
    17 plus 25 equals 42.

    The wording can vary. A generated answer shows that the model can run, rather than merely appear in the model list.

  7. Confirm that a client on the Docker host can generate a model reply through the API.
    $ curl --fail --silent --show-error http://127.0.0.1:11434/api/generate \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "qwen2.5:0.5b",
        "prompt": "What is 17 plus 25? Answer in one short sentence.",
        "stream": false
      }'

    The JSON response should contain the generated answer in response and done set to true. A model-not-found error means this server cannot find the requested tag; a connection error means the host cannot reach the API. The detached container keeps serving after the request finishes.

Share this guide