Several clients can reach a local model at once, but the server decides how many requests run together. On Linux, Ollama request parallelism belongs in the systemd service configuration so the limit survives service restarts.
Parallel requests share a loaded model but need additional context memory. A limit of two is a modest starting point for a small model; increasing it does not guarantee faster responses, especially when requests compete for CPU time or GPU memory.
Ollama must already run as a systemd service, with sudo access, curl, and the small qwen2.5:0.5b model available. The service-edit method requires systemd 256 or newer, and the overlapping-request check needs two terminals on the server.
Steps to set Ollama request parallelism:
- Inspect the Ollama service unit and its existing drop-ins.
$ sudo systemctl cat ollama.service
The service environment controls the server process; an export in a client terminal does not change an already running service. The dedicated parallelism.conf below must be new. If an existing drop-in already sets OLLAMA_NUM_PARALLEL, its owning file needs updating instead of a conflicting second definition.
Related: How to manage the Ollama service - Set a two-request limit in the new parallelism.conf drop-in.
$ sudo systemctl edit --drop-in=parallelism.conf --stdin ollama.service <<'EOF' [Service] Environment="OLLAMA_NUM_PARALLEL=2" EOF
OLLAMA_NUM_PARALLEL limits simultaneous requests per model. systemctl edit reloads systemd's configuration automatically and creates this drop-in.
/etc/systemd/system/ollama.service.d/parallelism.conf
The standard-input form replaces the named drop-in's contents, which is why the file must be new.
- Restart Ollama to apply the parallelism limit.
A restart interrupts active requests and unloads models, so a shared server needs a quiet period. If the new limit causes memory pressure, this rollback restores the previous configuration without removing other overrides.
$ sudo rm /etc/systemd/system/ollama.service.d/parallelism.conf $ sudo systemctl daemon-reload $ sudo systemctl restart ollama
$ sudo systemctl restart ollama
- Confirm that the restarted API responds.
$ curl -fsS --retry 5 --retry-connrefused --retry-delay 1 http://127.0.0.1:11434/api/version {"version":"0.40.12"}
The version varies with the installed release. A brief connection-refused message can appear while the service starts.
- Check the parallelism value in the service environment.
$ systemctl show ollama -p Environment --value PATH=/bin:/sbin:/usr/bin:/usr/sbin:. OLLAMA_NUM_PARALLEL=2
Other environment entries vary by installation. The effective assignment must include OLLAMA_NUM_PARALLEL=2.
- Start a streaming generation request in the first terminal.
$ curl -NfsS http://127.0.0.1:11434/api/generate \ -H 'Content-Type: application/json' \ -d '{"model":"qwen2.5:0.5b","prompt":"List twenty different animals, one per line.","stream":true,"options":{"num_ctx":2048,"num_predict":128}}'
-N displays chunks as they arrive. The request uses a 2048-token context and caps generation at 128 tokens. The model must already be installed.
Related: How to pull an Ollama model - Send a second request from the other terminal while the first is still generating.
$ curl -NfsS http://127.0.0.1:11434/api/generate \ -H 'Content-Type: application/json' \ -d '{"model":"qwen2.5:0.5b","prompt":"List twenty different trees, one per line.","stream":true,"options":{"num_ctx":2048,"num_predict":128}}'
Each JSON chunk includes created_at. Response chunks from both requests before either request's final chunk show overlapping generation. A second response that starts only after the first finishes does not demonstrate overlap.
- Check both streams for response text and a final “done”:true chunk.
Text appears in response fields, and the final chunk has done set to true. An error field or a curl failure means the request failed. Generated lists and timings vary; the two requests are a concurrency check rather than a throughput benchmark.
- Verify that the loaded runner has two parallel sequence slots.
$ sudo journalctl -u ollama --since today -b --no-pager --grep=n_seq_max Oct 28 12:26:11 host ollama[2141]: llama_context: n_seq_max = 2
The recent runner entry must show n_seq_max = 2 for the model that handled the requests. Together with overlapping response timestamps and two completed streams, this confirms the service applied the limit. Older entries can show previous settings; a model that does not support parallel processing may use one slot.
Related: How to view Ollama logs