Local chat clients often expect the OpenAI Chat Completions request shape even when the model runs on the same workstation. The llama.cpp chat completions API accepts those messages over HTTP and returns an assistant answer from a local GGUF model.

The server exposes /v1/chat/completions and returns a chat.completion object with a choices array. The model field identifies the loaded model by its ID from /v1/models. A configured server alias gives that model a stable name for client requests.

An existing chat-capable server listening only on the local machine, curl, and Python 3 are the starting requirements. The unauthenticated loopback connection keeps prompts on the local machine; it does not protect a server exposed to other computers.

Steps to call the llama.cpp chat completions API:

  1. Check that the local chat server has finished loading.
    $ curl --fail --silent --show-error http://127.0.0.1:8080/health
    {"status":"ok"}

    A loading model can return HTTP 503. The --fail option makes HTTP errors produce a nonzero curl exit status.
    Related: How to check llama.cpp server health
    Related: How to start the llama.cpp server

  2. List the model IDs available to the chat client.
    $ curl --fail --silent --show-error http://127.0.0.1:8080/v1/models
    {"models":[{"name":"local-chat","model":"local-chat","modified_at":"","size":"","digest":"","type":"model","description":"","tags":[""],"capabilities":["completion"],"parameters":"","details":{"parent_model":"","format":"gguf","family":"","families":[""],"parameter_size":"","quantization_level":""}}],"object":"list","data":[{"id":"local-chat","aliases":["local-chat"],"tags":[],"object":"model","architecture":{"input_modalities":["text"],"output_modalities":["text"]},"created":1793146028,"owned_by":"llamacpp","meta":{"vocab_type":2,"n_vocab":151936,"n_ctx":2048,"n_ctx_train":32768,"n_embd":896,"n_params":630167424,"size":485452288,"ftype":"Q4_K - Medium"}}]}

    The id in the data array is local-chat here. Without an alias, the model ID may be the model file path.

  3. Save the messages in chat-request.json using the discovered model ID.
    chat-request.json
    {
      "model": "local-chat",
      "messages": [
        {
          "role": "system",
          "content": "Reply in one short sentence."
        },
        {
          "role": "user",
          "content": "What is two plus two?"
        }
      ],
      "max_tokens": 64,
      "temperature": 0.2
    }

    The system message sets the response style, and the user message asks the question. The max_tokens value limits generated tokens rather than words.

  4. Submit chat-request.json to the chat completions endpoint.
    $ curl --fail --silent --show-error http://127.0.0.1:8080/v1/chat/completions \
      --header 'Content-Type: application/json' \
      --data-binary @chat-request.json \
      --output chat-response.json \
      --write-out 'HTTP %{http_code}\n'
    HTTP 200

    The JSON response is saved in chat-response.json. No Authorization header is needed for this local server without API-key checking.

  5. Validate the saved response fields before printing the assistant answer.
    $ python3 - <<'PYTHON'
    import json
    with open("chat-response.json") as response_file:
        response = json.load(response_file)
    message = response["choices"][0]["message"]
    assert response["object"] == "chat.completion"
    assert response["model"] == "local-chat"
    assert message["role"] == "assistant" and message["content"].strip()
    print(message["content"])
    print("finish_reason:", response["choices"][0]["finish_reason"])
    PYTHON
    Two plus two is four.
    finish_reason: stop

    The assertions reject an unexpected object, model, role, or empty answer. If the discovered model ID differs, the model comparison must match the ID in chat-request.json. The wording varies by model and prompt. A finish_reason of stop means generation ended normally; length means the token limit stopped it and the answer may be incomplete.

Share this guide