How to call the llama.cpp chat completions API

Local chat clients often expect the OpenAI Chat Completions request shape even when the model runs on the same workstation. A running llama-server can accept that JSON over HTTP, which makes it possible to test a local GGUF model with the endpoint shape many SDKs already know.

The server exposes /v1/chat/completions and returns a chat.completion object with a choices array. The model field should match a loaded model ID from /v1/models or an alias supplied with --alias when llama-server was started.

The request body assumes llama-server is already listening on localhost:8080. Use the same JSON shape against another host only after its API key, firewall, and model alias are known, because a chat request can consume tokens even when the prompt is only a smoke test.

1. Check that the server is ready.

   $ curl http://localhost:8080/health
   {"status":"ok"}

   A loading server can return HTTP 503 with Loading model. Wait for /health to return ok before sending chat prompts.
   Related: How to check llama.cpp server health

2. List the loaded model IDs.

   $ curl http://localhost:8080/v1/models
   {"object":"list","data":[{"id":"local-chat","object":"model","owned_by":"llamacpp"}]}

   Use the id value in the chat request body. Without --alias, the model ID may be the model file path.

3. Create the chat request body.

   chat-request.json

   {
     "model": "local-chat",
     "messages": [
       {
         "role": "system",
         "content": "Reply in one short sentence."
       },
       {
         "role": "user",
         "content": "Say ready when the chat API is reachable."
       }
     ],
     "max_tokens": 16,
     "temperature": 0.2
   }

   Replace local-chat with the model ID returned by /v1/models. The system message sets behavior, and the user message is the prompt sent to the model.

4. Send the request to the chat completions endpoint.

   $ curl --request POST http://localhost:8080/v1/chat/completions \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer no-key' \
     --data @chat-request.json
   {"object":"chat.completion","model":"local-chat","choices":[{"message":{"role":"assistant","content":"To ensure that the chat API is reachable, we need to ensure that the"}}],"usage":{"completion_tokens":16,"prompt_tokens":31,"total_tokens":47}}

   Replace no-key with the configured key when llama-server was started with --api-key. The default local server accepts a placeholder bearer value or no Authorization header.

5. Confirm the response contains an assistant message.

   "object":"chat.completion"
   "model":"local-chat"
   "role":"assistant"

   The generated text varies by model and prompt. The API call worked when the response has object set to chat.completion and a choices entry with an assistant message.

6. Remove the temporary request body.

   $ rm chat-request.json