A healthy model server can still be busy or idle. Enabling the llama.cpp server metrics endpoint exposes Prometheus-compatible token counters and throughput gauges so inference activity can be measured separately from service health.
The metrics endpoint is disabled by default and turns on when llama-server starts with --metrics. The endpoint lives at /metrics on the configured host and port, and it returns plain text in the Prometheus exposition format.
Token counters accumulate within a server process and reset when it restarts. A before-and-after comparison around one completion distinguishes an idle endpoint from one recording inference; other clients' requests also contribute to those totals.
Steps to enable llama.cpp server metrics:
- Enable metrics at server startup with --metrics.
The listener also serves inference requests and monitoring data. Binding to 127.0.0.1 limits access to the local machine; it does not add authentication. Restarting an existing server to add the flag interrupts its active requests.
$ llama-server -m models/chat-model.gguf --alias local-metrics --metrics --host 127.0.0.1 --port 8080 ##### snipped ##### 0.02.026.822 I srv server_run_i: model loaded 0.02.026.850 I srv server_on_li: listening on http://127.0.0.1:8080
The starting requirements are llama-server on the command path, curl, an existing GGUF model, and an unused port 8080. The model path models/chat-model.gguf represents the local model file. Host and port are explicit because defaults vary between builds. The server remains running in the foreground while requests come from a second terminal.
Related: How to start the llama.cpp server - Check that the model has finished loading.
$ curl --fail --silent --show-error http://127.0.0.1:8080/health {"status":"ok"}
A loading model can return HTTP 503. The --fail option makes HTTP errors produce a nonzero curl exit status; HTTP 200 with status set to ok indicates readiness for this single-model server.
Related: How to check llama.cpp server health - Read the token counters before sending a completion.
$ curl --fail --silent --show-error http://127.0.0.1:8080/metrics # HELP llamacpp:prompt_tokens_total Number of prompt tokens processed, excluding cached tokens # TYPE llamacpp:prompt_tokens_total counter llamacpp:prompt_tokens_total 0 ##### snipped ##### # HELP llamacpp:tokens_predicted_total Number of generation tokens processed # TYPE llamacpp:tokens_predicted_total counter llamacpp:tokens_predicted_total 0 ##### snipped #####
The counters start at zero on a fresh server without inference traffic. Without --metrics, the endpoint returns HTTP 501 with type set to not_supported_error. Router mode requires /metrics?model=MODEL_ID for a particular model; plain /metrics is the single-model form.
- Send a completion request through the metrics-enabled server.
$ curl --fail --silent --show-error --request POST http://127.0.0.1:8080/completion \ --header 'Content-Type: application/json' \ --data '{"prompt":"Write one short sentence about monitoring.","n_predict":32,"temperature":0.2,"seed":42}' {"index":0,"content":" To monitor is to observe and record the state of something over time. \n\nThis sentence effectively captures the essence of monitoring by highlighting its purpose and the process involved in","tokens":[],"id_slot":0,"stop":true,"model":"local-metrics","tokens_predicted":32,"tokens_evaluated":7, ##### snipped #####
The native /completion endpoint returns generated text and token counts. Text and counts vary with the model; n_predict caps generation at 32 tokens, so the response can end mid-sentence. The shortened response shows 7 evaluated prompt tokens and 32 generated tokens.
- Confirm that both token counters increased on the same server process.
$ curl --fail --silent --show-error http://127.0.0.1:8080/metrics # HELP llamacpp:prompt_tokens_total Number of prompt tokens processed, excluding cached tokens # TYPE llamacpp:prompt_tokens_total counter llamacpp:prompt_tokens_total 7 ##### snipped ##### # HELP llamacpp:tokens_predicted_total Number of generation tokens processed # TYPE llamacpp:tokens_predicted_total counter llamacpp:tokens_predicted_total 32 ##### snipped ##### # HELP llamacpp:predicted_tokens_seconds Average generation throughput in tokens/s # TYPE llamacpp:predicted_tokens_seconds gauge llamacpp:predicted_tokens_seconds 8.26679 ##### snipped #####
The prompt counter rose from 0 to 7 and the generated-token counter from 0 to 32, matching the completion response. The prompt counter excludes cached tokens; llamacpp:prompt_tokens_cached_total records reuse separately. Throughput is a gauge, so it need not increase with each request. A successful metrics response with unchanged token counters does not establish that the request reached this process.