A healthy model server can still be busy or idle. Enabling the llama.cpp server metrics endpoint exposes Prometheus-compatible token counters and throughput gauges so inference activity can be measured separately from service health.

The metrics endpoint is disabled by default and turns on when llama-server starts with --metrics. The endpoint lives at /metrics on the configured host and port, and it returns plain text in the Prometheus exposition format.

Token counters accumulate within a server process and reset when it restarts. A before-and-after comparison around one completion distinguishes an idle endpoint from one recording inference; other clients' requests also contribute to those totals.

Steps to enable llama.cpp server metrics:

  1. Enable metrics at server startup with --metrics.

    The listener also serves inference requests and monitoring data. Binding to 127.0.0.1 limits access to the local machine; it does not add authentication. Restarting an existing server to add the flag interrupts its active requests.

    $ llama-server -m models/chat-model.gguf --alias local-metrics --metrics --host 127.0.0.1 --port 8080
    ##### snipped #####
    0.02.026.822 I srv  server_run_i: model loaded
    0.02.026.850 I srv  server_on_li: listening on http://127.0.0.1:8080

    The starting requirements are llama-server on the command path, curl, an existing GGUF model, and an unused port 8080. The model path models/chat-model.gguf represents the local model file. Host and port are explicit because defaults vary between builds. The server remains running in the foreground while requests come from a second terminal.
    Related: How to start the llama.cpp server

  2. Check that the model has finished loading.
    $ curl --fail --silent --show-error http://127.0.0.1:8080/health
    {"status":"ok"}

    A loading model can return HTTP 503. The --fail option makes HTTP errors produce a nonzero curl exit status; HTTP 200 with status set to ok indicates readiness for this single-model server.
    Related: How to check llama.cpp server health

  3. Read the token counters before sending a completion.
    $ curl --fail --silent --show-error http://127.0.0.1:8080/metrics
    # HELP llamacpp:prompt_tokens_total Number of prompt tokens processed, excluding cached tokens
    # TYPE llamacpp:prompt_tokens_total counter
    llamacpp:prompt_tokens_total 0
    ##### snipped #####
    # HELP llamacpp:tokens_predicted_total Number of generation tokens processed
    # TYPE llamacpp:tokens_predicted_total counter
    llamacpp:tokens_predicted_total 0
    ##### snipped #####

    The counters start at zero on a fresh server without inference traffic. Without --metrics, the endpoint returns HTTP 501 with type set to not_supported_error. Router mode requires /metrics?model=MODEL_ID for a particular model; plain /metrics is the single-model form.

  4. Send a completion request through the metrics-enabled server.
    $ curl --fail --silent --show-error --request POST http://127.0.0.1:8080/completion \
      --header 'Content-Type: application/json' \
      --data '{"prompt":"Write one short sentence about monitoring.","n_predict":32,"temperature":0.2,"seed":42}'
    {"index":0,"content":" To monitor is to observe and record the state of something over time. \n\nThis sentence effectively captures the essence of monitoring by highlighting its purpose and the process involved in","tokens":[],"id_slot":0,"stop":true,"model":"local-metrics","tokens_predicted":32,"tokens_evaluated":7,
    ##### snipped #####

    The native /completion endpoint returns generated text and token counts. Text and counts vary with the model; n_predict caps generation at 32 tokens, so the response can end mid-sentence. The shortened response shows 7 evaluated prompt tokens and 32 generated tokens.

  5. Confirm that both token counters increased on the same server process.
    $ curl --fail --silent --show-error http://127.0.0.1:8080/metrics
    # HELP llamacpp:prompt_tokens_total Number of prompt tokens processed, excluding cached tokens
    # TYPE llamacpp:prompt_tokens_total counter
    llamacpp:prompt_tokens_total 7
    ##### snipped #####
    # HELP llamacpp:tokens_predicted_total Number of generation tokens processed
    # TYPE llamacpp:tokens_predicted_total counter
    llamacpp:tokens_predicted_total 32
    ##### snipped #####
    # HELP llamacpp:predicted_tokens_seconds Average generation throughput in tokens/s
    # TYPE llamacpp:predicted_tokens_seconds gauge
    llamacpp:predicted_tokens_seconds 8.26679
    ##### snipped #####

    The prompt counter rose from 0 to 7 and the generated-token counter from 0 to 32, matching the completion response. The prompt counter excludes cached tokens; llamacpp:prompt_tokens_cached_total records reuse separately. Throughput is a gauge, so it need not increase with each request. A successful metrics response with unchanged token counters does not establish that the request reached this process.

Share this guide