How to start the llama.cpp server

The llama.cpp server turns a loaded GGUF model into a local HTTP service for browser tests, scripts, and OpenAI-compatible clients. Starting it with a clear model path, alias, and listener keeps later API calls pointed at the intended runtime instead of a stale process or wrong port.

A normal local launch uses llama-server with -m pointing to a GGUF file. The process stays in the foreground, loads the model, binds to localhost on port 8080 by default, and serves the health endpoint, model listing, web UI, and OpenAI-compatible API routes from the same listener.

Use a dedicated terminal or service manager for long-running use. Bind beyond localhost only when another machine or reverse proxy must reach it, and protect that listener at the network edge because llama.cpp does not turn local testing into an authenticated production API by itself.

1. Open a terminal on the host that has the llama-server binary and the GGUF model file.
2. Start llama-server with the model path and a client-facing alias.

   $ llama-server -m ./models/model.gguf --alias local-chat --ctx-size 2048
   ##### snipped #####
   srv  llama_server: model loaded
   srv  llama_server: listening on http://127.0.0.1:8080

   Replace ./models/model.gguf with the real GGUF path. The alias becomes the model name API clients send later; omit --alias when using the file path as the model ID is acceptable.

3. Check the health endpoint from a second terminal.

   $ curl http://127.0.0.1:8080/health
   {"status":"ok"}

   A 503 response with Loading model means llama-server is still loading the model. Wait for the load to finish before changing client settings.

4. List the model visible to OpenAI-compatible clients.

   $ curl http://127.0.0.1:8080/v1/models
   {"object":"list","data":[{"id":"local-chat","object":"model","owned_by":"llamacpp","meta":{"n_ctx":2048}}]}

5. Send a small chat request to prove the server accepts inference calls.

   $ curl http://127.0.0.1:8080/v1/chat/completions \
     --header "Content-Type: application/json" \
     --data '{"model":"local-chat","messages":[{"role":"user","content":"Reply with one short sentence."}],"max_tokens":32}'
   {"choices":[{"message":{"role":"assistant","content":"The local server is ready."}}],"model":"local-chat","object":"chat.completion","usage":{"total_tokens":40}}

   Use a chat-tuned model for /v1/chat/completions. If the loaded GGUF only supports plain completion, verify startup with /health and /v1/models first, then use the endpoint that matches the model.