A model's next-token choices depend on the sampling controls as well as the prompt. In llama.cpp, generation parameters on a completion request let a client adjust randomness, repetition penalties, and output length for that call.
The native /completion endpoint applies sampling overrides to the current generation task. A client can tune its own request without changing the server startup command or the defaults used by another client.
A fixed seed helps isolate a parameter change, but identical text is not guaranteed across different models, builds, or backends. The returned generation_settings shows the effective values; the generated content and token count show that the request also produced text.
Steps to set llama.cpp generation parameters:
- Start llama-server with a local text-generation GGUF model.
$ llama-server -m models/chat-model.gguf --alias local-chat --ctx-size 2048 --host 127.0.0.1 --port 8080
models/chat-model.gguf must point to an existing model file. An instruction model such as Qwen2.5-0.5B-Instruct can answer the robot-painter prompt; generated wording depends on the model. The request and response checks require curl and Python 3.
Related: How to start the llama.cpp server - Check that the local server has finished loading from another terminal.
$ curl -fsS http://127.0.0.1:8080/health {"status":"ok"}
The -f option makes an HTTP error fail the curl command. The loopback listener accepts local connections; exposing it to other computers requires separate access controls.
- Send a completion request with a temperature of 0.35.
$ curl -fsS http://127.0.0.1:8080/completion \ -H 'Content-Type: application/json' \ -d '{ "prompt": "Write one short sentence about a robot painter.", "n_predict": 40, "temperature": 0.35, "top_p": 0.80, "repeat_penalty": 1.08, "seed": 42, "stop": [ "\n" ] }' \ -o generation-cool.json
The response is saved in generation-cool.json. temperature controls randomness, top_p limits the candidate pool by cumulative probability, and repeat_penalty penalizes repeated tokens. n_predict caps generated tokens rather than words, while stop ends generation at a newline if one appears. Other sampler settings retain their server defaults.
- Send the same prompt with only temperature raised to 0.80.
$ curl -fsS http://127.0.0.1:8080/completion \ -H 'Content-Type: application/json' \ -d '{ "prompt": "Write one short sentence about a robot painter.", "n_predict": 40, "temperature": 0.80, "top_p": 0.80, "repeat_penalty": 1.08, "seed": 42, "stop": [ "\n" ] }' \ -o generation-warm.json
The prompt, seed, token budget, stop string, and other sampling values remain unchanged, so the comparison isolates the temperature change.
- Verify both responses contain text and the requested generation parameters.
$ python3 - <<'PYTHON' import json import math for name, temperature in [("cool", 0.35), ("warm", 0.80)]: with open("generation-" + name + ".json") as response_file: response = json.load(response_file) settings = response["generation_settings"] expected = {"temperature": temperature, "top_p": 0.80, "repeat_penalty": 1.08, "seed": 42, "n_predict": 40} assert all(math.isclose(settings[k], v, rel_tol=1e-6) for k, v in expected.items()) assert settings["stop"] == ["\n"] assert response["content"].strip() and response["tokens_predicted"] > 0 print(name, "temperature:", settings["temperature"]) print("top_p:", settings["top_p"], "repeat_penalty:", settings["repeat_penalty"]) print("seed:", settings["seed"], "n_predict:", settings["n_predict"]) print("content:", response["content"]) print("tokens_predicted:", response["tokens_predicted"], "stop_type:", response["stop_type"]) PYTHONcool temperature: 0.3499999940395355 top_p: 0.800000011920929 repeat_penalty: 1.0800000429153442 seed: 42 n_predict: 40 content: A robot painter can paint on walls and other surfaces using a variety of tools and techniques, including brushes, rollers, and spray paint. tokens_predicted: 28 stop_type: eos warm temperature: 0.800000011920929 top_p: 0.800000011920929 repeat_penalty: 1.0800000429153442 seed: 42 n_predict: 40 content: A robot is able to paint on walls, using its brush to create vivid colors and textures. tokens_predicted: 20 stop_type: word
The assertions fail if a setting differs, a response is empty, or no tokens were generated. Floating-point values such as 0.3499999940395355 represent the requested 0.35 within the comparison tolerance. A stop_type of eos means the model ended the response, word means a stop string was reached, and limit means the token budget ended generation. Different temperatures can still produce the same wording.
Related: How to call the llama.cpp chat completions API