How to set context size in llama.cpp

The context window in llama.cpp controls how many prompt and generated tokens can stay available to a loaded model. Setting it explicitly helps when a local server needs room for longer prompts, retrieval chunks, or conversation history without depending only on model metadata or automatic runtime fitting.

llama-server and llama-cli both expose -c and --ctx-size for the prompt context size. A value of 0 lets llama.cpp load the size from the model metadata, while a positive value asks for a specific window and increases KV cache memory use as it grows.

Choose a value that the model and hardware can hold, then verify the loaded runtime instead of trusting the startup flag alone. Some models cap the slot context at their training context, and multi-slot server runs should be checked on the actual server configuration that clients will use.

1. Confirm the installed server binary exposes the context-size option.

   $ llama-server --help
   ##### snipped #####
   -c,    --ctx-size N                     size of the prompt context (default: 0, 0 = loaded from model)
                                           (env: LLAMA_ARG_CTX_SIZE)
   ##### snipped #####

   llama-cli exposes the same -c and --ctx-size option. Use LLAMA_ARG_CTX_SIZE when a wrapper or service manager should provide the value through the environment.

2. Choose the target context size for the model.

   The tiny test model used for verification reports n_ctx_train as 128. For a production chat or RAG model, replace 128 with a model-supported value such as 4096, 8192, or a larger window that fits the host memory budget.

3. Start llama-server with the model path, alias, context size, and one server slot.

   $ llama-server -m models/stories260K.gguf \
     --alias local-context \
     --ctx-size 128 \
     --parallel 1 \
     --host 127.0.0.1 \
     --port 8080
   ##### snipped #####
   srv    load_model: initializing, n_slots = 1, n_ctx_slot = 128, kv_unified = 'false'
   srv  llama_server: model loaded
   srv  llama_server: listening on http://127.0.0.1:8080

   Replace models/stories260K.gguf and 128 with the real GGUF file and target window. Keep --parallel 1 while validating a single-request server so the reported slot context is easy to compare.

4. Check the server health endpoint from another terminal.

   $ curl http://127.0.0.1:8080/health
   {"status":"ok"}

5. Confirm the loaded server properties report the active context size.

   $ curl http://127.0.0.1:8080/props
   {
     "default_generation_settings": {
       "n_ctx": 128
     },
     "total_slots": 1,
     "model_alias": "local-context"
   }

   GET /props is read-only by default. It reports the server's active generation settings without enabling runtime property changes.

6. Confirm OpenAI-compatible clients see the same model context metadata.

   $ curl http://127.0.0.1:8080/v1/models
   {"object":"list","data":[{"id":"local-context","object":"model","owned_by":"llamacpp","meta":{"n_ctx":128,"n_ctx_train":128}}]}

   n_ctx is the loaded context window reported to clients. n_ctx_train is the training context recorded in the model metadata. If n_ctx is lower than the requested value, check the startup log for a model cap, automatic fitting, slot allocation, or memory allocation failure.