Structured local inference is easier to wire into an application when the model response already has the object shape a parser expects. The llama.cpp server can apply a JSON Schema to a chat request so the assistant message contains a deployment status object instead of free-form prose.
The response_format request field turns supported schema rules into generation constraints. The schema is not injected into the prompt, so the messages still need to describe what the fields mean. Only a subset of JSON Schema is supported, and unsupported features may be skipped during grammar conversion.
A local instruction-tuned GGUF model with a supported chat template, curl, and Python 3 with the jsonschema package are the starting requirements. Application-side schema validation remains necessary because valid JSON syntax alone does not establish field types, allowed values, or required properties.
Steps to constrain llama.cpp server output with JSON schema:
- Start llama-server with the local-struct model alias on loopback.
A public unauthenticated listener exposes model access to other computers. The loopback bind limits access to the local machine; it does not add authentication.
$ llama-server -m models/chat-model.gguf --alias local-struct --ctx-size 2048 --host 127.0.0.1 --port 8080
The model path identifies your existing chat-capable GGUF file. The server remains running in this terminal while requests run in another terminal.
Related: How to start the llama.cpp server - Check that the model has finished loading.
$ curl --fail --silent --show-error http://127.0.0.1:8080/health {"status":"ok"}
A loading model can return HTTP 503. The --fail option makes HTTP errors produce a nonzero curl exit status.
Related: How to check llama.cpp server health - Save the deployment status request in deployment-status-request.json.
- deployment-status-request.json
{ "model": "local-struct", "messages": [ { "role": "system", "content": "Return only a JSON object with service, status, and priority fields." }, { "role": "user", "content": "Create one fictional deployment status for an edge inference service. Use a service name of 3 to 32 characters, status ready or blocked, and an integer priority from 1 to 5." } ], "max_tokens": 128, "temperature": 0, "response_format": { "type": "json_schema", "json_schema": { "name": "deployment_status", "strict": true, "schema": { "type": "object", "properties": { "service": {"type": "string", "minLength": 3, "maxLength": 32}, "status": {"type": "string", "enum": ["ready", "blocked"]}, "priority": {"type": "integer", "minimum": 1, "maximum": 5} }, "required": ["service", "status", "priority"], "additionalProperties": false } } } }
The response_format.json_schema.schema object constrains the generated content. The service string has length bounds, status permits two values, and priority requires an integer in the inclusive range 1–5. The required list demands all three fields, while additionalProperties: false rejects extra fields. The messages describe their meanings separately.
Tool: JSON Schema Generator - Submit the request to the chat completions endpoint.
$ curl --fail --silent --show-error http://127.0.0.1:8080/v1/chat/completions \ --header 'Content-Type: application/json' \ --data-binary @deployment-status-request.json \ --output deployment-status-response.json \ --write-out 'HTTP %{http_code}\n' HTTP 200
The API envelope is saved in deployment-status-response.json. The generated JSON is a string inside choices[0].message.content. No authorization header is needed for a local server without API-key checking. Reasoning-capable models may need chat_template_kwargs with enable_thinking: false when their template supports that setting.
- Save the response validator as validate-deployment.py.
- validate-deployment.py
import json from jsonschema import validate with open("deployment-status-request.json") as request_file: request = json.load(request_file) with open("deployment-status-response.json") as response_file: response = json.load(response_file) choice = response["choices"][0] if choice["finish_reason"] != "stop": raise ValueError("Generation did not finish normally") status = json.loads(choice["message"]["content"]) validate(status, request["response_format"]["json_schema"]["schema"]) print(json.dumps(status, indent=2)) print("schema: valid")
The validator parses the assistant content and checks it against the exact schema sent to the server. A ValidationError rejects a schema violation; malformed or truncated JSON also causes a nonzero exit. A finish_reason of length means the token budget stopped generation, so a partial object must not be accepted.
- Validate the generated deployment status against the request schema.
$ python3 validate-deployment.py { "service": "3to5", "status": "ready", "priority": 1 } schema: valid
Generated names and values vary by model. The schema: valid line appears only after parsing and validation succeed. Schema compliance does not establish whether a deployment status is factually true.