GGUF files bundle a local language model's weights with the metadata needed to load them. llama.cpp serves those files through llama-server, allowing Codex to send prompts to the loaded model over an OpenAI-compatible Responses connection.

llama-server publishes its loaded model through a model-list endpoint. A short alias keeps the identifier stable even when the GGUF file has a long name, and the model's chat template supplies its expected prompt format. A text reply confirms the connection; repository work also requires compatible tool support in both the server and model.

The environment needs Codex CLI 0.134.0 or later, a llama-server build with Responses support, curl, Bash or zsh, and a downloaded GGUF model. The computer needs enough memory for the model and its context, and CPU prompt processing can take minutes. An unauthenticated local provider does not require an OpenAI account or API key.

Related: How to use local models with Codex
Related: How to troubleshoot local models in Codex

Steps to use llama.cpp models with Codex:

  1. Start llama-server with the GGUF model on the loopback interface.

    An unauthenticated API accepts work from any client that can reach it, including other programs on the same computer. The loopback listener keeps it off the network; remote access requires separate authentication and transport security. 

    $ llama-server -m ~/Models/Qwen3-0.6B.gguf \
  --alias qwen3-0.6b \
  --host 127.0.0.1 --port 8080 --ctx-size 32768 --parallel 1 --jinja

    The model path must point to the downloaded file. The alias becomes its API identifier, and Jinja enables the model's chat template for tool calls. The 32768-token context leaves room for Codex's instructions and a short conversation; larger tasks need more context and memory. A single server slot reserves the context for one request at a time. The server stays running in this terminal while Codex runs in another.
    Related: How to start the llama.cpp server
    Related: How to set llama.cpp server host and port

  2. List the model identifiers exposed by llama-server. 
    $ curl --fail --silent --show-error http://127.0.0.1:8080/v1/models

    The data array should contain an id matching the alias set at server startup. A connection error means the server is unavailable; a different identifier means the profile's model value must match that identifier.

  3. Open ~/.codex/llamacpp.config.toml in your text editor.

    If the file does not exist, create it at that path. Codex normally keeps its base settings in ~/.codex/config.toml and layers this separate profile over them only with --profile llamacpp. If CODEX_HOME is set, both files belong in that directory instead, with the profile at the following path. 

    $CODEX_HOME/llamacpp.config.toml
  4. Set the top-level routing keys before any table heading in the profile file. 
    model_provider = "llamacpp"
model = "qwen3-0.6b"

    If either top-level key already exists, replace its value rather than adding another copy. Other settings remain in place. The model value must match the server's alias.

  5. Add the llama.cpp provider table on a new line at the end of the profile file. 
    [model_providers.llamacpp]
name = "llama.cpp"
base_url = "http://127.0.0.1:8080/v1"
wire_api = "responses"
requires_openai_auth = false

    If [model_providers.llamacpp] already exists, update these keys within that table instead of adding a duplicate table or duplicate keys. Other keys and tables remain in place. The URL ends at /v1 because Codex adds the Responses endpoint. No bearer token is needed for this loopback server.
    Related: How to override Codex configuration for a single run
    Related: How to set the default model in Codex

  6. Save the llama.cpp profile file.

    A new Codex session loads the saved profile. The base config.toml keeps its existing provider, and sessions started without --profile llamacpp continue to use their usual settings.

  7. Ask Codex a short question through the llama.cpp profile. 
    $ codex --profile llamacpp \
  "Suggest a Python function name for adding two numbers."

    Codex should show the server's alias as its model and return a function name that describes adding two numbers. A connection or context error, or a model name without an answer, means inference has not completed. The wording can vary, and this question needs no repository or model-generated shell command.

Share this guide