Reranking in llama.cpp scores candidate passages against a search query after an initial retrieval step. The rerank API returns their relevance order so a local search or RAG pipeline can put the closest matches into a model's context.
The llama-server rerank endpoint is separate from chat completions and embeddings. A reranker evaluates each query–passage pair rather than generating an answer or returning a reusable document vector.
Jina Reranker v1 turbo English is suitable for English query–passage comparisons. It needs its own reranker GGUF file; a chat or embedding model cannot substitute for it.
Steps to call the llama.cpp rerank API:
The starting requirements are llama-server on PATH, curl with --fail-with-body support, and a local copy of ggml-org's Jina reranker GGUF at models/reranker-model.gguf. The upstream filename is Jina-Bert-Implementation-38M-F16.gguf, converted from Jina AI's reranker.
The server has no API-key checking enabled. A 127.0.0.1 listener limits network access to this machine, but other local processes can still submit queries and documents. Remote access needs an appropriate access policy before the listener is exposed.
- Start llama-server with the Jina reranker on 127.0.0.1:8080.
$ llama-server -m models/reranker-model.gguf --alias local-reranker --reranking --host 127.0.0.1 --port 8080
--reranking enables the endpoint and selects rank pooling. The --embedding and --pooling rank switches are unnecessary with this option. The server stays in the foreground; the HTTP requests run in another terminal.
Related: How to start the llama.cpp server - Check that the reranker has finished loading.
$ curl --fail-with-body http://127.0.0.1:8080/health {"status":"ok"}
A loading model can return HTTP 503. The --fail-with-body option retains the error body and makes HTTP errors produce a nonzero curl exit status.
Related: How to check llama.cpp server health - Save the query and three candidate passages in rerank-request.json.
$ cat > rerank-request.json <<'JSON' { "model": "local-reranker", "query": "What is the capital of France?", "top_n": 2, "documents": [ "Paris is the capital of France.", "Bananas grow in warm climates.", "Berlin is the capital of Germany." ] } JSON
documents preserves the original candidate order. Its zero-based positions are 0 for Paris, 1 for bananas, and 2 for Berlin. The top_n value requests only the two highest-ranked results.
- Submit the passages to /v1/rerank for their relevance order.
$ curl --fail-with-body http://127.0.0.1:8080/v1/rerank \ --header "Content-Type: application/json" \ --data-binary @rerank-request.json {"model":"local-reranker","object":"list","usage":{"prompt_tokens":56,"total_tokens":56},"results":[{"index":0,"relevance_score":-0.03818938508629799},{"index":2,"relevance_score":-0.07548195123672485}]}
The two results entries point to original indexes 0 and 2, placing Paris before Berlin and excluding the banana passage. They arrive in descending relevance_score order, which can differ from the input order. Higher scores mean a closer match within the same request; scores can be negative and have no universal relevance cutoff across models.