A local llama.cpp server often starts as a terminal process, but a host that should keep answering API clients after logout or reboot needs service management. systemd gives llama-server a named unit, boot enablement, restart policy, journal logs, and a standard service state for operators to inspect.

Without a model path, llama-server starts in router mode. A fresh service account has no cached models, so a healthy HTTP listener and an empty model list are valid; model loading is a separate task before clients can request inference.

A Linux system service runs independently of a login session and needs an installed binary and libraries that its dedicated account can access. The listener stays on loopback; remote access requires authentication, network restrictions, and encrypted transport before exposure.

This creates a system-wide service and a dedicated account, so the setup needs sudo privileges. The prerequisites are curl and a system-wide llama-server installation, such as the matching official Ubuntu release unpacked under /opt/llama.cpp with its companion libraries and a /usr/local/bin/llama-server symlink. A binary inside a private home directory may be inaccessible to the service account.

For an existing llama-server.service, a backup of its unit file and current settings is needed before replacement. The recovery path is to restore that file, reload the manager, and restart the service. For a new unit that must be withdrawn, sudo systemctl disable –now llama-server.service stops it and removes boot enablement.

Steps to run llama.cpp server as a systemd service:

  1. Find the absolute path to llama-server.
    $ command -v llama-server
    /usr/local/bin/llama-server

    The executable path belongs in ExecStart=. A different installation location requires its own absolute path there.

  2. Create a dedicated llama service account.
    $ sudo useradd --system --home-dir /srv/llama --shell /usr/sbin/nologin --user-group llama

    The account has no interactive login shell. User=llama and Group=llama keep the server process separate from root and human login accounts.

  3. Create the service working directory owned by llama.
    $ sudo install -d -o llama -g llama -m 0755 /srv/llama
  4. Write the llama-server.service system unit.
    $ sudo tee /etc/systemd/system/llama-server.service > /dev/null <<'EOF'
    [Unit]
    Description=llama.cpp server
    After=network.target
     
    [Service]
    Type=exec
    User=llama
    Group=llama
    WorkingDirectory=/srv/llama
    ExecStart=/usr/local/bin/llama-server --host 127.0.0.1 --port 8080
    Restart=on-failure
    RestartSec=5
    TimeoutStopSec=30
    SyslogIdentifier=llama-server
     
    [Install]
    WantedBy=multi-user.target
    EOF

    With Type=exec, a failure to execute the binary is reported during startup; HTTP readiness comes later. An unexpectedly failed process restarts after five seconds because of Restart=on-failure. The explicit host and port keep the router on 127.0.0.1:8080 across builds with different defaults. No model is selected in this unit.

  5. Verify that systemd accepts the unit file.
    $ sudo systemd-analyze verify /etc/systemd/system/llama-server.service

    A zero exit status indicates that the unit passed the check. Diagnostics name their source units, which can include other installed services. An error in llama-server.service needs correction before the manager reload.

  6. Reload the systemd manager configuration.
    $ sudo systemctl daemon-reload
  7. Enable llama-server.service at boot.
    $ sudo systemctl enable llama-server.service

    The target link controls boot enablement without starting the process.
    Related: Enable a systemd service

  8. Start llama-server.service.
    $ sudo systemctl start llama-server.service

    A restart briefly interrupts HTTP connections. The subsequent status and API requests check the newly started router.

  9. Restart llama-server.service through the service manager.
    $ sudo systemctl restart llama-server.service
  10. Confirm that the service is running and enabled.
    $ systemctl show llama-server.service -p ActiveState -p SubState -p UnitFileState
    ActiveState=active
    SubState=running
    UnitFileState=enabled

    If the unit fails, sudo journalctl -u llama-server.service -n 30 –no-pager shows recent service logs, including executable, permission, or listener errors.
    Related: Check systemd service status

  11. Request the router health endpoint after the restart.
    $ curl --fail --silent --show-error http://127.0.0.1:8080/health
    {"status":"ok"}

    The router can return ok with no models loaded. If the listener has not opened yet, a repeated request after startup can succeed; a persistent failure calls for the service logs.
    Related: Check llama.cpp server health

  12. List the models advertised by the restarted router.
    $ curl --fail --silent --show-error http://127.0.0.1:8080/v1/models
    {"data":[],"object":"list"}

    An empty data array is valid for a fresh account with no model cache. It confirms that the router answers model-list requests, not that a model is ready for inference. The router's model-management setup handles discovery and loading.
    Related: Load a model with the llama.cpp server router

Share this guide