How to enable Logstash persistent queues
Persistent queues in Logstash put in-flight events on disk instead of keeping them only in memory, which reduces data loss during restarts and gives downstream systems time to recover from short output slowdowns or indexing outages.
When queue.type is set to persisted, Logstash stores queue pages under path.queue, which defaults to path.data/queue. On Debian and RPM package installs, path.data normally points to /var/lib/logstash, so the queue usually lives under /var/lib/logstash/queue unless that path is overridden. Queue sizing settings apply per pipeline unless a pipeline-specific override is set in /etc/logstash/pipelines.yml.
Persistent queues are disabled by default and they are not a replacement for end-to-end acknowledgement or replicated storage. Inputs that cannot acknowledge receipt to the sender, such as raw TCP or UDP, can still lose data before Logstash receives it, and persistent queue data is not replicated across machines. Elastic also does not support placing queue files on NFS, so size the queue for the total local disk available across all pipelines and use queue.drain: true only when shutdowns should wait for queued events to empty.
Steps to enable Logstash persistent queues:
- Open /etc/logstash/logstash.yml in a text editor.
$ sudoedit /etc/logstash/logstash.yml
- Enable the persisted queue and set the backlog size.
queue.type: persisted queue.max_bytes: 1gb path.queue: /var/lib/logstash/queue
On packaged installs, path.queue normally defaults to path.data/queue, which is usually /var/lib/logstash/queue. Leaving the default is fine when the standard data path already has enough local disk space.
queue.max_bytes applies per pipeline unless a specific pipeline overrides it in /etc/logstash/pipelines.yml, so total disk usage must cover the sum of all persistent queues. A smaller value is fine when it still covers the expected outage window and is not lower than queue.page_capacity.
Elastic does not support storing persistent queue data on NFS. Use local storage and leave enough free space for queue pages, checkpoints, JVM temp files, and normal host operations.
- Create the queue directory with ownership and permissions that match the logstash service account.
$ sudo install -o logstash -g logstash -m 0750 -d /var/lib/logstash/queue $ sudo ls -ld /var/lib/logstash/queue drwxr-x--- 2 logstash logstash 4096 Apr 8 00:52 /var/lib/logstash/queue
If path.queue is omitted and the packaged default /var/lib/logstash is still in use, Logstash can create /var/lib/logstash/queue automatically at startup. Creating it explicitly makes ownership and filesystem placement obvious before the restart.
- Validate the packaged configuration before restarting the service.
$ sudo -u logstash /usr/share/logstash/bin/logstash --path.settings /etc/logstash --path.data /tmp/logstash-configtest-pq --config.test_and_exit Using bundled JDK: /usr/share/logstash/jdk [2026-04-08T00:58:12,821][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"9.3.1", "jruby.version"=>"jruby 9.4.13.0 (3.1.4) 2025-06-10 9938a3461f OpenJDK 64-Bit Server VM 21.0.9+10-LTS on 21.0.9+10-LTS +indy +jit [aarch64-linux]"} Configuration OK [2026-04-08T00:58:16,815][INFO ][logstash.runner ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting LogstashThe temporary --path.data directory keeps the syntax test away from the live service data path in /var/lib/logstash.
Current Logstash releases default allow_superuser to false, so this packaged validation should run as the logstash user instead of plain root unless that setting was changed intentionally.
- Restart the Logstash service so the queue settings take effect.
$ sudo systemctl restart logstash.service
Restarting Logstash pauses every active pipeline while inputs, filters, and outputs are rebuilt. Changes in /etc/logstash/logstash.yml always require a service restart even when automatic pipeline reload is enabled.
- Confirm that the service and API endpoint came back cleanly after the restart.
$ sudo journalctl -u logstash.service --since "5 minutes ago" --no-pager --lines=20 Apr 08 01:00:41 logstash-01 systemd[1]: Started logstash.service - logstash. Apr 08 01:00:41 logstash-01 logstash[2147]: [2026-04-08T01:00:41,184][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false} Apr 08 01:00:41 logstash-01 logstash[2147]: [2026-04-08T01:00:41,413][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"}If the API line does not appear, inspect /var/log/logstash/logstash-plain.log or rerun the configuration test before retrying the restart.
- Query the pipeline stats API and confirm that the queue type is persisted.
$ curl -s http://127.0.0.1:9600/_node/stats/pipelines/main?pretty { "pipelines" : { "main" : { "queue" : { "type" : "persisted", "capacity" : { "max_queue_size_in_bytes" : 1073741824, "page_capacity_in_bytes" : 67108864, "queue_size_in_bytes" : 9767 }, "events_count" : 0, "queue_size_in_bytes" : 8225, "max_queue_size_in_bytes" : 1073741824 } } } ##### snipped ##### }Replace main with the pipeline ID from /etc/logstash/pipelines.yml when the service runs multiple pipelines.
Current Logstash settings use api.http.host for the bind address and api.http.port for the port or port range. Packaged installs usually expose the API on 127.0.0.1 and the first free port in the default 9600-9700 range.
- Confirm that the persistent queue created on-disk page and checkpoint files.
$ sudo ls -lh /var/lib/logstash/queue/main total 65544 -rw-r--r-- 1 logstash logstash 34 Apr 8 01:00 checkpoint.head -rw-r--r-- 1 logstash logstash 64M Apr 8 01:00 page.1
The main directory name matches the pipeline ID. Additional pipelines create separate queue directories under the same path.queue root.
Positive queue_persisted_growth_bytes or queue_persisted_growth_events in the same API output means the queue is growing because inputs are producing faster than outputs are draining.