How to debug Logstash pipelines

Debugging Logstash pipelines is the fastest way to prove where events are changing shape, losing fields, or failing a conditional before they reach downstream outputs such as Elasticsearch.

The stdout output plugin prints the event that leaves the pipeline, so the captured output reflects the combined result of the current input, filter, and output flow. Using the rubydebug codec with metadata enabled exposes the final field structure after parsing, renaming, enrichment, and tagging, which makes it easier to spot problems such as missing fields, unexpected ECS field names, or _grokparsefailure tags.

Commands below assume a packaged Logstash install on Linux with settings under /etc/logstash and the service managed by systemd. When pipelines are defined in /etc/logstash/pipelines.yml with custom path.config values, add the temporary debug output in the directory for the pipeline being investigated. Current releases also block superuser runs by default, and the debug dump can expose sensitive event data, so validate as the logstash user and remove the temporary output as soon as the needed event sample is captured.

Steps to debug Logstash pipelines:

Open a terminal on the Logstash host.
Add a temporary stdout debug output in /etc/logstash/conf.d/90-debug.conf.
```
output {
  stdout {
    codec => rubydebug {
      metadata => true
    }
  }
}
```
The stdout output defaults to the rubydebug codec, and enabling metadata exposes hidden fields such as [@metadata] when they influence conditionals or routing.

Every event that reaches this output is duplicated into the service console stream, so sensitive fields and high-volume traffic can spill into captured logs very quickly.
Test the packaged configuration as the logstash service account.
```
$ sudo -u logstash /usr/share/logstash/bin/logstash --path.settings /etc/logstash --path.data /tmp/logstash-configtest --config.test_and_exit
Using bundled JDK: /usr/share/logstash/jdk
Configuration OK
[2026-04-08T00:41:11,203][INFO ][logstash.runner          ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash
```
The temporary --path.data directory must be writable by the logstash user and keeps the validation away from the service data directory under /var/lib/logstash.

Current Logstash releases default allow_superuser to false, so running the same test as root fails with Logstash cannot be run as superuser unless that setting is explicitly changed.

Related: How to test a Logstash pipeline configuration
Restart the Logstash service to load the temporary debug output.
```
$ sudo systemctl restart logstash.service
```
A restart gives a deterministic reload point during troubleshooting. When config.reload.automatic is enabled, later pipeline-file edits can be picked up without another full service restart.

Related: How to manage the Logstash service with systemctl in Linux

Follow the systemd journal for the emitted event dump.

$ sudo journalctl -u logstash.service -f --no-pager
Apr 08 00:40:00 logstash-01 logstash[22164]: {
Apr 08 00:40:00 logstash-01 logstash[22164]:                "@timestamp" => 2026-04-08T00:40:00.288985210Z,
Apr 08 00:40:00 logstash-01 logstash[22164]:                  "message" => "10.20.30.40 GET /api/health 200",
Apr 08 00:40:00 logstash-01 logstash[22164]:              "client.ip" => "10.20.30.40",
Apr 08 00:40:00 logstash-01 logstash[22164]:     "http.request.method" => "GET",
Apr 08 00:40:00 logstash-01 logstash[22164]: "http.response.status_code" => 200,
Apr 08 00:40:00 logstash-01 logstash[22164]:                 "url.path" => "/api/health",
Apr 08 00:40:00 logstash-01 logstash[22164]:          "pipeline_debug" => "enabled"
Apr 08 00:40:00 logstash-01 logstash[22164]: }

On packaged installs, /var/log/logstash/logstash-plain.log still holds the internal application logs, while the temporary stdout output is usually captured by the service manager.

Trigger one representative event through the target pipeline.

Use an input sample that exercises the suspected filter, conditional, or output branch so the debug dump shows the exact field path that is failing.
Compare the printed event with the expected post-filter result.

Look for missing fields, unexpected tags such as _grokparsefailure, wrong data types, conditionally-added fields that never appear, or ECS-compatible dotted field names that no longer match older filter logic.

Review the packaged Logstash application log when the service restarts cleanly but the debug event never appears.

$ sudo tail --lines=60 /var/log/logstash/logstash-plain.log
[2026-04-08T00:39:58,213][INFO ][logstash.runner          ] Log4j configuration path used is: /etc/logstash/log4j2.properties
[2026-04-08T00:39:58,872][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2026-04-08T00:40:00,283][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
##### snipped #####

Use this file to confirm the pipeline compiled, the API endpoint started, and the expected pipeline ID actually came up. A missing pipeline start or repeated compile error points to a startup problem rather than to event-shape debugging.

Remove the temporary debug output file after the needed event sample is captured.
```
$ sudo rm --force /etc/logstash/conf.d/90-debug.conf
```
Leaving the file in place keeps duplicating events to the journal and can increase disk usage or expose data long after the investigation is finished.
Restart the Logstash service to stop emitting the debug dump.
```
$ sudo systemctl restart logstash.service
```
Related: How to manage the Logstash service with systemctl in Linux

Confirm the service returned to a normal running state after cleanup.

$ sudo systemctl status logstash.service --no-pager --lines=0
● logstash.service - logstash
     Loaded: loaded (/usr/lib/systemd/system/logstash.service; enabled; preset: enabled)
     Active: active (running) since Tue 2026-04-08 00:51:17 UTC; 6s ago