How to debug Logstash pipelines

Debugging a Logstash pipeline shows the event exactly as it leaves the filter stage, which helps locate missing fields, failed conditionals, parsing tags, and unexpected ECS field names before the event reaches downstream outputs such as Elasticsearch.

The temporary stdout output prints each event to the standard output stream of the Logstash process. The rubydebug codec is the default stdout codec, and setting metadata to true exposes hidden [@metadata] fields that can control conditionals, index names, document IDs, or output routing without appearing in normal downstream events.

Packaged Linux installs usually keep settings under /etc/logstash and run the service through systemd. Use the active path.config from /etc/logstash/pipelines.yml instead of guessing a directory, validate as the logstash service account because allow_superuser defaults to false, and remove the temporary output after the needed sample is captured because full events can expose sensitive data and grow service logs quickly.

Steps to debug Logstash pipelines:

Open a terminal on the Logstash host.

Check the active packaged pipeline manifest.

$ sudo cat /etc/logstash/pipelines.yml
# This file is where you define your pipelines. You can define multiple.
# For more information on multiple pipelines, see the documentation:
#   https://www.elastic.co/guide/en/logstash/current/multiple-pipelines.html

- pipeline.id: main
  path.config: "/etc/logstash/conf.d/*.conf"

Add the temporary debug output to the path.config directory for the pipeline being investigated. A host with multiple pipeline IDs may use a different directory for the affected pipeline.
Related: How to configure Logstash pipelines

Add a temporary stdout debug output in /etc/logstash/conf.d/90-debug.conf.
```
output {
  stdout {
    id => "stdout_pipeline_debug"
    codec => rubydebug {
      metadata => true
    }
  }
}
```
The explicit output id makes the temporary plugin easier to identify in pipeline metrics, while metadata exposes hidden [@metadata] fields when they influence conditionals or routing.
Related: How to configure a Logstash stdout output

Every event that reaches this output is duplicated into the service console stream, so sensitive fields and high-volume traffic can spill into captured logs very quickly.
Test the packaged configuration as the logstash service account.
```
$ sudo -u logstash /usr/share/logstash/bin/logstash --path.settings /etc/logstash --path.data /tmp/logstash-configtest --config.test_and_exit
Using bundled JDK: /usr/share/logstash/jdk
Configuration OK
[2026-06-18T20:40:55,757][INFO ][logstash.runner          ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash
```
The temporary --path.data directory must be writable by the logstash user and keeps the validation away from the service data directory under /var/lib/logstash. Logstash defaults allow_superuser to false, so package-based validation should run as the service account unless that setting was intentionally changed.
Related: How to test a Logstash pipeline configuration
Restart the Logstash service to load the temporary debug output.
```
$ sudo systemctl restart logstash.service
```
A restart gives a deterministic reload point during troubleshooting. When config.reload.automatic is enabled, later pipeline-file edits can be picked up without another full service restart.

Related: How to manage the Logstash service with systemctl in Linux
Trigger one representative event through the target pipeline.

Use an input sample that exercises the suspected filter, conditional, or output branch so the debug dump shows the exact field path that is failing.

Read the emitted event dump from the service stdout stream.

$ sudo journalctl -u logstash.service --since "5 minutes ago" --no-pager --output=cat
{
     "@metadata" => {
  "debug_route" => "pipeline-debug"
},
       "client" => {
      "ip" => "10.20.30.40"
},
         "http" => {
  "request" => {"method" => "GET"},
 "response" => {"status_code" => "200"}
},
"pipeline_debug" => "enabled"
}

On packaged installs started by systemd, the stdout output is usually captured by the journal. The internal application log remains under /var/log/logstash for startup, plugin, and pipeline compiler messages.

Compare the printed event with the expected post-filter result.

Look for missing fields, unexpected tags such as _grokparsefailure, wrong data types, conditionally-added fields that never appear, or ECS-compatible field names that no longer match older filter logic.

Review recent service logs when the service restarts cleanly but the debug event never appears.

$ sudo journalctl -u logstash.service --since "5 minutes ago" --no-pager
Jun 18 20:40:55 logstash-01 logstash[22164]: [2026-06-18T20:40:55,756][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
Jun 18 20:40:55 logstash-01 logstash[22164]: [2026-06-18T20:40:55,757][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
Jun 18 20:40:55 logstash-01 logstash[22164]: [2026-06-18T20:40:55,759][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}

Confirm that the expected pipeline ID started. A missing pipeline start, repeated compile error, or different pipeline ID points to startup or routing trouble rather than to event-shape debugging.

Remove the temporary debug output file after the needed event sample is captured.
```
$ sudo rm --force /etc/logstash/conf.d/90-debug.conf
```
Leaving the file in place keeps duplicating events to the journal and can increase disk usage or expose data long after the investigation is finished.
Restart the Logstash service to stop emitting the debug dump.
```
$ sudo systemctl restart logstash.service
```
Related: How to manage the Logstash service with systemctl in Linux
Confirm the service returned to a running state after cleanup.
```
$ systemctl is-active logstash.service
active
```
If ingestion is still wrong after cleanup, use pipeline metrics to confirm the target pipeline is receiving and sending events.
Related: How to check Logstash pipeline metrics
Related: How to manage the Logstash service with systemctl in Linux