Configuring a Logstash pipeline defines how events enter, change, and leave the instance, which directly affects whether logs arrive with the right fields, reach the right destination, and survive routine restarts or reloads.

On current package-based installs, Logstash reads pipeline definitions from /etc/logstash/pipelines.yml and then loads each pipeline's path.config location. The default packaged layout usually keeps the main pipeline pointed at /etc/logstash/conf.d/*.conf, where files are concatenated in lexical order into one effective pipeline, so numeric prefixes such as 10-...conf, 40-...conf, and 90-...conf keep the processing order predictable.

Current local-file workflows assume centralized pipeline management is not enabled, because that feature disables local pipelines.yml, path.config, and config.string settings. Current Logstash releases also default allow_superuser to false, so syntax checks should run as the logstash service account with a temporary --path.data directory. The monitoring API is enabled by default and usually binds to loopback on a port in the 9600-9700 range, which makes it the quickest way to confirm that the expected pipeline ID actually loaded.

Related: How to test a Logstash pipeline configuration
Related: How to enable Logstash pipeline reloads
Related: How to check Logstash pipeline metrics

Steps to configure Logstash pipelines:

  1. Confirm which pipeline ID and config path the packaged service is using. 
    $ sudo sed -n '1,20p' /etc/logstash/pipelines.yml
- pipeline.id: main
  path.config: "/etc/logstash/conf.d/*.conf"

    If the file lists a different pipeline.id or path.config, write the new configuration under that location instead of assuming /etc/logstash/conf.d.

    If xpack.management.enabled is set to true in /etc/logstash/logstash.yml, local pipeline files are inactive and changes must be made through centralized pipeline management instead.

  2. Create or update a pipeline file under the active path.config location. 
    input {
  beats {
    id => "beats_5044"
    port => 5044
  }
}
 
filter {
  mutate {
    add_tag => ["configured_pipeline"]
  }
}
 
output {
  stdout {
    id => "stdout_pipeline_check"
    codec => rubydebug
  }
}

    The example keeps one complete pipeline in a single file for clarity. When the same pipeline is split across multiple files, Logstash still merges them into one effective configuration in lexical order, so the filename prefix decides whether an input, filter, or output block appears earlier or later in the compiled pipeline.

    The temporary stdout output keeps the first validation focused on whether the pipeline compiles and starts. Replace it with the intended destination output after the pipeline structure is confirmed.

    If the real output connects to Elasticsearch or another protected destination, use TLS plus authentication and keep secrets in the Logstash keystore instead of writing literal credentials into the pipeline file.

    Related: How to configure a Logstash stdout output
    Related: How to configure Logstash output to Elasticsearch
    Related: How to add a secret to a Logstash keystore

  3. Test the assembled pipeline configuration with the packaged settings directory and a temporary data path. 
    $ sudo -u logstash /usr/share/logstash/bin/logstash --path.settings /etc/logstash --path.data /tmp/logstash-configtest --config.test_and_exit
Using bundled JDK: /usr/share/logstash/jdk
[2026-04-07T08:51:30,826][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"9.3.2", "jruby.version"=>"jruby 9.4.13.0 (3.1.4) 2025-06-10 9938a3461f OpenJDK 64-Bit Server VM 21.0.10+7-LTS on 21.0.10+7-LTS +indy +jit [aarch64-linux]"}
Configuration OK
[2026-04-07T08:51:32,042][INFO ][logstash.runner          ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash

    The temporary --path.data directory must be writable by the logstash user and keeps the validation away from the live service data directory in /var/lib/logstash.

    --config.test_and_exit validates syntax and plugin settings assembly only. It does not prove that remote outputs, credentials, TLS trust, or sender traffic will work at runtime.

    Current Logstash releases reject superuser runs by default. Running the same validation as root fails unless allow_superuser was explicitly changed in /etc/logstash/logstash.yml.

    Related: How to test a Logstash pipeline configuration

  4. Restart the Logstash service when automatic config reload is not already enabled. 
    $ sudo systemctl restart logstash.service

    Restarting the service briefly pauses every active pipeline while inputs reopen, filters recompile, and outputs reconnect.

    If /etc/logstash/logstash.yml already sets config.reload.automatic: true, a validated pipeline file change can be picked up without a full service restart. Changes to logstash.yml, plugin installs, JVM options, or service unit overrides still require one.

    Related: How to manage the Logstash service with systemctl in Linux
    Related: How to enable Logstash pipeline reloads

  5. Confirm that the service returned to the running state. 
    $ sudo systemctl status logstash.service --no-pager --lines=20
● logstash.service - logstash
     Loaded: loaded (/usr/lib/systemd/system/logstash.service; enabled; preset: enabled)
     Active: active (running) since Tue 2026-04-07 08:53:45 UTC; 11s ago
   Main PID: 22164 (java)
      Tasks: 95 (limit: 28486)
     Memory: 962.4M
        CPU: 15.104s
##### snipped #####
Apr 07 08:53:45 logstash-01 systemd[1]: Started logstash.service - logstash.

    If the unit does not stay active (running), read the first failure in /var/log/logstash/logstash-plain.log or sudo journalctl --unit=logstash.service --no-pager --lines=80 before retrying the restart.

    Related: How to manage the Logstash service with systemctl in Linux

  6. Query the monitoring API and confirm that the expected pipeline ID is loaded. 
    $ curl -s 'http://localhost:9600/_node/pipelines?pretty'
{
  "pipelines" : {
    "main" : {
      "workers" : 8,
      "batch_size" : 125,
      "batch_delay" : 50,
      "config_reload_automatic" : false,
      "config_reload_interval" : 3000000000,
      "dead_letter_queue_enabled" : false
    }
  }
}

    If the configured pipeline ID is not main, replace it in later stats or debug requests with the ID returned here.

    If this request fails on localhost:9600, check api.enabled, api.http.host, and api.http.port in /etc/logstash/logstash.yml. Current releases use the api.http.* setting names and default to the 9600-9700 port range.

    Related: How to check Logstash pipeline metrics

  7. Review recent startup lines and confirm that the pipeline actually entered the running state. 
    $ sudo journalctl --unit=logstash.service --since "5 minutes ago" --no-pager --lines=20
Apr 07 08:53:45 logstash-01 logstash[22164]: [2026-04-07T08:53:45,171][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
Apr 07 08:53:45 logstash-01 logstash[22164]: [2026-04-07T08:53:45,191][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}

    The journal confirms that Logstash compiled the pipeline and kept it running after the service action. With the temporary stdout output shown earlier, decoded events usually appear in the same journal stream on package-managed hosts.

    After the pipeline structure is confirmed, replace the temporary stdout output with the real destination output and repeat the same config test before the next restart or reload.

    Related: How to debug Logstash pipelines
    Related: How to configure a Logstash stdout output