How to configure a Logstash file input

Shipping local application logs into a central pipeline starts with reliably tailing files on disk. The Logstash file input converts each new log line into an event so downstream outputs (such as Elasticsearch) can index, search, and alert on it.

The file input watches one or more file paths (including glob patterns) and emits events as files grow. To handle restarts and log rotation without rereading from the start, Logstash persists per-file offsets in a sincedb file keyed to the file identifier.

Reading from the beginning is controlled by start_position, but only on first sight of a file without a recorded sincedb offset. A unique sincedb_path per input avoids offset collisions, and correct file permissions are required because the Logstash service typically runs as the logstash user.

1. Create a pipeline configuration file at /etc/logstash/conf.d/40-file.conf.

   input {
     file {
       path => "/var/log/app/*.log"
       exclude => ["*.gz"]
       start_position => "beginning"
       sincedb_path => "/var/lib/logstash/sincedb-app"
     }
   }
   
   output {
     elasticsearch {
       hosts => ["http://elasticsearch.example.net:9200"]
       index => "app-logs-%{+YYYY.MM.dd}"
     }
   }

   Use a unique sincedb_path per file input to prevent missed lines or duplicate ingestion caused by shared offsets.

   start_position affects only files without an existing sincedb entry; subsequent restarts resume from the stored offset.

   start_position set to beginning can ingest large historical logs and rapidly increase downstream storage and indexing load.

2. List the log files matched by the path glob pattern.

   $ sudo ls -1 /var/log/app/*.log
   /var/log/app/app-error.log
   /var/log/app/app.log

3. Read a line from a matched log file as the logstash user.

   $ sudo -u logstash tail -n 1 /var/log/app/app.log
   2026-01-07T04:52:12Z INFO request_id=9b2a5f2c status=200 path=/health

   Permission denied errors indicate directory execute permission or file read permission problems for the logstash user.

4. Test the pipeline configuration for syntax errors.

   $ sudo /usr/share/logstash/bin/logstash --path.settings /etc/logstash --config.test_and_exit
   ##### snipped #####
   Configuration OK

   Related: How to test a Logstash pipeline configuration

5. Restart the Logstash service to load the updated pipeline.

   $ sudo systemctl restart logstash

   Related: How to manage the Logstash service with systemctl in Linux

6. Check the Logstash service status for an active (running) state.

   $ sudo systemctl status logstash
   ● logstash.service - logstash
        Loaded: loaded (/usr/lib/systemd/system/logstash.service; enabled; preset: enabled)
        Active: active (running) since Wed 2026-01-07 04:52:28 UTC; 2s ago
   ##### snipped #####

   Related: How to manage the Logstash service with systemctl in Linux

7. Verify the sincedb file exists at the configured sincedb_path.

   $ sudo ls -l /var/lib/logstash/sincedb-app
   -rw-r--r-- 1 logstash logstash 113 Jan  7 04:52 /var/lib/logstash/sincedb-app

   Deleting or reusing a sincedb file can cause rereads from earlier offsets, leading to duplicated events.