How to configure a filestream input in Filebeat

Configuring a filestream input in Filebeat keeps log collection reliable when applications write continuously and rotate files in place. A well-scoped input reduces missed lines, duplicate rereads, and noisy harvesting from paths that were never meant to be collected.

The filestream input is the current replacement for the legacy log input. It watches glob-based paths, tracks per-file state in the registry, and uses a required stable id so Filebeat can recognize the same input after restarts, rotations, and renames.

Current Filebeat releases use fingerprint-based file identity by default, so a very small new test log can wait until it grows beyond the first 1024 bytes before ingestion starts. Changing the id, duplicating it across inputs, or overlapping the same paths can trigger duplicate events, and any YAML mistake in /etc/filebeat/filebeat.yml will stop the filebeat service from starting.

1. Create a backup copy of the current Filebeat configuration.

   $ sudo cp /etc/filebeat/filebeat.yml /etc/filebeat/filebeat.yml.bak

2. Open the Filebeat configuration file for editing.

   $ sudoedit /etc/filebeat/filebeat.yml

3. Add a filestream input under filebeat.inputs with a unique id and the target log path pattern.

   filebeat.inputs:
     - type: filestream
       id: app-logs
       enabled: true
       paths:
         - /var/log/app/*.log

   If filebeat.inputs already exists, add another - type: filestream item under the existing key instead of creating a second filebeat.inputs block.

   Each filestream input must have a unique, stable id. Reusing or changing the id can make Filebeat lose state and reread files from the beginning.

4. Confirm the configured paths glob matches the intended files.

   $ sudo ls -1 /var/log/app/*.log
   /var/log/app/app.log

   If the glob matches nothing, Filebeat starts cleanly but harvests nothing until a file appears at that path.

5. Test the Filebeat configuration before restarting the service.

   $ sudo filebeat test config -c /etc/filebeat/filebeat.yml
   Config OK

   Related: How to test a Filebeat configuration

6. Restart the Filebeat service to load the new input.

   $ sudo systemctl restart filebeat

   Related: How to manage the Filebeat service with systemctl in Linux

7. Check that the Filebeat service returned to an active state.

   $ sudo systemctl status filebeat --no-pager --lines=20
   ● filebeat.service - Filebeat sends log files to Logstash or Elasticsearch.
        Loaded: loaded (/usr/lib/systemd/system/filebeat.service; enabled; preset: enabled)
        Active: active (running) since Wed 2026-04-02 11:44:34 UTC; 5s ago
        CGroup: /system.slice/filebeat.service
                └─3372 /usr/share/filebeat/bin/filebeat --environment systemd -c /etc/filebeat/filebeat.yml --path.home /usr/share/filebeat --path.config /etc/filebeat --path.data /var/lib/filebeat --path.logs /var/log/filebeat
   ##### snipped #####

   Related: How to manage the Filebeat service with systemctl in Linux

8. Review recent Filebeat logs for the filestream startup messages.

   $ sudo journalctl -u filebeat.service -n 50 --no-pager -o cat | grep -E "Starting input|Loading and starting Inputs completed|Input 'filestream' starting"
   {"log.level":"info","@timestamp":"2026-04-02T11:44:34.341Z","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.(*crawler).startInput","file.name":"beater/crawler.go","file.line":148},"message":"Starting input (ID: 15513294902723155206)","service.name":"filebeat","ecs.version":"1.6.0"}
   {"log.level":"info","@timestamp":"2026-04-02T11:44:34.341Z","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.(*crawler).Start","file.name":"beater/crawler.go","file.line":111},"message":"Loading and starting Inputs completed. Enabled inputs: 1","service.name":"filebeat","ecs.version":"1.6.0"}
   {"log.level":"info","@timestamp":"2026-04-02T11:44:34.341Z","log.logger":"input.filestream","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/input/v2/compat.(*runner).Start.func1","file.name":"compat/compat.go","file.line":141},"message":"Input 'filestream' starting","service.name":"filebeat","id":"app-logs","ecs.version":"1.6.0"}

   If Filebeat is configured to log to files instead of journald, inspect /var/log/filebeat/ and grep for the same messages there. Very small new test files can also wait until they grow beyond the default 1024-byte fingerprint length.