How to configure a filestream input in Filebeat

Configuring a filestream input in Filebeat enables reliable collection of continuously written logs, keeping ingestion steady during rotations, renames, and truncations. A correct input definition reduces missing lines and duplicate re-reads when log management tools reshuffle files.

The filestream input is the modern file harvester for Filebeat and is designed to replace the legacy log input. Each input uses a unique id and maintains per-file offsets in the registry so harvesting resumes from the correct position after restarts or rotations, while paths globs decide which files are watched.

A stable id is critical because changing it (or overlapping the same paths across multiple inputs) can make existing files appear new and trigger re-ingestion. YAML indentation errors in /etc/filebeat/filebeat.yml prevent the filebeat service from starting, so configuration validation should happen before any restart. Ensure the service account running Filebeat can read the target log files and traverse their parent directories.

1. Create a backup copy of the current Filebeat configuration.

   $ sudo cp /etc/filebeat/filebeat.yml /etc/filebeat/filebeat.yml.bak

2. Open the Filebeat configuration file for editing.

   $ sudo nano /etc/filebeat/filebeat.yml

3. Add a filestream input under filebeat.inputs with a unique id and matching paths patterns.

   filebeat.inputs:
     - type: filestream
       id: system-logs
       enabled: true
       paths:
         - /var/log/app.log

   If filebeat.inputs already exists, add another list item (- type: filestream) under the existing key instead of creating a second filebeat.inputs block.

4. Confirm the configured paths pattern matches the intended files.

   $ ls -1 /var/log/app/*.log
   /var/log/app.log

   No matches means nothing is harvested until a file appears, so adjust the glob or log location if needed.

5. Test the Filebeat configuration for syntax errors.

   $ sudo filebeat test config
   Config OK

   Related: How to test a Filebeat configuration

6. Restart the Filebeat service to apply the updated input.

   $ sudo systemctl restart filebeat

   Related: How to manage the Filebeat service with systemctl in Linux

7. Check that the service is running after the restart.

   $ sudo systemctl status filebeat --no-pager
   ● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
        Loaded: loaded (/usr/lib/systemd/system/filebeat.service; enabled; preset: enabled)
        Active: active (running) since Tue 2026-01-06 22:14:06 UTC; 5s ago
   ##### snipped #####

   Related: How to manage the Filebeat service with systemctl in Linux

8. Confirm the filestream input starts.

   $ sudo journalctl -u filebeat -n 50 --no-pager
   Jan 06 22:14:06 host filebeat[7639]: {"log.level":"info","@timestamp":"2026-01-06T22:14:06.174Z","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.(*crawler).startInput","file.name":"beater/crawler.go","file.line":148},"message":"Starting input (ID: 15094845207509643253)","service.name":"filebeat","ecs.version":"1.6.0"}
   Jan 06 22:14:06 host filebeat[7639]: {"log.level":"info","@timestamp":"2026-01-06T22:14:06.175Z","log.logger":"input.filestream","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/input/v2/compat.(*runner).Start.func1","file.name":"compat/compat.go","file.line":141},"message":"Input 'filestream' starting","service.name":"filebeat","id":"system-logs","ecs.version":"1.6.0"}
   ##### snipped #####

   If the service is configured to log to files, check /var/log/filebeat instead of journalctl.