How to filter and drop events in Filebeat

Filtering and dropping events in Filebeat trims noisy data at the source, keeping indexes lean and alerting focused on signals instead of chatter.

Filebeat turns each harvested log line into an event, enriches it with metadata, runs processors in order, and only then sends the result to the configured output. The drop_event processor discards matching events entirely, while drop_fields removes selected fields to reduce payload size without discarding the event.

On Linux, processor rules are typically stored in /etc/filebeat/filebeat.yml and applied by the filebeat service on startup. A top-level processors: block affects every input, while per-input or per-module processors affect only that source. Dropped events cannot be recovered after leaving the agent, so conditions should be specific, reviewed, and validated before restarting the service.

1. Open the Filebeat configuration file for editing.

   $ sudo nano /etc/filebeat/filebeat.yml

2. Create a backup copy of the current configuration file.

   $ sudo cp /etc/filebeat/filebeat.yml /etc/filebeat/filebeat.yml.bak

   Restore quickly by copying filebeat.yml.bak back into place.

3. Add processors rules under the top-level processors: key.

   processors:
     - drop_event:
         when:
           or:
             - equals:
                 log.level: "debug"
             - regexp:
                 message: '^GET /healthz\b'
     - drop_fields:
         fields:
           - "agent.ephemeral_id"
           - "log.offset"
         ignore_missing: true

   Keep a single top-level processors: block and merge processor items under it; place the same list under an input or module stanza to scope filtering to that source.

   drop_event removes matching events permanently, and aggressive drop_fields rules can break downstream parsing; avoid dropping core fields such as @timestamp and message.

4. Test the configuration for syntax errors before applying changes.

   $ sudo filebeat test config -c /etc/filebeat/filebeat.yml
   Config OK

   Related: How to test a Filebeat configuration

5. Restart the Filebeat service to apply the updated processors.

   $ sudo systemctl restart filebeat

   Related: How to manage the Filebeat service with systemctl in Linux

6. Confirm the Filebeat service is active after the restart.

   $ sudo systemctl status filebeat --no-pager --lines=25
   ● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
        Loaded: loaded (/usr/lib/systemd/system/filebeat.service; enabled; preset: enabled)
       Drop-In: /etc/systemd/system/filebeat.service.d
                └─env.conf
        Active: active (running) since Tue 2026-01-06 22:29:49 UTC; 5s ago
   ##### snipped #####

   Related: How to manage the Filebeat service with systemctl in Linux

7. Review recent logs for startup errors and processor troubleshooting clues.

   $ sudo journalctl --unit=filebeat --no-pager --lines=80
   Jan 06 22:29:49 host filebeat[9158]: {"log.level":"info","@timestamp":"2026-01-06T22:29:49.905Z","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.(*crawler).Start","file.name":"beater/crawler.go","file.line":76},"message":"Loading Inputs: 1","service.name":"filebeat","ecs.version":"1.6.0"}
   Jan 06 22:29:49 host filebeat[9158]: {"log.level":"info","@timestamp":"2026-01-06T22:29:49.906Z","log.logger":"input.filestream","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/input/v2/compat.(*runner).Start.func1","file.name":"compat/compat.go","file.line":141},"message":"Input 'filestream' starting","service.name":"filebeat","id":"app-logs","ecs.version":"1.6.0"}
   ##### snipped #####

   Set logging.level: debug and logging.selectors: ["processors"] temporarily to log drop decisions.