How to configure Filebeat processors

Configuring Filebeat processors changes events before they leave the host, which reduces downstream noise and adds the metadata that search, dashboards, and alert rules need to stay useful. Applying field cleanup, tagging, and metadata enrichment at the agent edge also avoids repeating the same work later in Logstash or ingest pipelines.

Filebeat runs processors in order for each event. Elastic's current processor model still supports a top-level processors list that affects every event, a per-input processors list that affects only one input, and module-specific processors under the module's input section when a module needs different handling from the rest of the agent.

On package-based Linux installs, processor changes usually live in /etc/filebeat/filebeat.yml and require a successful filebeat test config before the service is restarted. The config test confirms that YAML structure and processor settings are valid, but it does not prove that a condition matches the intended events or that downstream outputs, ingest pipelines, and dashboards will interpret the changed fields as expected.

Steps to configure Filebeat processors:

  1. Create a backup of the current Filebeat configuration before editing the processor chain.
    $ sudo cp /etc/filebeat/filebeat.yml /etc/filebeat/filebeat.yml.bak

    Restoring the previous settings is faster when a later config test fails or a field change breaks downstream parsing.

  2. Open the Filebeat configuration file with elevated privileges.
    $ sudoedit /etc/filebeat/filebeat.yml
  3. Add or update the top-level processors list when the rules should affect every event that Filebeat ships.
    processors:
      - add_host_metadata:
          cache.ttl: 5m
      - drop_fields:
          fields:
            - log.offset
            - agent.ephemeral_id
          ignore_missing: true

    Keep field-removal or rename processors near the end of the list so earlier processors can still read the original event data.

    drop_fields cannot remove @timestamp or type.

  4. Move the processors list under a specific input or module input section when the change should affect only one log source.
    filebeat.inputs:
      - type: filestream
        id: app-logs
        enabled: true
        paths:
          - /var/log/app/*.log
        processors:
          - add_fields:
              target: ''
              fields:
                ingest_source: app_logs

    Elastic's current processor documentation still limits module-scoped processors to the module's input section rather than the module root.

  5. Test the Filebeat configuration before applying the updated processor chain.
    $ sudo filebeat test config -c /etc/filebeat/filebeat.yml
    Config OK
  6. Restart the Filebeat service to load the updated processor configuration.
    $ sudo systemctl restart filebeat
  7. Export the resolved configuration to confirm that the active processors block matches the intended processor order and values.
    $ sudo filebeat export config -c /etc/filebeat/filebeat.yml | sed -n '/^processors:/,/^output:/p'
    processors:
    - add_host_metadata:
        cache:
          ttl: 5m
    - drop_fields:
        fields:
        - log.offset
        - agent.ephemeral_id
        ignore_missing: true

    filebeat export config shows the fully merged runtime configuration, so it is useful for checking inheritance and normalized YAML output after edits.

  8. Verify that the Filebeat service returned to the active state after the restart.
    $ sudo systemctl is-active filebeat
    active