How to reset the Filebeat registry

Resetting the Filebeat registry makes tracked log files look unseen again, which is useful after corrupted state, incorrect offsets, or a deliberate replay of previously harvested data. The same action can resend events that downstream outputs have already accepted, so it belongs in a controlled maintenance window.

Filebeat stores persistent state under path.data, including the registry that records file identity and the last acknowledged offset for each harvested file. Current DEB and RPM packages place that data under /var/lib/filebeat, keep the main configuration in /etc/filebeat, and start the service with a systemd unit that passes those path options to /usr/share/filebeat/bin/filebeat.

The reset must happen while filebeat.service is stopped so the registry is not rewritten during the move. Current filestream inputs also tie state to the input ID and file identity method; in 9.x, the default fingerprint identity hashes the first 1024 bytes of a file, so very small files might not be ingested again until they grow past that threshold even after the registry is reset.

1. Stop the Filebeat service.

   $ sudo systemctl stop filebeat

   Related: How to manage the Filebeat service with systemctl in Linux

2. Move the current registry directory out of the way instead of deleting it permanently.

   $ sudo mv /var/lib/filebeat/registry /var/lib/filebeat/registry.bak

   On current DEB and RPM installs the registry lives under /var/lib/filebeat; replace that path with the active path.data directory when Filebeat was started with a custom data path.

   Moving the whole /registry directory preserves the current log.json and meta.json state files for rollback or comparison.

   Resetting the registry can resend events that have already been indexed or forwarded.

3. Start the Filebeat service again so it creates a new registry.

   $ sudo systemctl start filebeat

   Related: How to manage the Filebeat service with systemctl in Linux

4. Confirm the Filebeat service returned to the running state.

   $ sudo systemctl is-active filebeat
   active

   When the unit does not return active, check sudo journalctl -u filebeat.service --no-pager --lines=30 because current systemd package installs store service logs in the journal by default.

   Related: How to manage the Filebeat service with systemctl in Linux

5. List the recreated registry files.

   $ sudo find /var/lib/filebeat/registry -maxdepth 2 -type f | sort
   /var/lib/filebeat/registry/filebeat/log.json
   /var/lib/filebeat/registry/filebeat/meta.json

   A new /registry directory confirms Filebeat wrote fresh state files after startup. On current filestream inputs, the first eligible scan repopulates those files, but very small files can still wait until they reach the default 1024-byte fingerprint threshold.