How to reset the Filebeat registry

Resetting the Filebeat registry makes Filebeat forget the saved offsets it uses to resume harvesting log files. It is useful after a corrupt registry, a failed test run, or a deliberate replay, but it can resend events that Elasticsearch, Logstash, Redis, or another output has already accepted.

The packaged DEB and RPM service keeps persistent data under /var/lib/filebeat by default, and filebeat.registry.path defaults to ${path.data}/registry. The systemd unit can override path.data through BEAT_PATH_OPTS, so confirm the active data path before moving any registry directory on hosts with service drop-ins or custom packaging.

Stop filebeat.service before moving the registry so Filebeat cannot write fresh offset state during the reset. A recreated /registry/filebeat directory with meta.json and log.json confirms that Filebeat started and wrote new state; current filestream inputs using the default fingerprint identity still wait for files to exceed 1024 bytes before harvesting them.

1. Check the active Filebeat service path options.

   $ sudo systemctl show filebeat --property=Environment
   Environment=BEAT_LOG_OPTS= BEAT_CONFIG_OPTS=-c /etc/filebeat/filebeat.yml BEAT_PATH_OPTS=--path.home /usr/share/filebeat --path.config /etc/filebeat --path.data /var/lib/filebeat --path.logs /var/log/filebeat

   Use the directory after --path.data as the base path for the registry. The default package path is /var/lib/filebeat.

   Related: How to manage the Filebeat service with systemctl in Linux

2. Stop the Filebeat service.

   $ sudo systemctl stop filebeat

   Stopping Filebeat pauses event shipping until the service starts again.

   Related: How to manage the Filebeat service with systemctl in Linux

3. Move the current registry directory out of the active data path.

   $ sudo mv /var/lib/filebeat/registry /var/lib/filebeat/registry.bak

   Resetting the registry can replay log lines that were already indexed, forwarded, or written to the configured output. Keep the backup until the replayed data is reviewed.

4. Start the Filebeat service.

   $ sudo systemctl start filebeat

   Related: How to manage the Filebeat service with systemctl in Linux

5. Confirm the service returned to the running state.

   $ sudo systemctl is-active filebeat
   active

   When the unit does not return active, review recent logs with sudo journalctl -u filebeat.service --no-pager --lines=40 before changing the registry again.

   Related: How to manage the Filebeat service with systemctl in Linux

6. List the recreated registry files.

   $ sudo find /var/lib/filebeat/registry -type f
   /var/lib/filebeat/registry/filebeat/meta.json
   /var/lib/filebeat/registry/filebeat/log.json

   A fresh /registry directory confirms Filebeat wrote new state after startup. If the active path.data is not /var/lib/filebeat, run the same check under that custom data path.