How to create a Filebeat keystore

Creating a Filebeat keystore prepares a host to keep output passwords, API keys, and tokens outside /etc/filebeat/filebeat.yml. The local keystore gives Filebeat a place to read secret values during startup while the visible YAML can contain only placeholder names.

The filebeat keystore command writes the store under the active path.data directory and saves arbitrary key names such as ES_PWD. Filebeat resolves ${KEY} placeholders from the keystore before environment variables, so a key can supply output.elasticsearch.password without placing the password in the configuration file.

Run the command as the same operating-system user and against the same path.data used by the service. DEB and RPM services normally use /var/lib/filebeat through the packaged systemd unit, while archive or custom service installs can use another data path. Use --force with keystore create only when replacing the whole keystore intentionally, because it removes stored entries.

1. Check the data path used by the packaged Filebeat service.

   $ sudo systemctl cat filebeat
   # /lib/systemd/system/filebeat.service
   ##### snipped #####
   [Service]
   Environment="BEAT_CONFIG_OPTS=-c /etc/filebeat/filebeat.yml"
   Environment="BEAT_PATH_OPTS=--path.home /usr/share/filebeat --path.config /etc/filebeat --path.data /var/lib/filebeat --path.logs /var/log/filebeat"
   ExecStart=/usr/share/filebeat/bin/filebeat --environment systemd $BEAT_CONFIG_OPTS $BEAT_PATH_OPTS
   ##### snipped #####

   Use the service's --path.data value when it differs from /var/lib/filebeat. Archive installs usually store data under the extracted Filebeat directory unless the launcher passes another value.

   Related: How to manage the Filebeat service with systemctl in Linux

2. Create the Filebeat keystore under the service data path.

   $ sudo filebeat keystore create --path.data /var/lib/filebeat
   Created filebeat keystore

   Use --force only when replacing the keystore intentionally, because it deletes every stored key in the existing keystore.

3. Verify that the keystore file exists with restrictive permissions.

   $ sudo ls -l /var/lib/filebeat/filebeat.keystore
   -rw------- 1 root root 130 Jun 18 12:10 /var/lib/filebeat/filebeat.keystore

4. Add the first secret key when the keystore should be used immediately.

   $ sudo filebeat keystore add ES_PWD --path.data /var/lib/filebeat
   Enter value for ES_PWD:
   Successfully updated the keystore

   Enter the value at the prompt so the secret is not printed in the terminal. For automation, pass the value from a secret manager through --stdin instead of typing it into shell history.

5. List the stored key names to confirm the keystore is readable.

   $ sudo filebeat keystore list --path.data /var/lib/filebeat
   ES_PWD

   filebeat keystore list prints key names only. It does not print stored secret values.