How to use an Elasticsearch API key with Filebeat output

API keys let Filebeat publish to Elasticsearch without reusing an account password in /etc/filebeat/filebeat.yml. A dedicated key per Filebeat host or environment limits blast radius, simplifies rotation, and keeps writer privileges scoped to the exact log destination.

The output.elasticsearch.api_key setting authenticates Filebeat to the Elasticsearch HTTP API with a single id:api_key value. Current Elastic documentation still expects the raw id and api_key joined by a colon, while the one-time API response also includes an encoded helper value meant for direct Authorization: ApiKey headers instead of the Filebeat setting.

A publishing key only covers normal event delivery. Current Filebeat documentation still recommends loading templates, ILM assets, and module pipelines separately with a more privileged setup credential, then running Filebeat with a lower-privilege writer key. Keep the key in the Filebeat keystore instead of cleartext YAML, add ssl.certificate_authorities or another trusted TLS setting when the cluster uses HTTPS, and include read_pipeline only when enabled modules rely on ingest pipelines.

Steps to use an Elasticsearch API key with Filebeat output:

Create an API key scoped to Filebeat publishing privileges.
```
$ curl --silent --show-error --fail \
  --user elastic:password \
  --header "Content-Type: application/json" \
  --request POST "https://node-01-secure:9200/_security/api_key?pretty" \
  --data '{
  "name": "filebeat-writer-host",
  "role_descriptors": {
    "filebeat_writer": {
      "cluster": ["monitor"],
      "index": [
        {
          "names": ["filebeat-*"],
          "privileges": ["create_doc", "auto_configure"]
        }
      ]
    }
  }
}'
{
  "id" : "Vh3kPZkB9Yj7m1Q2Lx4R",
  "name" : "filebeat-writer-host",
  "api_key" : "o0uH8Q72RiqP2a1Xv8cD7w",
  "encoded" : "Vmgza1Baa0I5WWo3bTFRMkx4NFI6bzB1SDhRNzJSaXFQMmExWHY4Y0Q3dw=="
}
```
Use the id and api_key fields as a single id:api_key value for Filebeat. The returned encoded value is for direct Authorization: ApiKey headers, not for output.elasticsearch.api_key.

Add expiration when short-lived publishing keys are preferred. If enabled modules depend on ingest pipelines, add the read_pipeline cluster privilege before switching Filebeat to the lower-privilege key.

If Filebeat setup has not already loaded templates, ILM assets, or pipelines, complete that first with a broader setup credential before using a write-only publishing key.

Related: How to run Filebeat setup for templates and dashboards
Related: How to create an Elasticsearch API key
Create the Filebeat keystore if this host does not already use one.
```
$ sudo filebeat keystore create
Created filebeat keystore
```
Skip this step when /var/lib/filebeat/filebeat.keystore or the deployment's configured path.data keystore already exists.

Related: How to create a Filebeat keystore
Store the id:api_key value in the Filebeat keystore.
```
$ printf 'Vh3kPZkB9Yj7m1Q2Lx4R:o0uH8Q72RiqP2a1Xv8cD7w' | sudo filebeat keystore add FILEBEAT_API_KEY --stdin
Successfully updated the keystore
```
Use --stdin to avoid leaving the secret in shell history, and target the same path.data used by the running Filebeat service.

Related: How to add a secret to a Filebeat keystore
Configure the Elasticsearch output to use the keystore-backed API key.
```
setup.ilm.check_exists: false

output.elasticsearch:
  hosts: ["https://node-01-secure:9200"]
  api_key: "${FILEBEAT_API_KEY}"
  ssl.certificate_authorities: ["/etc/filebeat/certs/elastic-ca.crt"]
```
Keep the placeholder quoted because the resolved secret contains a colon. Set setup.ilm.check_exists: false when templates and ILM assets are already loaded and the publishing key should stay minimal.

Add ssl.certificate_authorities only when the Elasticsearch CA is not already trusted by the host or container running Filebeat.
Test the Filebeat configuration file for syntax errors.
```
$ sudo filebeat test config -c /etc/filebeat/filebeat.yml
Config OK
```
Related: How to test a Filebeat configuration

Test the Elasticsearch output connection using the stored API key.

$ sudo filebeat test output -c /etc/filebeat/filebeat.yml
elasticsearch: https://node-01-secure:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 172.25.0.3
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 9.3.2

filebeat test output confirms endpoint reachability, TLS, and authentication. It does not publish an event.

Restart the Filebeat service to apply the updated output settings.
```
$ sudo systemctl restart filebeat
```
Related: How to manage the Filebeat service with systemctl in Linux

Verify Filebeat is writing to the expected indices or data streams.

$ curl --silent --show-error --fail \
  --user elastic:password \
  --cacert /etc/filebeat/certs/elastic-ca.crt \
  "https://node-01-secure:9200/_cat/indices/filebeat-*,.ds-filebeat-*?v&expand_wildcards=all"
health status index                                 uuid                   pri rep docs.count docs.deleted store.size pri.store.size dataset.size
green  open   .ds-filebeat-9.3.2-2026.04.02-000001  3WkHUrIgS_GZcNdCISr52A   1   0         30            0     58.3kb         58.3kb       58.3kb

Write-only API keys often cannot run verification queries, so use a separate read-capable credential for this check.

If enabled inputs are idle, wait for the next log line or generate a fresh event before querying Elasticsearch.