How to run Filebeat setup for templates and dashboards

Running Filebeat setup loads the Elasticsearch index-management assets and Kibana saved objects that packaged Filebeat dashboards expect. It belongs before a new rollout, after rebuilding a Kibana space, or after an upgrade where the Filebeat template, data stream, or dashboards are missing.

The filebeat setup command reads the active /etc/filebeat/filebeat.yml configuration, connects to Elasticsearch through output.elasticsearch, and sends dashboard assets to the setup.kibana endpoint. It does not start the Filebeat service or prove that live log events are being harvested.

A setup credential needs permission to create index templates, ILM policy, data streams, and Kibana saved objects, and the running shipper can use a narrower publishing credential afterward when setup and ingest are separated. Filebeat still needs at least one enabled input or module before the setup command will pass config loading, even when the setup run is only loading assets.

1. Confirm the active Filebeat config has an enabled input, Elasticsearch output, and Kibana setup endpoint.

   /etc/filebeat/filebeat.yml

   filebeat.inputs:
     - type: filestream
       id: app-logs
       enabled: true
       paths:
         - /var/log/myapp/*.log
   
   output.elasticsearch:
     hosts: ["https://es.example.net:9200"]
     username: "filebeat_setup"
     password: "${ES_SETUP_PASSWORD}"
     ssl.certificate_authorities: ["/etc/filebeat/certs/http-ca.crt"]
   
   setup.kibana:
     host: "https://kibana.example.net:5601"
     username: "filebeat_setup"
     password: "${KIBANA_SETUP_PASSWORD}"
     space.id: "default"
     ssl.certificate_authorities: ["/etc/filebeat/certs/http-ca.crt"]

   An enabled module under /etc/filebeat/modules.d also satisfies the input requirement. Store setup passwords in the Filebeat keystore or inject them through the service environment instead of leaving cleartext secrets in the YAML file.
   Related: How to add a secret to a Filebeat keystore
   Related: How to enable a Filebeat module

2. Test the Filebeat configuration before loading assets.

   $ sudo filebeat test config -c /etc/filebeat/filebeat.yml
   Config OK

   If this exits with no modules or inputs enabled and configuration reloading disabled, enable one input or module before rerunning setup.
   Related: How to test a Filebeat configuration

3. Test the configured Elasticsearch output.

   $ sudo filebeat test output -c /etc/filebeat/filebeat.yml
   elasticsearch: https://es.example.net:9200...
     parse url... OK
     connection...
       parse host... OK
       dns lookup... OK
       addresses: 192.0.2.20
       dial up... OK
     TLS...
       security: server's certificate chain verification is enabled
       handshake... OK
       TLS version: TLSv1.3
       dial up... OK
     talk to server... OK
     version: 9.4.2

   talk to server… OK followed by the cluster version confirms that setup can reach Elasticsearch with the active output settings.
   Related: How to test Filebeat output connectivity

4. Run the one-time setup command.

   $ sudo filebeat setup -e -c /etc/filebeat/filebeat.yml
   Overwriting lifecycle policy is disabled. Set `setup.ilm.overwrite: true` to overwrite.
   Index setup finished.
   Loading dashboards (Kibana must be running and reachable)
   Loaded dashboards
   Loaded Ingest pipelines

   Re-running setup can overwrite matching Kibana saved objects, and enabling setup.ilm.overwrite: true can replace the installed lifecycle policy.

5. Verify the Elasticsearch index template.

   $ curl --silent \
     --user "filebeat_setup:${ES_SETUP_PASSWORD}" \
     --cacert /etc/filebeat/certs/http-ca.crt \
     "https://es.example.net:9200/_index_template/filebeat-*?filter_path=index_templates.name,index_templates.index_template.index_patterns"
   {"index_templates":[{"name":"filebeat-9.4.2","index_template":{"index_patterns":["filebeat-9.4.2"]}}]}

   The version in the template name follows the installed Filebeat package, so newer releases show a different suffix.

6. Verify the Filebeat data stream.

   $ curl --silent \
     --user "filebeat_setup:${ES_SETUP_PASSWORD}" \
     --cacert /etc/filebeat/certs/http-ca.crt \
     "https://es.example.net:9200/_data_stream/filebeat-*?filter_path=data_streams.name,data_streams.template"
   {"data_streams":[{"name":"filebeat-9.4.2","template":"filebeat-9.4.2"}]}

   A matching template and data stream confirm that the index-management portion of setup completed.

7. Confirm that Kibana contains Filebeat dashboards.

   $ curl --silent \
     --user "filebeat_setup:${KIBANA_SETUP_PASSWORD}" \
     --cacert /etc/filebeat/certs/http-ca.crt \
     -H 'kbn-xsrf: true' \
     "https://kibana.example.net:5601/api/saved_objects/_find?type=dashboard&search_fields=title&search=filebeat&per_page=1"
   {"page":1,"per_page":1,"total":76,"saved_objects":[{"type":"dashboard","attributes":{"title":"[Filebeat AWS] CloudTrail"}}]}

   Prefix the API path with /s/<space_id> when setup.kibana.space.id targets a non-default Kibana space.
   Related: How to check Kibana status

8. Confirm that the filebeat-* data view exists.

   $ curl --silent \
     --user "filebeat_setup:${KIBANA_SETUP_PASSWORD}" \
     --cacert /etc/filebeat/certs/http-ca.crt \
     -H 'kbn-xsrf: true' \
     "https://kibana.example.net:5601/api/saved_objects/_find?type=index-pattern&search_fields=title&search=filebeat&per_page=1"
   {"page":1,"per_page":1,"total":1,"saved_objects":[{"type":"index-pattern","attributes":{"title":"filebeat-*"}}]}

   The Saved Objects API still uses index-pattern for data views. In the Kibana UI, the imported dashboards should appear under Analytics → Dashboards after searching for filebeat.