How to enable Filebeat load balancing

High-volume log shipping can overwhelm a single Elasticsearch endpoint, turning one slow or failed node into a backlog across every connected system. Enabling load balancing in Filebeat distributes publish traffic across multiple nodes for better throughput and improved resilience during maintenance or outages.

Filebeat supports multiple output hosts in the output.elasticsearch.hosts list and publishes events in bulk requests. With load balancing disabled, Filebeat typically sticks to one host and only fails over when the selected host becomes unreachable; enabling loadbalance: true spreads batches across reachable hosts to reduce hotspots.

Every configured host must be interchangeable from Filebeat’s perspective—same cluster, same authentication, and compatible TLS configuration—or publish failures can become intermittent and difficult to diagnose. The workflow assumes a Linux package installation using /etc/filebeat/filebeat.yml and a systemd unit named filebeat.

1. Open the Filebeat configuration file.

   $ sudo nano /etc/filebeat/filebeat.yml

2. Configure multiple Elasticsearch hosts with loadbalance: true enabled.

   output.elasticsearch:
     hosts:
       - "http://node-01:9200"
       - "http://node-02:9200"
     loadbalance: true
     worker: 2

   Keep existing username, password, and ssl.* settings in the same output.elasticsearch block so every host uses identical authentication and TLS.

   Invalid YAML (indentation or quoting) can prevent the filebeat service from starting.

3. Test the configuration for syntax errors.

   $ sudo filebeat test config -c /etc/filebeat/filebeat.yml
   Config OK

   Related: How to test a Filebeat configuration

4. Test the configured Elasticsearch outputs for connectivity.

   $ sudo filebeat test output -c /etc/filebeat/filebeat.yml
   elasticsearch: http://node-01:9200...
     parse url... OK
     connection...
       parse host... OK
       dns lookup... OK
       addresses: 172.19.0.2
       dial up... OK
     TLS... WARN secure connection disabled
     talk to server... OK
     version: 8.12.2
   elasticsearch: http://node-02:9200...
   ##### snipped #####

   This check validates reachability and authentication using the saved configuration.

5. Restart the filebeat service to apply the load balancing settings.

   $ sudo systemctl restart filebeat

   Related: How to manage the Filebeat service with systemctl in Linux

6. Confirm the filebeat service is running.

   $ sudo systemctl status filebeat --no-pager --lines=12
   ● filebeat.service - Filebeat sends log files to Logstash or Elasticsearch.
        Loaded: loaded (/usr/lib/systemd/system/filebeat.service; enabled; preset: enabled)
       Drop-In: /etc/systemd/system/filebeat.service.d
                └─env.conf
        Active: active (running) since Wed 2026-01-07 04:21:26 UTC; 4s ago
   ##### snipped #####

7. Review recent logs for connections to multiple output hosts.

   $ sudo journalctl --unit=filebeat --no-pager --lines=30
   Jan 07 04:21:42 host filebeat[12563]: {"log.level":"info","@timestamp":"2026-01-07T04:21:42.493Z","log.logger":"elasticsearch.esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.NewConnection","file.name":"eslegclient/connection.go","file.line":132},"message":"elasticsearch url: http://node-01:9200","service.name":"filebeat","ecs.version":"1.6.0"}
   Jan 07 04:21:42 host filebeat[12563]: {"log.level":"info","@timestamp":"2026-01-07T04:21:42.493Z","log.logger":"elasticsearch.esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.NewConnection","file.name":"eslegclient/connection.go","file.line":132},"message":"elasticsearch url: http://node-02:9200","service.name":"filebeat","ecs.version":"1.6.0"}
   ##### snipped #####