How to ship logs from Filebeat to Elasticsearch

Sending logs directly from Filebeat to Elasticsearch simplifies the pipeline. This approach avoids intermediate processing tools, making it a quick solution for smaller setups.

By defining the output.elasticsearch section, Filebeat can write directly to an index. Authentication and TLS options ensure secure transmission.

Direct ingestion enables near real-time visibility of logs in Kibana without extra configuration.

1. Edit filebeat.yml and locate the output.elasticsearch section.

   $ sudo nano /etc/filebeat/filebeat.yml
   (no direct output)

   Point Filebeat directly to Elasticsearch by specifying hosts and credentials.

2. Set hosts to “http://localhost:9200” or your Elasticsearch endpoint.
3. Include credentials if X-Pack Security is enabled.
4. Test configuration.

   $ sudo filebeat test output
   elasticsearch: http://localhost:9200 ... OK

   filebeat test output verifies connectivity to Elasticsearch.

5. Restart Filebeat.

   $ sudo systemctl restart filebeat
   (no output)

   Check Elasticsearch indices for incoming logs. Adjust index names if needed.

6. Confirm log data is visible in Kibana.

   Direct output reduces complexity for simpler environments.