Data enrichment in Logstash adds lookup results and consistent labels before events are indexed, which makes dashboards, detections, and aggregations depend on stable fields instead of repeated query-time parsing.
Logstash enriches events in the filter stage, where lookup plugins such as geoip, dns, translate, and useragent can add context while transformation filters such as mutate can stamp fields that help route, group, and search the data later. Current releases can keep those fields in an Elastic Common Schema (ECS)-shaped layout, so field paths such as [client][ip] and [client][geo][location] line up cleanly with common Elastic mappings.
Enrichment should stay predictable: lookup databases must stay current, enriched fields must not conflict with older index mappings, and the destination index or template still needs mappings that accept the resulting fields. Secure clusters may also require authentication on the Elasticsearch verification step, and air-gapped environments need a plan for GeoIP database refreshes.
Steps to use Logstash filters for data enrichment:
- Create a pipeline configuration file at /etc/logstash/conf.d/43-enrich.conf.
input { file { path => "/var/lib/logstash/examples/enrich.log" start_position => "beginning" sincedb_path => "/var/lib/logstash/sincedb-enrich" codec => json } } filter { if [log][file][path] == "/var/lib/logstash/examples/enrich.log" { mutate { id => "enrich_labels" add_field => { "[labels][ingest_profile]" => "web_access" "[event][dataset]" => "app.access" } } if [client][ip] { geoip { id => "geoip_client" source => "[client][ip]" target => "client" ecs_compatibility => "v8" } } } } output { if [log][file][path] == "/var/lib/logstash/examples/enrich.log" { elasticsearch { hosts => ["http://elasticsearch.example.net:9200"] index => "logstash-enrich-%{+YYYY.MM.dd}" manage_template => false } } }Current Logstash releases inherit pipeline.ecs_compatibility from the pipeline, and current package defaults commonly use v8. Setting ecs_compatibility => "v8" on the geoip filter keeps the example aligned with ECS-shaped fields even if the pipeline default changes.
The json codec may log an informational warning when it writes parsed fields at the event root without a target. That warning is expected here because the sample input already uses ECS field names such as client.ip, http.request.method, and url.path.
Events without a usable public IP skip the lookup, and failed lookups add the default _geoip_lookup_failure tag.
Default GeoIP database management needs successful update checks. Air-gapped hosts can eventually stop enriching new events with the managed database until the database is refreshed.
- Test the pipeline configuration for syntax errors before restarting the service.
$ sudo -u logstash /usr/share/logstash/bin/logstash --path.settings /etc/logstash --path.data /tmp/logstash-configtest --config.test_and_exit Using bundled JDK: /usr/share/logstash/jdk [2026-04-07T08:23:10,267][INFO ][logstash.javapipeline ][main] Pipeline `main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise. Configuration OK [2026-04-07T08:23:10,271][INFO ][logstash.runner ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash
- Restart the Logstash service and confirm it returns to an active state.
$ sudo systemctl restart logstash $ sudo systemctl status logstash --no-pager -l ● logstash.service - logstash Loaded: loaded (/usr/lib/systemd/system/logstash.service; enabled; preset: enabled) Active: active (running) since Tue 2026-04-07 08:24:41 UTC; 4s ago Main PID: 41476 (java) Tasks: 31 (limit: 28486) Memory: 525.8M (peak: 525.8M) CPU: 16.642s ##### snipped #####Restarting logstash briefly pauses ingestion while the pipeline is reloaded.
- Append a representative JSON event to the example input file.
$ sudo install -d -o logstash -g logstash /var/lib/logstash/examples $ printf '%s\n' '{"@timestamp":"2026-04-07T08:15:00Z","client":{"ip":"8.8.8.8"},"url":{"path":"/checkout"},"http":{"request":{"method":"GET"}}}' | sudo tee -a /var/lib/logstash/examples/enrich.log {"@timestamp":"2026-04-07T08:15:00Z","client":{"ip":"8.8.8.8"},"url":{"path":"/checkout"},"http":{"request":{"method":"GET"}}}If this example file was already consumed, remove or change the configured sincedb_path so the file input re-reads the test line from the beginning.
- Check the monitoring API for the enrichment filters and their event counters.
$ curl -s http://localhost:9600/_node/stats/pipelines/main?pretty { "pipelines" : { "main" : { "plugins" : { "filters" : [ { "id" : "geoip_client", "name" : "geoip", "events" : { "in" : 1, "out" : 1, "duration_in_millis" : 3 } }, { "id" : "enrich_labels", "name" : "mutate", "events" : { "in" : 1, "out" : 1, "duration_in_millis" : 0 } } ] } } } }Explicit plugin IDs make the relevant filters easy to find in the monitoring API output.
The API usually answers on localhost:9600; if that endpoint does not respond, check the configured api.http.host and api.http.port values in /etc/logstash/logstash.yml.
- Query Elasticsearch for a recent event that contains the enriched fields.
$ curl -sS --fail -H "Content-Type: application/json" -X POST "http://elasticsearch.example.net:9200/logstash-enrich-*/_search?pretty" -d '{ "size": 1, "sort": [ { "@timestamp": "desc" } ], "_source": [ "@timestamp", "client.ip", "client.geo.country_name", "client.geo.country_iso_code", "client.geo.location", "event.dataset", "labels.ingest_profile", "http.request.method", "url.path" ], "query": { "bool": { "filter": [ { "term": { "labels.ingest_profile": "web_access" } }, { "exists": { "field": "client.geo.location" } } ] } } }' { "took" : 6, "timed_out" : false, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 1, "relation" : "eq" }, "max_score" : null, "hits" : [ { "_index" : "logstash-enrich-2026.04.07", "_id" : "SP2pmpsBMfcBipKWYBE2", "_score" : null, "_source" : { "@timestamp" : "2026-04-07T08:15:00.000Z", "client" : { "ip" : "8.8.8.8", "geo" : { "country_name" : "United States", "country_iso_code" : "US", "location" : { "lat" : 37.751, "lon" : -97.822 } } }, "event" : { "dataset" : "app.access" }, "labels" : { "ingest_profile" : "web_access" }, "http" : { "request" : { "method" : "GET" } }, "url" : { "path" : "/checkout" } }, "sort" : [ 1775552100000 ] } ] } }Add authentication options when Elasticsearch security is enabled, and adjust the host name or index pattern to match the target cluster.
If older indices already mapped client as a scalar value instead of an object, this enrichment can fail with a mapping conflict. Apply the change to a fresh index or update the destination template before mixing old and new document shapes.
Mohd Shakir Zakaria is a cloud architect with deep roots in software development and open-source advocacy. Certified in AWS, Red Hat, VMware, ITIL, and Linux, he specializes in designing and managing robust cloud and on-premises infrastructures.
