How to create a Kibana data view for Logstash indices

Creating a Kibana data view is required before Discover, visualizations, and dashboards can query Logstash-generated indices with reliable time-based filtering.

A data view (formerly an index pattern) is a Kibana saved object that points to one or more Elasticsearch indices via a wildcard such as logs-* and provides field metadata for search and aggregation features.

Field discovery works best after at least one document exists in the target indices, and the selected time field (commonly @timestamp) must be mapped as a date; deployments using TLS must use https endpoints and a trusted CA in curl.

1. Confirm indices matching the Logstash pattern exist in Elasticsearch.

   $ curl --silent --show-error --fail --user elastic:password "http://localhost:9200/_cat/indices/logs-*?h=index,docs.count"
   logs-2024.01.10 9124

   Replace logs-* with the index name pattern produced by Logstash, such as logstash-*.

   Related: How to configure Logstash output to Elasticsearch

2. Confirm the intended time field exists as a date in the matching indices.

   $ curl --silent --show-error --fail --user elastic:password "http://localhost:9200/logs-*/_field_caps?fields=@timestamp&pretty"
   {
     "indices" : [
       "logs-2024.01.10"
     ],
     "fields" : {
       "@timestamp" : {
         "date" : {
           "type" : "date",
           "searchable" : true,
           "aggregatable" : true
         }
       }
     }
   }

   The timeFieldName must be a date field for Kibana time filtering to work.

3. Create the data view for the Logstash index pattern.

   $ curl --silent --show-error --fail --user elastic:password --request POST "http://localhost:5601/api/data_views/data_view" --header "kbn-xsrf: true" --header "Content-Type: application/json" --data '{
     "data_view": {
       "title": "logs-*",
       "name": "Logstash logs",
       "timeFieldName": "@timestamp"
     }
   }'
   {
     "data_view": {
       "id": "1e2b6b30-7b51-11ee-9c5f-9b2a1d4d1c2a",
       "title": "logs-*"
     }
   }

   Kibana Spaces require inserting /s/<space_id> before /api in the request URL.

4. Fetch the data view by id to confirm creation.

   $ curl --silent --show-error --fail --user elastic:password --header "kbn-xsrf: true" "http://localhost:5601/api/data_views/data_view/1e2b6b30-7b51-11ee-9c5f-9b2a1d4d1c2a"
   {
     "data_view": {
       "id": "1e2b6b30-7b51-11ee-9c5f-9b2a1d4d1c2a",
       "name": "Logstash logs",
       "title": "logs-*",
       "timeFieldName": "@timestamp"
     }
   }