How to create a data stream in Elasticsearch

Creating a data stream in Elasticsearch gives logs, metrics, and other append-only events one fixed write target while Elasticsearch rolls hidden backing indices in the background. That keeps ingest clients, searches, and dashboards pointed at one name instead of forcing them to follow rollover index names.

A data stream is created from a matching composable index template. That template must enable data_stream and supply mappings for time-series fields such as @timestamp, while the stream itself writes to a hidden backing index such as .ds-app-logs-2026.04.02-000001. Searches run against the stream name and span all backing indices automatically.

Manual setup is most useful for application-owned pipelines and custom retention flows. A matching template must exist before PUT /_data_stream/<name> succeeds, current Elastic guidance recommends keeping custom template priority above built-in defaults when patterns might overlap, and secured clusters usually require HTTPS, authentication, and template-management privileges. This worked example sets number_of_replicas to 0 so a single-node validation cluster stays GREEN; increase it on multi-node clusters.

1. Create a component template for the example stream's mappings and single-node settings.

   $ curl -sS --fail -H "Content-Type: application/json" -X PUT "http://localhost:9200/_component_template/app-logs-mappings?pretty" -d '{
     "template": {
       "settings": {
         "number_of_replicas": 0
       },
       "mappings": {
         "properties": {
           "@timestamp": { "type": "date" },
           "message": { "type": "wildcard" },
           "service": {
             "properties": {
               "name": { "type": "keyword" }
             }
           }
         }
       }
     },
     "_meta": {
       "description": "Mappings and single-node settings for app-logs data stream"
     }
   }'
   {
     "acknowledged" : true
   }

   This example keeps the data stream GREEN on a one-node cluster by setting number_of_replicas to 0. Raise the replica count when the stream should stay available across multiple data nodes.

   Related: How to create component templates in Elasticsearch

2. Create an index template that matches the stream name and enables data_stream.

   $ curl -sS --fail -H "Content-Type: application/json" -X PUT "http://localhost:9200/_index_template/app-logs-template?pretty" -d '{
     "index_patterns": ["app-logs"],
     "data_stream": {},
     "composed_of": ["app-logs-mappings"],
     "priority": 500,
     "_meta": {
       "description": "Template for app-logs data stream"
     }
   }'
   {
     "acknowledged" : true
   }

   The stream name must match at least one entry in index_patterns. Using priority 500 keeps this custom example above lower-priority built-in templates if patterns ever overlap.

   Related: How to create and manage index templates in Elasticsearch

3. Create the data stream.

   $ curl -sS --fail -X PUT "http://localhost:9200/_data_stream/app-logs?pretty"
   {
     "acknowledged" : true
   }

   Creation fails if an index, alias, or data stream named app-logs already exists.

4. Review the data stream status, generation, and backing index name.

   $ curl -sS --fail "http://localhost:9200/_data_stream/app-logs?pretty&filter_path=data_streams.name,data_streams.status,data_streams.generation,data_streams.indices.index_name"
   {
     "data_streams" : [
       {
         "name" : "app-logs",
         "indices" : [
           {
             "index_name" : ".ds-app-logs-2026.04.02-000001"
           }
         ],
         "generation" : 1,
         "status" : "GREEN"
       }
     ]
   }

   Backing indices roll over behind the stream, so names like .ds-app-logs-2026.04.02-000001 change over time while the write target remains app-logs.

5. Index a test document into the data stream.

   $ curl -sS --fail -H "Content-Type: application/json" -X POST "http://localhost:9200/app-logs/_doc?refresh=wait_for&pretty&filter_path=_index,_id,result" -d '{
     "@timestamp": "2026-04-02T05:00:00Z",
     "message": "login ok",
     "service": {
       "name": "payments"
     }
   }'
   {
     "_index" : ".ds-app-logs-2026.04.02-000001",
     "_id" : "6yjJTJ0B-33KE_yp-11t",
     "result" : "created"
   }

   Data streams accept only create operations. Using POST /<stream>/_doc without an explicit document ID keeps the request in create mode while Elasticsearch generates the ID automatically.

   Documents without an @timestamp field are rejected by this data stream mapping.

6. Search the data stream for the newest event.

   $ curl -sS --fail "http://localhost:9200/app-logs/_search?pretty&size=1&sort=%40timestamp:desc&filter_path=hits.hits._index,hits.hits._id,hits.hits._source"
   {
     "hits" : {
       "hits" : [
         {
           "_index" : ".ds-app-logs-2026.04.02-000001",
           "_id" : "6yjJTJ0B-33KE_yp-11t",
           "_source" : {
             "@timestamp" : "2026-04-02T05:00:00Z",
             "message" : "login ok",
             "service" : {
               "name" : "payments"
             }
           }
         }
       ]
     }
   }

   Searching app-logs spans all backing indices transparently, so there is usually no reason to query .ds-* names directly.