How to create an Elasticsearch user for Logstash output

Creating a dedicated Elasticsearch identity for Logstash output limits write access to only the index pattern the pipeline should populate, avoiding the habit of pointing ingestion at the built-in elastic superuser.

Elasticsearch authorizes bulk indexing through roles and users exposed by the /_security/ APIs. Current Elastic guidance keeps the role and the user separate: a custom role such as logstash_writer carries the cluster and index privileges, while a dedicated user such as logstash_internal authenticates the elasticsearch output plugin. The built-in logstash_system account remains a monitoring user and is not intended for pipeline output writes.

Privilege scope must match the actual target pattern and lifecycle behavior. If Logstash installs index templates or manages ILM for the target indices, the role needs the related cluster and index privileges; if those jobs are handled elsewhere, those permissions should be removed. Secured clusters also require a trusted HTTP CA path for both curl administration and the Logstash output block, and pipeline secrets should be stored in the Logstash keystore or another secret manager instead of plain text.

Steps to create an Elasticsearch user for Logstash output:

Create a role that matches the Logstash target index pattern and lifecycle needs.

$ curl --silent --show-error --fail --user elastic:password --header "Content-Type: application/json" --request PUT "https://localhost:9200/_security/role/logstash_writer?pretty" --data '{
  "cluster": ["manage_index_templates", "monitor", "manage_ilm"],
  "indices": [
    {
      "names": ["logs-*"],
      "privileges": ["write", "create", "create_index", "manage", "manage_ilm"]
    }
  ]
}'
{
  "role" : {
    "created" : true
  }
}

Change logs-* to the actual output index or data stream pattern. Remove the template or ILM privileges if those parts are managed outside Logstash or are disabled in the output plugin.

Create a dedicated user and assign the new role.

$ curl --silent --show-error --fail --user elastic:password --header "Content-Type: application/json" --request PUT "https://localhost:9200/_security/user/logstash_internal?pretty" --data '{
  "password": "strong-password",
  "roles": ["logstash_writer"]
}'
{
  "created" : true
}

Credentials passed on the command line can be exposed via shell history or process listings on multi-user systems.

Confirm the user is enabled and mapped to the expected role.

$ curl --silent --show-error --fail --user elastic:password "https://localhost:9200/_security/user/logstash_internal?pretty&filter_path=*.username,*.roles,*.enabled"
{
  "logstash_internal" : {
    "username" : "logstash_internal",
    "roles" : [
      "logstash_writer"
    ],
    "enabled" : true
  }
}

Update the Logstash elasticsearch output to use the dedicated user and the cluster CA certificate.
```
output {
  elasticsearch {
    hosts => ["https://elasticsearch.example.net:9200"]
    ssl_enabled => true
    ssl_certificate_authorities => ["/etc/logstash/certs/http_ca.crt"]
    user => "logstash_internal"
    password => "${LOGSTASH_INTERNAL_PASSWORD}"
    index => "logs-%{+YYYY.MM.dd}"
  }
}
```
Prefer storing the password in the Logstash keystore and referencing it via ${LOGSTASH_INTERNAL_PASSWORD} instead of placing secrets directly in pipeline files.

Related: How to configure Logstash output to Elasticsearch
Related: How to add a secret to a Logstash keystore

Test the Logstash pipeline configuration using a temporary path.data directory.

$ sudo -u logstash /usr/share/logstash/bin/logstash --path.settings /etc/logstash --path.data /tmp/logstash-configtest --config.test_and_exit
Using bundled JDK: /usr/share/logstash/jdk
Configuration OK
[2026-04-02T08:11:41,726][INFO ][logstash.runner          ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash

Configuration validation confirms pipeline syntax and plugin settings, not Elasticsearch credentials or index privileges.

Restart the Logstash service to load the updated pipeline credentials.
```
$ sudo systemctl restart logstash
```
Related: How to manage the Logstash service with systemctl in Linux

Review recent Logstash logs for authentication or authorization failures.

$ sudo journalctl --unit logstash --since "5 minutes ago" --no-pager
Jan 07 11:38:50 host logstash[20457]: [2026-01-07T11:38:50,351][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://logstash_internal:xxxxxx@elasticsearch.example.net:9200/]}}
Jan 07 11:38:50 host logstash[20457]: [2026-01-07T11:38:50,425][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://elasticsearch.example.net:9200/]}}
##### snipped #####

Authentication failures typically show 401 or 403 responses, while index privilege problems usually mention missing rights to create the index, write documents, or manage ILM.

Verify matching indices are receiving documents.

$ curl --silent --show-error --fail --user elastic:password "https://localhost:9200/logs-*/_count?pretty"
{
  "count" : 1,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  }
}

Use a credential with read privileges for this verification query; the dedicated Logstash output user can stay write-only.

Author: Mohd Shakir Zakaria
Mohd Shakir Zakaria is a cloud architect with deep roots in software development and open-source advocacy. Certified in AWS, Red Hat, VMware, ITIL, and Linux, he specializes in designing and managing robust cloud and on-premises infrastructures.