How to check Logstash pipeline metrics

Logstash pipeline metrics highlight ingestion slowdowns and backpressure early, helping prevent growing queues, delayed indexing, and dropped events under load.

The built-in Logstash monitoring API exposes pipeline statistics as JSON over HTTP, including event counters, queue metrics, and flow-rate indicators such as throughput and worker utilization.

The API typically listens on 9600 and may be configured to bind beyond localhost or move ports, so treat it as sensitive operational data and restrict access when reachable outside the host.

1. Query the Logstash monitoring API root endpoint.

   $ curl -s http://localhost:9600/?pretty=true
   {
     "host" : "host",
     "version" : "8.19.9",
     "http_address" : "127.0.0.1:9600"
   }

   Non-error JSON confirms the monitoring API is responding on localhost:9600.

2. List pipeline names and per-pipeline settings.

   $ curl -s http://localhost:9600/_node/pipelines?pretty=true
   {
     "pipelines" : {
       "main" : {
         "workers" : 1,
         "batch_size" : 125,
         "batch_delay" : 50,
         "config_reload_automatic" : true,
         "dead_letter_queue_enabled" : true
       }
     }
   }

3. Query detailed metrics for a single pipeline.

   $ curl -s http://localhost:9600/_node/stats/pipelines/main?pretty=true
   {
     "pipelines" : {
       "main" : {
         "events" : {
           "in" : 185865,
           "out" : 139675,
           "queue_push_duration_in_millis" : 85400,
           "duration_in_millis" : 86229,
           "filtered" : 139675
         },
         "flow" : {
           "queue_persisted_growth_bytes" : { "current" : -42.62, "last_1_minute" : -14.18, "lifetime" : -10.81 },
           "queue_persisted_growth_events" : { "current" : 4829.0, "last_1_minute" : 1227.0, "lifetime" : 666.8 },
           "worker_utilization" : { "current" : 98.75, "last_1_minute" : 98.94, "lifetime" : 98.11 },
           "output_throughput" : { "current" : 1595.0, "last_1_minute" : 1652.0, "lifetime" : 1589.0 },
           "input_throughput" : { "current" : 6547.0, "last_1_minute" : 1447.0, "lifetime" : 2115.0 },
           "queue_backpressure" : { "current" : 0.9626, "last_1_minute" : 0.9918, "lifetime" : 0.9716 },
           "worker_concurrency" : { "current" : 0.9876, "last_1_minute" : 0.9894, "lifetime" : 0.9811 },
           "filter_throughput" : { "current" : 1595.0, "last_1_minute" : 1652.0, "lifetime" : 1589.0 }
         },
         "queue" : {
           "type" : "persisted",
           "events_count" : 1475830,
           "queue_size_in_bytes" : 1073741826,
           "max_queue_size_in_bytes" : 1073741824
         }
   ##### snipped #####
       }
     }
   }

   Replace main with a pipeline name from the previous step, and keep pretty=true only for human-readable output.

4. Focus on event and queue counters for quick health checks.

   $ curl -s http://localhost:9600/_node/stats/pipelines/main?pretty=true | rg -n -C 2 '"events"|"queue" :|queue_push_duration_in_millis|events_count|queue_size_in_bytes|max_queue_size_in_bytes'
   15-  "pipelines" : {
   16-    "main" : {
   17:      "events" : {
   18-        "in" : 371703,
   19-        "out" : 328634,
   20:        "queue_push_duration_in_millis" : 194268,
   21-        "duration_in_millis" : 194851,
   22-        "filtered" : 328634
   ##### snipped #####
   803:          "queue_size_in_bytes" : 1073741868,
   805:          "max_queue_size_in_bytes" : 1073741824,
   808:        "events" : 1472584,
   809-        "type" : "persisted",
   810:        "events_count" : 1472584,
   811:        "queue_size_in_bytes" : 1073741868,
   812:        "max_queue_size_in_bytes" : 1073741824

   Persistent queues report additional fields such as free_space_in_bytes and path, and on-disk size can drop in chunks when old pages are reclaimed.

5. Focus on flow metrics to detect saturation and backpressure.

   $ curl -s http://localhost:9600/_node/stats/pipelines/main?pretty=true | rg -n -C 2 "input_throughput|filter_throughput|output_throughput|worker_utilization|worker_concurrency|queue_backpressure|queue_persisted_growth"
   23-      },
   24-      "flow" : {
   25:        "queue_persisted_growth_bytes" : {
   26-          "current" : 0.0,
   27-          "last_1_minute" : 0.6851,
   28-          "lifetime" : -4.314
   29-        },
   30:        "queue_persisted_growth_events" : {
   31-          "current" : -2218.0,
   32-          "last_1_minute" : -325.6,
   33-          "lifetime" : 180.3
   34-        },
   35:        "worker_utilization" : {
   36-          "current" : 98.99,
   37-          "last_1_minute" : 98.89,
   38-          "lifetime" : 98.58
   39-        },
   40:        "output_throughput" : {
   41-          "current" : 1708.0,
   42-          "last_1_minute" : 1741.0,
   43-          "lifetime" : 1666.0
   44-        },
   45:        "input_throughput" : {
   46-          "current" : 0.0,
   47-          "last_1_minute" : 1515.0,
   48-          "lifetime" : 1813.0
   49-        },
   50:        "queue_backpressure" : {
   51-          "current" : 0.9999,
   52-          "last_1_minute" : 0.9922,
   53-          "lifetime" : 0.9833
   54-        },
   55:        "worker_concurrency" : {
   56-          "current" : 0.9899,
   57-          "last_1_minute" : 0.9889,
   58-          "lifetime" : 0.9857
   59-        },
   60:        "filter_throughput" : {
   61-          "current" : 1708.0,
   62-          "last_1_minute" : 1741.0

   High worker_utilization (approaching 100) indicates a saturated pipeline, while non-zero queue_backpressure indicates inputs spending time blocked on the queue; plugin-level metrics under plugins can identify which filter/output is consuming worker time.

6. Confirm the monitoring API is bound to a local address on Linux.

   $ sudo ss -lntp | rg ':9600\b'
   LISTEN 0      50     [::ffff:127.0.0.1]:9600             *:*    users:(("java",pid=30226,fd=73))

   Binding to 0.0.0.0:9600 or a public IP exposes pipeline and plugin details to the network.