How to configure batch and memory limiter processors in the OpenTelemetry Collector

The OpenTelemetry Collector can receive telemetry faster than a backend can store it, especially when several agents or applications send data through one pipeline. Adding the memory_limiter and batch processors gives the pipeline a memory ceiling and sends telemetry downstream in grouped exports instead of one item at a time.

The processors are defined under processors, but they do not run until they are listed in a service pipeline. Put memory_limiter first so receivers can receive backpressure before later processors allocate more memory, and put batch near the end after processors that drop, sample, or route telemetry.

Use limits that match the Collector deployment boundary. A container with an enforced memory limit can use percentage-based settings, while a fixed host service can use limit_mib and spike_limit_mib values sized from observed throughput. Set GOMEMLIMIT separately when the service wrapper or container runtime allows it, because it controls the Go heap target while memory_limiter controls pipeline backpressure.

1. Open the active Collector configuration file.

   $ sudoedit /etc/otelcol/config.yaml

   The official Docker image reads /etc/otelcol/config.yaml when the file is mounted there. Packaged services may pass a different file with --config in the service unit or deployment manifest.

2. Add the memory_limiter and batch processors to the configuration.

   receivers:
     otlp:
       protocols:
         grpc:
           endpoint: 0.0.0.0:4317
         http:
           endpoint: 0.0.0.0:4318
   
   processors:
     memory_limiter:
       check_interval: 1s
       limit_mib: 400
       spike_limit_mib: 80
     batch:
       timeout: 5s
       send_batch_size: 1024
       send_batch_max_size: 2048
   
   exporters:
     debug:
       verbosity: normal
   
   service:
     pipelines:
       traces:
         receivers: [otlp]
         processors: [memory_limiter, batch]
         exporters: [debug]
       metrics:
         receivers: [otlp]
         processors: [memory_limiter, batch]
         exporters: [debug]
       logs:
         receivers: [otlp]
         processors: [memory_limiter, batch]
         exporters: [debug]

   The debug exporter keeps the smoke test local. Use the same processors list when replacing debug with your OTLP, Prometheus, or vendor exporter.

   limit_mib is the hard target for heap allocation, and spike_limit_mib is subtracted from it to form the soft limit. A common starting point is a spike limit near 20 percent of the hard limit, then adjust from real traffic and host memory.

3. Validate the Collector configuration.

   $ otelcol validate --config /etc/otelcol/config.yaml

   No output with a zero exit status means the file parsed successfully.

4. Start the Collector with the updated configuration.

   $ otelcol --config /etc/otelcol/config.yaml
   2026-06-18T06:14:01.047Z info Memory limiter configured limit_mib=400 spike_limit_mib=80 check_interval=1s
   2026-06-18T06:14:01.048Z info Everything is ready. Begin running and processing data.

   For a system service, restart the unit that owns the Collector process, such as otelcol or otelcol-contrib. For a container, recreate the container with the updated mounted configuration file.

5. Send a sample trace to the OTLP HTTP receiver.

   $ curl -sS -i -X POST http://127.0.0.1:4318/v1/traces \
     -H 'Content-Type: application/json' \
     --data @- <<'JSON'
   {"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"checkout-api"}}]},"scopeSpans":[{"scope":{"name":"manual-smoke"},"spans":[{"traceId":"5b8efff798038103d269b633813fc60c","spanId":"eee19b7ec3c1b174","name":"checkout","kind":1,"startTimeUnixNano":"1717260000000000000","endTimeUnixNano":"1717260001000000000"}]}]}]}
   JSON
   HTTP/1.1 200 OK
   Content-Type: application/json
   Content-Length: 21
   
   {"partialSuccess":{}}

   The HTTP response confirms that the receiver accepted the payload. A backend exporter may return before the backend displays the span because the batch processor waits until timeout or send_batch_size is reached.

6. Confirm the trace leaves the pipeline after the batch timeout.

   2026-06-18T06:14:26.055Z info Traces exporter=debug resource_spans=1 spans=1
   2026-06-18T06:14:26.057Z info ResourceTraces #0 service.name=checkout-api
   ScopeTraces #0 manual-smoke
   checkout 5b8efff798038103d269b633813fc60c eee19b7ec3c1b174

   With a production exporter, confirm the same service name or trace ID in the destination backend instead of keeping the debug exporter enabled.