How to deploy OpenTelemetry Collectors in agent and gateway mode

An OpenTelemetry Collector agent-to-gateway deployment keeps application traffic local first, then forwards telemetry to a shared Collector layer for centralized export. Use this pattern when every host or node needs a nearby OTLP endpoint, but backend credentials, heavier processors, or egress control should stay in a smaller gateway tier.

The agent Collector listens for OTLP/gRPC from the application side and exports the same trace data to the gateway Collector over OTLP. The gateway Collector receives traffic from agents and sends it to the destination backend; the local smoke test uses the debug exporter so the handoff can be proven without a vendor endpoint.

The Docker Compose setup runs one agent, one gateway, and one telemetrygen client on a private Compose network. The agent publishes 127.0.0.1:4317 for same-host clients, while the gateway has no host port in the sample. Replace the local insecure agent-to-gateway link with TLS, authentication, retry queues, and backend exporter settings before carrying production traffic.

1. Create the gateway Collector configuration.

   gateway-collector.yaml

   receivers:
     otlp:
       protocols:
         grpc:
           endpoint: 0.0.0.0:4317
   
   processors:
     batch:
       timeout: 2s
   
   exporters:
     debug:
       verbosity: detailed
   
   service:
     pipelines:
       traces:
         receivers: [otlp]
         processors: [batch]
         exporters: [debug]

   The gateway receives OTLP from agents and owns the final exporter. Keep debug only for the smoke test, then replace it with the OTLP, vendor, or backend exporter used by the environment.
   Related: How to export telemetry from the OpenTelemetry Collector with OTLP

2. Create the agent Collector configuration.

   agent-collector.yaml

   receivers:
     otlp:
       protocols:
         grpc:
           endpoint: 0.0.0.0:4317
   
   processors:
     batch:
       timeout: 2s
   
   exporters:
     otlp/gateway:
       endpoint: otel-gateway:4317
       tls:
         insecure: true
   
   service:
     pipelines:
       traces:
         receivers: [otlp]
         processors: [batch]
         exporters: [otlp/gateway]

   tls.insecure: true is only for this private local Compose test. Use TLS and authentication when agents send telemetry across a shared cluster, data center, region, or vendor network.

3. Create the Compose file for the two Collectors and the test client.

   compose.yaml

   services:
     otel-gateway:
       image: otel/opentelemetry-collector-contrib:0.154.0
       command: ["--config=/etc/otelcol-contrib/gateway-collector.yaml"]
       volumes:
         - ./gateway-collector.yaml:/etc/otelcol-contrib/gateway-collector.yaml:ro
   
     otel-agent:
       image: otel/opentelemetry-collector-contrib:0.154.0
       command: ["--config=/etc/otelcol-contrib/agent-collector.yaml"]
       volumes:
         - ./agent-collector.yaml:/etc/otelcol-contrib/agent-collector.yaml:ro
       ports:
         - "127.0.0.1:4317:4317"
       depends_on:
         - otel-gateway
   
     telemetrygen:
       image: ghcr.io/open-telemetry/opentelemetry-collector-contrib/telemetrygen:v0.154.0
       command:
         - traces
         - --otlp-endpoint=otel-agent:4317
         - --otlp-insecure
         - --traces=1
         - --service=checkout-api
       depends_on:
         - otel-agent

   The telemetrygen service sends one trace to the agent over the Compose network. Host-based applications can send to 127.0.0.1:4317 because only the agent publishes that port.

4. Validate the gateway Collector configuration.

   $ docker run --rm --volume "$PWD/gateway-collector.yaml:/etc/otelcol-contrib/gateway-collector.yaml:ro" otel/opentelemetry-collector-contrib:0.154.0 validate --config=/etc/otelcol-contrib/gateway-collector.yaml

   No output with a zero exit status means the gateway file parsed and the referenced components are available in the selected Collector image.

5. Validate the agent Collector configuration.

   $ docker run --rm --volume "$PWD/agent-collector.yaml:/etc/otelcol-contrib/agent-collector.yaml:ro" otel/opentelemetry-collector-contrib:0.154.0 validate --config=/etc/otelcol-contrib/agent-collector.yaml

   Validation checks Collector component names and pipeline wiring, but it does not contact otel-gateway or prove network reachability.

6. Start the agent and gateway Collectors.

   $ docker compose --file compose.yaml up --detach otel-gateway otel-agent
   [+] Running 3/3
    ✔ Network collector_default         Created
    ✔ Container collector-otel-gateway  Started
    ✔ Container collector-otel-agent    Started

7. Confirm that only the agent publishes the local OTLP port.

   $ docker compose --file compose.yaml ps
   NAME                     IMAGE                                          SERVICE        STATUS        PORTS
   collector-otel-agent     otel/opentelemetry-collector-contrib:0.154.0   otel-agent     Up            127.0.0.1:4317->4317/tcp
   collector-otel-gateway   otel/opentelemetry-collector-contrib:0.154.0   otel-gateway   Up            4317/tcp

   The gateway still listens inside the Compose network so the agent can reach otel-gateway:4317, but no host port exposes the gateway in this local setup.

8. Send a trace to the agent Collector.

   $ docker compose --file compose.yaml run --rm telemetrygen
   2026-06-18T07:28:26.037Z INFO starting gRPC exporter
   2026-06-18T07:28:27.043Z INFO traces generated {"worker": 0, "traces": 1}
   2026-06-18T07:28:27.370Z INFO stopping the exporter

   telemetrygen can print extra gRPC connection lines in some releases. The useful signal is the generated trace count and a zero exit status.

9. Check the gateway logs for the forwarded trace.

   $ docker compose --file compose.yaml logs otel-gateway
   otel-gateway | 2026-06-18T07:27:54.647Z info Starting GRPC server {"otelcol.component.id":"otlp","otelcol.component.kind":"receiver","endpoint":"[::]:4317"}
   otel-gateway | 2026-06-18T07:27:54.647Z info Everything is ready. Begin running and processing data.
   otel-gateway | 2026-06-18T07:28:30.693Z info Traces {"otelcol.component.id":"debug","otelcol.component.kind":"exporter","otelcol.signal":"traces","resource spans":2,"spans":2}
   otel-gateway | Resource attributes:
   otel-gateway |      -> service.name: Str(checkout-api)
   otel-gateway | InstrumentationScope telemetrygen
   otel-gateway | Span #0
   otel-gateway |     Trace ID       : df9dc3dd94ed0ed3b62dd6b6fd83df75
   otel-gateway |     Name           : okey-dokey-0
   otel-gateway |     Kind           : Server
   ##### snipped #####
   otel-gateway | Span #1
   otel-gateway |     Trace ID       : df9dc3dd94ed0ed3b62dd6b6fd83df75
   otel-gateway |     Name           : lets-go
   otel-gateway |     Kind           : Client

   The span appears in the gateway log, not only in the agent container, so the data path is telemetrygen → otel-agent → otel-gateway → debug exporter.

10. Stop the local deployment after the smoke test.

    $ docker compose --file compose.yaml down --volumes
    [+] Running 3/3
     ✔ Container collector-otel-agent    Removed
     ✔ Container collector-otel-gateway  Removed
     ✔ Network collector_default         Removed

    For a real deployment, keep the agent and gateway running, remove telemetrygen from the Compose file, and point instrumented services at the agent endpoint.