The OTel Collector is a receive process export pipeline that decouples app instrumentation from backend choice and centralizes policies like sampling, redaction, and routing.
Picture a mailroom in a big office building. Every floor drops outgoing letters into a single chute that lands in the mailroom. The mailroom staff sort, weigh, redact sensitive addresses, package the letters into batches, and send each batch to whichever courier company the company uses today. If the courier changes next quarter, the mailroom updates one routing rule; nobody on the floors changes how they write or address their letters. The OTel Collector is that mailroom for tracing data. Apps drop spans into a local chute, the Collector does the sorting, batching, scrubbing, and routing, and the backend can change without touching app code.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Application instrumentation libraries can technically export telemetry directly to a backend. They have OTLP exporters, they can hold auth tokens, they can apply attribute filters. Doing it without a Collector is technically possible. Doing it without a Collector at production scale is operationally painful in ways that only show up six months in, when you want to change backends, add a redaction rule, or sample down high-volume traffic and discover that the change requires touching every service.
The OTel Collector exists to absorb those changes at one configuration boundary. This card walks through the receive process export pipeline, the three pillars (decoupling, policy, resilience) that justify the extra process, the tail-based sampling architecture that pushes most production deployments to a two-tier Collector setup, and the failure modes to plan for.
The pipeline: receivers, processors, exporters
The Collector is structured as a directed graph of three component types.
Receivers
Receivers accept incoming telemetry. The canonical receiver is OTLP over gRPC (port 4317) or HTTP (port 4318), the default OpenTelemetry wire format. Other receivers translate from legacy formats: Jaeger, Zipkin, Prometheus scrape, Fluentd forward, host-metrics from /proc. A Collector can have many receivers active simultaneously; each emits into the same processor pipeline.
Processors
Processors transform telemetry in flight. The most useful ones in LLM observability stacks include batch (group spans for export efficiency), memory_limiter (drop spans when memory pressure builds rather than OOM), attributes (add, update, redact fields), filter (drop spans matching a predicate), tail_sampling (decide keep/drop based on full-trace properties), resource detection (auto-attach cluster, region, k8s pod metadata), and transform (general-purpose attribute manipulation via OTTL).
Exporters
Exporters push processed telemetry to backends. OTLP exporters target any OTel-native backend (Langfuse, Phoenix, Jaeger, Tempo). Vendor-specific exporters target Datadog, Honeycomb, New Relic, Splunk. A pipeline can fan out to multiple exporters (the common dual-export pattern: a hot vendor for incidents plus an archive backend on object storage).
The pipeline is declarative YAML. Adding a new backend means adding an exporter and one line in the pipeline definition; no application code changes.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- The Datadog Agent and the Grafana Alloy distribution are both built on or interoperate with the OpenTelemetry Collector codebase.
- Self-hosted Langfuse deployments commonly front the app with an OTel Collector running the OTLP receiver and a Langfuse exporter, plus an attributes processor for PII redaction.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does the Collector's tail-based sampling decision get made on a trace that spans multiple hosts?
Sidecar Collectors forward spans to a Gateway tier; the Gateway buffers by trace id, waits for a configurable decision-window, and then applies the sampling rule across the complete trace.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating the Collector as optional infrastructure; without one, backend changes, redaction policy, and sampling rules each require a fresh deploy of every service.
60 second bullets to scan on the way to the call.
Three stages: receivers, processors, exporters
Why decoupling app code from backend choice matters
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.