Observability Fundamentals: Metrics, Logs, and Traces as One Event Stream
Why the three pillars of observability are not separate silos: unify metrics, logs, and traces into a single correlated telemetry event stream.
On this page
When production incidents strike at 3:00 AM, engineers follow a familiar, exhausting troubleshooting dance:
- An alert fires on a Grafana dashboard showing a spike in HTTP 500 errors.
- The engineer opens Datadog or Kibana to grep through millions of unstructured log lines trying to find a matching timestamp.
- Once an error log is found, the engineer tries to guess which upstream microservice triggered the invalid payload.
This friction exists because organizations treat Metrics, Logs, and Traces as three disconnected data silos managed by three different tools.
True observability does not treat telemetry as three separate pillars. Metrics, logs, and traces are simply three different analytical projections of the exact same underlying reality: timestamped events occurring inside running systems.
The Common Event Stream
Every execution of a function, incoming HTTP request, or database query is fundamentally a discrete event:
{
"timestamp": "2026-06-18T14:32:01.104Z",
"trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
"span_id": "00f067aa0ba902b7",
"service": "checkout-service",
"user_id": "usr_99182",
"route": "/v1/checkout",
"duration_ms": 42.5,
"status_code": 500,
"error_message": "Database connection pool timeout"
}
This single structured event feeds all three observability views:
+-----------------------------------------------------------------------+
| The Three Telemetry Projections |
+-----------------------------------------------------------------------+
| |
| [ Single Event Occurrence ] |
| | |
| +--------------------------+--------------------------+ |
| | | | |
| v v v |
| [ 1. Metrics ] [ 2. Logs ] [ 3. Traces ] |
| - Aggregated counts & - Discrete string / JSON - Causal tree |
| gauges over time record with full context linking hops |
| - Cheap to store & alert - Expensive; high volume - Context ID |
| - Tells you: *IF* broke - Tells you: *WHAT* broke - *WHERE* broke|
+-----------------------------------------------------------------------+
1. Metrics (Aggregations Over Many Events)
A metric strips away individual customer IDs and error strings, retaining only numerical measurements aggregated across time buckets (e.g., counters, gauges, histograms):
http_requests_total{status="500", service="checkout"} += 1
Strength: Extremely cheap to store and query across years of historical data. Metrics are the foundation of real-time alerting.
2. Logs (The Full Record of a Single Event)
A log captures the granular detail of an event at an exact millisecond:
[2026-06-18 14:32:01] ERROR checkout-service: DB connection timeout on pool 'payments_db' for user usr_99182
Strength: Provides the deep context and stack trace necessary to diagnose why a specific operation failed. Limitation: High storage cost; indexing millions of lines per second requires expensive logging infrastructure.
3. Traces (Causal Graphs Connecting Multiple Events)
In distributed microservices, a single customer checkout hops across five distinct services (API Gateway -> Checkout -> Inventory -> Payment -> Notification).
A trace stitches individual service spans together using a shared trace_id propagated across network calls in HTTP headers (W3C traceparent):
[ API Gateway: 45ms ]
├── [ Inventory Service: 8ms ] (200 OK)
└── [ Payment Service: 35ms ] (500 ERROR)
└── [ PostgreSQL Pool: 32ms ] (Timeout)
Strength: Pinpoints the exact hop and service responsible for latency regressions or cascading failures.
Unifying Telemetry with OpenTelemetry
Historically, combining telemetry required installing separate vendor agents: a Prometheus exporter for metrics, Fluentd for logs, and Jaeger for traces.
Modern architectures standardize on OpenTelemetry (OTel):
- A vendor-neutral, open-source collection framework governed by the CNCF.
- Provides unified SDKs across language runtimes (Go, TypeScript, Java, Python).
- Implements a single agent: the OpenTelemetry Collector: that receives, processes, and exports correlated telemetry to any backend (Prometheus, Datadog, Honeycomb, ClickHouse).
Correlating Telemetry in Code
By configuring OpenTelemetry context propagation, every log line automatically inherits the active trace_id and span_id:
// With OpenTelemetry tracing active:
logger.error("Payment failed", {
userId: "usr_99182",
// OTel automatically injects:
// trace_id: "4bf92f3577b34da6",
// span_id: "00f067aa0ba902b7"
});
When an alert fires on a metric spike, an engineer clicks a single button in Grafana to jump directly into the exact distributed trace, and clicks any span in that trace to see the exact associated log lines. Troubleshooting time drops from 45 minutes to 30 seconds.
Key Engineering Takeaways
- Stop treating metrics, logs, and traces as silos: They are three analytical projections of the same underlying event stream.
- Always propagate W3C traceparent headers: Without distributed trace IDs flowing across network calls, debugging microservice dependencies is pure guesswork.
- Adopt OpenTelemetry: Never hardcode vendor-specific proprietary monitoring SDKs into application code. OpenTelemetry provides complete portability across observability backends.
/* Comments */
Comments are offline right now — we reconnect automatically, nothing is lost.