core practice · system design prompt
Design exercise: Observability & OpenTelemetry
Metrics, logs, traces, context propagation, semantic conventions, sampling, collectors, and telemetry pipelines. Implement designOutline() returning non-empty values for: signals, contextPropagation, sampling. Each value must name a concrete mechanism or decision.
The mechanism
The three pillars answer different questions: metrics say something is wrong, traces say where, logs say why. A structured log plus a request id is what stitches them together. OpenTelemetry is the reason that stitching survives a service hop — it standardises the signals and, more importantly, propagates trace context across process boundaries, so instrumentation is portable and a trace does not end at the edge of the service that started it.
Expected evidence
A design outline with signals, contextPropagation, sampling plus an explicit failure mode or trade-off.
Try this drill in the practice workspace →
Hints when you need them
- Name the failure mode before selecting the mechanism.
Watch for these mistakes
- Unstructured logs you cannot query
- Alerting on causes instead of user-facing symptoms (SLOs)
- No correlation id across services
- Instrumenting with a vendor SDK, so switching backends means re-instrumenting everything
- Dropping trace context at an async boundary, which silently truncates every downstream span
- Head sampling at a rate that discards exactly the rare failures you needed
Check the sources
- Site Reliability Engineering — Monitoring Distributed Systems
- Canopy: An End-to-End Performance Tracing And Analysis System (SOSP '17)
- USENIX SREcon22 Asia/Pacific — OpenTelemetry and Observability: What, Why, and Why Now?
- Dapper, a Large-Scale Distributed Systems Tracing Infrastructure
- OpenTelemetry Documentation