advanced

Distributed tracing

Propagate trace context across services so latency, errors, retries, and fan-out paths can be diagnosed from one request.

Tracing follows one request across services via trace and span IDs propagated in headers (W3C trace context). Each span records timing, service name, and errors — exposing fan-out, retry duplication, and slow hops in one waterfall view.

Trade-off: instrumentation overhead and storage cost versus mean time to diagnose cross-service incidents.

On interviews: explain span parent-child relationships; how retries appear in traces; sampling strategies at high QPS.

Common pitfalls: broken context propagation on async boundaries; missing spans on message consumers; 100% sampling in production.

Checklist:

Propagate trace context on HTTP, gRPC, and messages.
Name spans after operations, not class names only.
Link logs and metrics to trace IDs.
Sample adaptively under load.