advanced

Distributed tracing

Follow a request through frontend, gateway, service, database, and worker hops to locate latency and failure boundaries.

Distributed tracing follows one logical operation across processes: browser → API → database → queue → worker → third-party HTTP. It exposes where latency accumulates and which dependency owns a failure.

					[browser]──►[api]──►[postgres]
                └──►[payment-api]
                └──►[queue]──►[worker]
				

Sampling balances cost and coverage: always trace errors or slow paths; sample happy paths. Traces become useful when they share IDs with logs and metrics exemplars.

On interviews: critical path for checkout, head-based vs tail-based sampling, and tracing async work that continues after the HTTP response.

Common pitfalls: broken context at queue boundaries; tracing only the frontend; spans too granular (every function) or too coarse (one span per service).

The trade-off is storage and instrumentation overhead versus mean time to understand cross-service incidents.

Checklist:

  • Propagate context on HTTP and messaging.
  • Create spans at dependency boundaries.
  • Link traces to logs via trace ID.
  • Sample strategically, not uniformly 100%.