API gateway observability

The gateway is the natural telemetry aggregation point: every external request passes through. Instrument request ID generation/propagation, structured access logs, latency histograms, error rates per route and consumer, and distributed traces across upstream hops.

					trace-id: abc123
gateway_span: 45ms (auth 5ms, route 2ms, upstream 35ms, transform 3ms)
				

Per-consumer SLOs and error budgets catch partner integrations breaking before origin-only metrics reveal it. Log policy decisions (blocked, rate-limited, auth failed) with reason codes — not raw tokens.

On interviews: design dashboards for edge vs origin latency split; sampling strategies under high QPS; correlating 502/504 with specific upstream pools.

Common pitfalls: logging Authorization headers; missing upstream span when proxy hides failures; alert fatigue on WAF blocks unrelated to user impact.

The trade-off is balancing simplicity, performance, safety, and operability — name which axis you optimized and what cost you accepted.

Checklist:

• Generate or forward a correlation ID on every request.
• Break down latency: auth, routing, upstream, transforms.
• Track 4xx/5xx by route, consumer, and upstream target.
• Redact secrets from gateway access logs.