advanced

Retry queues

Use bounded retry queues or delayed retries with backoff, attempt metadata, observability, and a final failure path.

Retry queues delay failed messages before re-attempting processing. Implement delays with TTL plus DLX hop, dedicated delay queues, or broker plugins — avoid tight spin loops that hammer downstream systems. Store attempt count, first-failure time, and last error in headers or a side table.

Cap maximum attempts and route exhausted messages to a dead-letter queue for human review. Exponential backoff with jitter spreads retry storms after outages. Make retry policy explicit per message type: payment retries differ from email notification retries.

On interviews: sketch a three-queue retry ladder, explain why jitter matters, and define when a failure should not retry.

Common pitfalls: unbounded retries on poison messages; retrying non-idempotent side effects without deduplication; fixed delay that synchronizes thundering herds; no visibility into retry depth.

The trade-off is flexibility versus complexity—know when the simpler path is enough.

Checklist:

  • Bound attempt count per message type.
  • Use backoff with jitter.
  • Persist attempt metadata.
  • Route exhausted messages to DLQ with context.