intermediate

RabbitMQ work queues

Use work queues for task distribution with competing consumers, explicit ack behavior, bounded retries, and operational visibility.

Work queues distribute tasks among multiple consumers attached to the same queue. RabbitMQ delivers round-robin by default; prefetch and consumer priority adjust fairness. Use durable queues, persistent messages, and manual ack for tasks that must survive restarts.

Design visibility: expose queue depth, consumer count, processing latency, and failure rates. Bound retries with DLX and cap requeue attempts. For ordered tasks per entity, use consistent-hash exchange or shard into multiple queues instead of one global work queue.

On interviews: describe scaling workers for an image-processing backlog, explain competing consumers versus pub/sub, and list metrics you would alert on.

Common pitfalls: single queue for heterogeneous job durations; no idempotency when workers retry; scaling consumers without broker capacity planning; treating work queues as a log with no retention policy.

The trade-off is flexibility versus complexity—know when the simpler path is enough.

Checklist:

  • Use durable queues and persistent messages for tasks.
  • Tune prefetch for job duration mix.
  • Add DLX and bounded retries.
  • Monitor depth, lag, and consumer health.