Idempotent consumers

Idempotent consumers produce the same outcome whether a message is processed once or multiple times. Techniques include natural unique keys (order ID), processed-message tables with deduplication windows, compare-and-set on version columns, and outbox patterns that tie publishing to database commits.

Design idempotency at the business-operation level, not only at the transport layer. A handler that charges a card twice is wrong even if the broker delivers exactly once. Store enough state to detect replays within your redelivery window.

On interviews: design idempotency for a `PaymentCaptured` event, compare dedup table versus unique constraint approaches, and explain interaction with at-least-once brokers.

Common pitfalls: idempotency keys that expire before max redelivery time; race between parallel consumers on the same key; side effects outside the transaction (email sent before commit); assuming Kafka offsets alone guarantee business idempotency.

The trade-off is flexibility versus complexity—know when the simpler path is enough.

Checklist:

• Identify the natural idempotency key per message type.
• Persist processing outcome before ack.
• Cover external side effects with outbox or compensations.
• Test duplicate delivery explicitly.