Kafka compaction

Compaction retains the latest record per key, deleting older versions — useful for changelog topics representing entity state (user profile, config). Tombstones with null value delete keys after delete.retention.ms.

Trade-off: lose full event history; consumers rebuilding state must handle compaction semantics and tombstones.

On interviews: compacted topic for product catalog updates versus immutable order event log.

Common pitfalls: compacting topics that need full history; missing tombstones leaving zombie keys; consumers assuming all events replay.

Checklist:

• Use compaction only for keyed latest-value streams.
• Publish tombstones on delete with correct timing.
• Test consumer rebuild from compacted log.
• Separate changelog topics from immutable event logs.