advanced

Kafka replication

Balance replication factor, in-sync replicas, acknowledgements, and leader election to trade throughput for durability and availability.

Each partition has a leader and followers replicating the log. Replication factor and min.insync.replicas define how many copies must acknowledge a write before it is considered committed. Leader election promotes a follower when the leader fails.

Trade-off: higher replication and stricter acks improve durability but add latency and broker load.

On interviews: explain acks=all with min.insync.replicas=2 on a three-broker cluster; unclean leader election risk.

Common pitfalls: replication factor 1 in production; acks=1 losing data on leader crash; ISR shrink without alerting.

Checklist:

Set RF and min.insync.replicas for durability goals.
Monitor under-replicated and offline partitions.
Test broker failure and leader election drills.
Document producer ack configuration per topic class.