advanced

Kafka topics

Model topics around durable event streams with clear names, schemas, ownership, retention expectations, and compatibility rules.

Kafka topics are durable, partitioned logs named by domain (e.g., orders.events). Design includes ownership team, schema subject, partition count, retention, compaction policy, and ACLs. Topic sprawl without governance becomes a integration maze.

Trade-off: fine-grained topics simplify consumers versus operational overhead; coarse topics mix ordering concerns.

On interviews: name and partition a user-activity stream; who owns schema and consumer groups.

Common pitfalls: too few partitions blocking scale; PII in broadly subscribed topics; no naming convention.

Checklist:

Convention: domain.entity.action or similar.
Document owner, schema, and subscribers.
Right-size partitions for throughput and consumers.
Set retention and cleanup policies explicitly.