intermediate
Streams
Use streams for append-only event logs with consumer groups when lightweight replay is enough.
Streams are append-only logs identified by auto-generated IDs (timestamp-sequence). Consumer groups let multiple workers claim entries, acknowledge processing, and replay pending messages — a lightweight alternative to Kafka for modest event volumes.
XADD orders * type "paid" orderId "o-9" amount 42
XGROUP CREATE orders billing $ MKSTREAM
XREADGROUP GROUP billing c1 COUNT 10 BLOCK 5000 STREAMS orders >
XACK orders billing 1718448000123-0
XPENDING orders billing - + 10
| Concept | Meaning | |---------|---------| | `XADD` | Append event; `*` auto ID | | Consumer group | Partition work across consumers | | `>` | New messages only | | Pending entries list | Unacked messages — retry target | | Trim (`MAXLEN` / `MINID`) | Bound memory |
Streams give at-least-once within a retained window — not infinite durability. Use explicit trimming and monitor pending list growth. For cross-service contracts, define schema versioning on payload fields.
On interviews: contrast streams with pub/sub and with lists as a queue; mention ack and pending recovery.
Common pitfalls: no trimming → memory blowup; consumers that ack before side effects complete; ignoring poison messages in PEL; expecting Kafka-level retention on default Redis deployment.
The trade-off is built-in consumer groups and replay versus operational limits — streams fit event fan-out with recovery; a dedicated log broker wins at very large scale or long retention.
Checklist:
- Model event fields and versioning.
- Create consumer group and ack policy.
- Trim stream length or age deliberately.
- Monitor pending entries and lag.