intermediate

Streams

Use streams for append-only event logs with consumer groups when lightweight replay is enough.

Streams are append-only logs identified by auto-generated IDs (timestamp-sequence). Consumer groups let multiple workers claim entries, acknowledge processing, and replay pending messages — a lightweight alternative to Kafka for modest event volumes.

					XADD orders * type "paid" orderId "o-9" amount 42
XGROUP CREATE orders billing $ MKSTREAM
XREADGROUP GROUP billing c1 COUNT 10 BLOCK 5000 STREAMS orders >
XACK orders billing 1718448000123-0
XPENDING orders billing - + 10
				

| Concept | Meaning | |---------|---------| | `XADD` | Append event; `*` auto ID | | Consumer group | Partition work across consumers | | `>` | New messages only | | Pending entries list | Unacked messages — retry target | | Trim (`MAXLEN` / `MINID`) | Bound memory |

Streams give at-least-once within a retained window — not infinite durability. Use explicit trimming and monitor pending list growth. For cross-service contracts, define schema versioning on payload fields.

On interviews: contrast streams with pub/sub and with lists as a queue; mention ack and pending recovery.

Common pitfalls: no trimming → memory blowup; consumers that ack before side effects complete; ignoring poison messages in PEL; expecting Kafka-level retention on default Redis deployment.

The trade-off is built-in consumer groups and replay versus operational limits — streams fit event fan-out with recovery; a dedicated log broker wins at very large scale or long retention.

Checklist:

  • Model event fields and versioning.
  • Create consumer group and ack policy.
  • Trim stream length or age deliberately.
  • Monitor pending entries and lag.