advanced
Backpressure
Respect producer/consumer speed differences with stream return values, pipeline, queues, limits, and memory-aware flow control.
Backpressure signals that a downstream consumer cannot keep pace — producers must slow down or buffer bounded memory explodes.
In streams, `writable.write(chunk)` returns `false` when internal buffer is full — pause reading until `'drain'`:
function pump(readable, writable) {
readable.on('data', (chunk) => {
const ok = writable.write(chunk);
if (!ok) readable.pause();
});
writable.on('drain', () => readable.resume());
}
`pipeline()` coordinates this automatically. Beyond streams: bounded job queues, HTTP 429, Kafka consumer pause, database batch size limits.
On interviews: define backpressure; show manual vs pipeline handling; relate to memory growth under slow clients.
Common pitfalls: unbounded in-memory arrays collecting stream chunks; ignoring `highWaterMark`; no timeout on slow consumers.
The trade-off is balancing simplicity, performance, safety, and operability — name which axis you optimized and what cost you accepted.
Checklist:
- Prefer pipeline for stream chains.
- Bound queues with drop or reject policies.
- Monitor buffer depth and consumer lag.
- Apply limits at API gateway and app.