advanced

Backpressure

Respect producer/consumer speed differences with stream return values, pipeline, queues, limits, and memory-aware flow control.

Backpressure signals that a downstream consumer cannot keep pace — producers must slow down or buffer bounded memory explodes.

In streams, `writable.write(chunk)` returns `false` when internal buffer is full — pause reading until `'drain'`:

					function pump(readable, writable) {
  readable.on('data', (chunk) => {
    const ok = writable.write(chunk);
    if (!ok) readable.pause();
  });
  writable.on('drain', () => readable.resume());
}
				

`pipeline()` coordinates this automatically. Beyond streams: bounded job queues, HTTP 429, Kafka consumer pause, database batch size limits.

On interviews: define backpressure; show manual vs pipeline handling; relate to memory growth under slow clients.

Common pitfalls: unbounded in-memory arrays collecting stream chunks; ignoring `highWaterMark`; no timeout on slow consumers.

The trade-off is balancing simplicity, performance, safety, and operability — name which axis you optimized and what cost you accepted.

Checklist:

  • Prefer pipeline for stream chains.
  • Bound queues with drop or reject policies.
  • Monitor buffer depth and consumer lag.
  • Apply limits at API gateway and app.