Throughput

Throughput is how much work completes per unit time—requests per second, jobs per minute, megabytes streamed. It rises until a resource saturates: CPU, memory, disk I/O, network, database connections, or external rate limits.

| Bottleneck signal | Typical limit | |-------------------|---------------| | CPU at 100% | Compute-bound handlers, JSON parsing, crypto | | I/O wait high | Database or remote HTTP | | Connection errors | Pool or DB `max_connections` | | 429 responses | Upstream or self rate limits |

Improve throughput with efficient algorithms, batching, async I/O, horizontal scale, and backpressure—not unbounded parallelism. In Node, too many concurrent promises can **reduce** throughput by thrashing memory and the event loop.

On interviews: latency vs throughput trade-off; Little's Law intuition (`concurrency ≈ throughput × latency`); when batching helps; vertical vs horizontal scale.

Common pitfalls: infinite `Promise.all` over DB rows; scaling replicas while DB is saturated; measuring throughput without error rate.

The trade-off is maximizing RPS versus tail latency and failure behavior under overload.

Checklist:

• Measure RPS with error and latency percentiles.
• Find saturating resource with USE metrics.
• Bound concurrency explicitly.
• Load-test before capacity promises.