Load balancing

Load balancers distribute traffic across healthy backends—L4 (TCP) or L7 (HTTP)—using algorithms like round robin, least connections, or consistent hash for limited stickiness.

| Feature | Purpose | |---------|---------| | Health checks | Stop sending to failing instances | | TLS termination | Centralize certificates at edge | | Retries | Retry idempotent requests on another node—carefully | | Timeouts | Prevent hung connections consuming slots |

					Client → ALB/Ingress → Service endpoints (ready pods only)
				

Configure **graceful shutdown**: app drains in-flight requests before SIGKILL so LB does not route to dying pods. Retry storms happen when clients and LB both retry 5xx without jitter.

On interviews: L4 vs L7; when sticky sessions help WebSockets; retry safety; connection draining during deploy.

Common pitfalls: health check too shallow (200 OK while DB dead); infinite retries amplifying overload; ignoring cross-AZ latency.

The trade-off is even spread versus session affinity and retry-induced load multiplication.

Checklist:

• Readiness reflects real dependencies.
• Drain connections on shutdown.
• Limit retries with backoff and idempotency keys.
• Monitor per-backend error rates.