Even when a service reports healthy, a load balancer can still route traffic to a dead instance. The problem stems from how health checks are performed and who runs them. In a server‑side model, a central proxy like HAProxy probes backends, while client‑side routing pushes the logic into each service.

Server‑side health checks run on a fixed schedule: a TCP connect or an HTTP /health probe every 5 seconds, waiting up to two seconds for a reply. Two successes lift an instance to healthy, three failures mark it unhealthy. This design avoids flapping but can delay failure detection by up to 15 seconds.

Client‑side routing distributes health logic across every caller. Active checks mirror the server‑side probe, but each of the 500 clients might ping 20 instances every 5 seconds, generating 2,000 probe requests per second before real traffic. Passive checks watch real responses, marking an instance unhealthy immediately after a timeout or 500 error.

Choosing a model hinges on scale and operational simplicity. Server‑side load balancing offers a single, consistent view and instant removal of unhealthy nodes, ideal for modest fleets. Client‑side shines when thousands of services call each other, eliminating a proxy bottleneck and delivering sub‑millisecond failure detection, though it spreads complexity across libraries.