Topology › Load Balancing Algorithms
Load Balancing Algorithms
Policy cards filed under Load Balancing Algorithms.
Load Balancing Algorithms
Stop Round-Robining Everything: A Load Balancer Algorithm Field Guide
LayerLoad balancer connection and request distribution state (per-backend connection pools, health status, and routing tables), not a content cache
TtlHealth check interval and weight decay window; typically seconds for liveness and tens of seconds to minutes for smoothed latency or load estimates
Hit rateShare of requests routed to a backend that can serve them without queueing or retry; affected by backend capacity variance, request cost distribution, connection reuse, and how quickly the algorithm learns changing latency
Round-robin is the default most teams reach for, and for good reason.
Load Balancing Algorithms
Choosing the Right Load Balancing Algorithm: A Practitioner's Guide
LayerLoad balancer connection and session state, often stored in a shared cache or local to the balancer
TtlShort TTLs for health state and connection pools, longer TTLs for session affinity when sticky sessions are required
Hit rateAlgorithm choice, health-check accuracy, session persistence, and traffic skew affect how often requests reuse warm backend connections or cached session data
Load balancing algorithms decide which backend receives each request.
Load Balancing Algorithms
Stop Round-Robin: Hash Your Way to a Smarter Load Balancer
LayerApplication-layer cache sharding across load-balanced backends, including in-memory key-value stores and HTTP reverse-proxy caches.
TtlSet TTLs based on data volatility and tolerance for staleness; use shorter TTLs for rapidly changing data and longer for stable content, with jitter to avoid synchronized expiry.
Hit rateKey distribution uniformity, request skew, backend capacity, and whether the hash function maps hot keys evenly across nodes.
Round-robin load balancing is often the first algorithm engineers reach for because it is simple, stateless, and easy to reason about.
Load Balancing Algorithms
Why Round Robin Fails Your API and What to Use Instead
LayerLoad-balancer-level request distribution and connection pooling, not a content cache; any caching sits behind the balancer and is unaffected by the algorithm choice.
TtlHealth-check interval and failure threshold act as the effective TTL; typical guidance is a few seconds between probes with two to three consecutive failures before eviction.
Hit rateNot a cache metric here; the analogous metric is the fraction of requests routed to a backend that can serve them without queueing, which depends on request cost variance and backend capacity spread.
Round robin is the default load balancing algorithm in most tutorials, and for good reason.
Load Balancing Algorithms
Load Balancing Algorithms: A Field Guide for Your First Production Outage
LayerLoad balancing algorithms operate at the connection and request layer, not a data cache; they distribute traffic across backend servers, which may themselves cache responses.
TtlNo TTL applies to load balancing algorithms; health check intervals and timeouts are the relevant timers, typically set based on application tolerance for stale server state.
Hit rateHit rate is not directly applicable to load balancing; however, consistent hashing can improve cache hit rates on backends by routing the same keys to the same servers.
Load balancing algorithms decide which backend server receives each incoming request.