1 Overview
A common piece of architecture advice says to put the load balancer first and the cache behind it. The reasoning sounds tidy: the load balancer distributes traffic, and the cache sits close to the application servers where it can be managed alongside them. In practice, this ordering often wastes the cache's most valuable property, which is its ability to answer requests without involving the balancing tier at all. When the cache sits in front of the load balancer, a hit never reaches the balancer, never consumes a backend connection, and never touches an application server. That is the case this article examines.
The phrase cache key before load balancer is not about literal network ordering alone. It is about where the cache decision is made relative to routing. If the cache key is evaluated first, the system can short-circuit the request path for repeated content. If the load balancer is evaluated first, every request, including repeat requests for identical content, consumes balancing capacity and backend selection logic. The distinction matters most for read-heavy workloads with high request repetition, where the same objects are requested by many clients.
This is not a universal rule. It is a trade-off that pays off under specific conditions and creates real costs under others. The goal here is to explain the mechanism, the conditions, and the measurement approach so you can decide for your own system rather than adopting a slogan.
2 Where the common advice breaks down
The standard advice assumes that the load balancer is cheap and the cache is expensive or stateful. That assumption comes from an era when caches were application-level and tightly coupled to specific servers. In that model, putting the cache behind the balancer made sense because the cache lived with the application and needed to be addressed per backend. Modern shared caches, however, are often independent services or edge layers that can answer requests without any backend involvement. When that is true, routing every request through the balancer first adds a hop that provides no value for cacheable content.
Another weak point is the claim that caching before the balancer breaks session affinity. Session affinity is a concern for requests that carry user-specific state and cannot be served from a shared cache. Those requests should bypass or miss the cache anyway. The presence of a cache in front of the balancer does not force it to serve personalized responses; it simply gives the system an opportunity to serve shared responses earlier. The failure mode is not caching itself but caching the wrong things, such as responses that vary by user or carry sensitive data.
The advice also tends to ignore the cost of connection handling. A load balancer that terminates TLS, inspects headers, and selects backends does real work per request. If a large fraction of requests are repeats of the same content, that work is duplicated effort. The cache can absorb it, but only if the cache is positioned to see the request before the balancer commits resources to routing it.
3 How the mechanism actually works
When a cache sits before the load balancer, an incoming request first reaches the cache tier. The cache computes a cache key from the request, typically derived from the method, host, path, query string, and selected headers or cookies. If the key matches a stored, unexpired entry, the cache returns the stored response directly. The load balancer is never consulted, no backend is selected, and no application server processes the request. If the key does not match, the request is forwarded to the load balancer, which then routes it to a backend. The backend response may be stored under that key for future requests, depending on cacheability rules.
The key design question is what goes into the key. A key that is too coarse, such as path only, will serve incorrect content to clients that should receive different responses. A key that is too fine, such as including every header or a unique request identifier, will produce almost no hits because every request looks different. The practical approach is to include only the attributes that genuinely change the response body: the path, relevant query parameters, content negotiation headers such as Accept-Encoding, and any cookie or header that selects a variant. Everything else should be excluded so that equivalent requests collapse to the same key.
Invalidation follows the same key. When content changes, the purge must target the exact key or a surrogate key that groups related entries. If the cache key includes attributes that the purge mechanism does not know about, stale content can persist. This is why key design and invalidation design are the same conversation, not separate ones.
4 Trade-offs you are really making
The main trade-off is between offload and routing flexibility. Caching before the load balancer maximizes offload because hits never reach the balancing tier. But it also means the cache must make decisions without the load balancer's knowledge of backend health, capacity, or geographic proximity. If a backend is degraded, the cache cannot route around it for cached content, because cached content does not need a backend at all. For misses, the load balancer still applies its normal logic, so the flexibility loss is limited to the hit path.
A second trade-off involves session affinity and personalization. Requests that must be pinned to a specific backend or that carry user-specific state should not be served from a shared cache. If the cache is in front, it must be configured to bypass or vary on the attributes that make a response personal. This adds configuration complexity and creates a risk of accidentally caching personalized content. The benefit is that genuinely shared content is served earlier, reducing load on both the balancer and the backends.
A third trade-off is operational. Adding a cache tier in front of the load balancer introduces another component to monitor, configure, and invalidate. It can also complicate debugging because a response may come from the cache rather than a backend, and logs must be correlated across tiers. These costs are real and should be weighed against the offload benefit. The choice is not free; it shifts complexity from the backend to the cache layer.
5 Implementing it correctly
Start by classifying content into cacheable and non-cacheable categories. Static assets, public API responses with no user-specific data, and shared fragments are candidates for early caching. Authenticated responses, personalized pages, and anything with per-user state should bypass the shared cache or be varied by the attributes that make them personal. This classification drives both the cache key design and the bypass rules. Without it, you risk serving the wrong content, which is worse than a low hit rate.
Next, define the cache key explicitly. Include the method, host, path, and the query parameters that affect the response. Include content negotiation headers such as Accept-Encoding and Accept-Language only if the origin actually varies on them. Include cookies or headers that select a variant only when necessary, and prefer a small number of well-known variant selectors over broad header inclusion. Document the key so that invalidation and debugging can rely on it. A key that is not documented will drift as the application changes.
Then configure invalidation. Use TTLs as a safety net and explicit purges for content that changes on a known event. Purge by the same key or by surrogate keys that map to groups of entries. Test invalidation in staging, because a purge that targets the wrong key leaves stale content in production. Finally, place the cache tier so that it sees requests before the load balancer, and ensure that bypass rules are enforced at that tier. Monitor hit rate, miss rate, and backend load to confirm the placement is doing what you intended.
6 Measuring whether it helped
The primary metric is cache hit rate, but it must be interpreted alongside request volume and content mix. A high hit rate on a small number of requests may not reduce backend load meaningfully. A lower hit rate on a high-volume path may still produce substantial offload. Break the hit rate down by content class and by key pattern so you can see which parts of the workload benefit. If a key is too granular, the hit rate for that class will be low, and the fix is key design rather than cache placement.
Measure backend request rate and connection count before and after the change. If the cache is absorbing hits before the load balancer, the balancer should see fewer requests and the backends should see fewer still. If those numbers do not move, the cache is either missing often or serving content that was not reaching the backends anyway. Also measure latency at the client and at the cache tier. A hit should be faster than a miss, but the absolute difference depends on network topology and cache location. Do not assume a specific speedup; measure it in your environment.
Finally, track invalidation correctness. Count purges, measure how quickly stale content disappears, and watch for reports of outdated responses. A cache that improves hit rate but serves stale content is not a success. The measurement plan should include both efficiency metrics and correctness checks, because the trade-off between freshness and offload is the core decision this architecture forces.
7 Frequently asked questions
Does putting the cache before the load balancer always improve performance? No. It improves offload for cacheable, repeated content. For workloads with little repetition or mostly personalized responses, the cache adds a hop without reducing backend work. The benefit depends on the hit rate and the cost of the requests that hit.
What about TLS termination and security inspection? Those functions can remain at the load balancer for traffic that reaches it. If the cache terminates TLS, it must handle certificates and security policy for cached content. Many deployments keep TLS at the edge and let the cache operate on already-terminated traffic, but the ordering of those functions is a separate decision from cache placement.
How does this interact with session affinity? Requests that require affinity should not be served from a shared cache, because the cache has no concept of backend sessions. Configure the cache to bypass or vary on the attributes that make a response session-specific. The load balancer still handles affinity for misses and bypassed requests.
Can I use the same cache key for invalidation and lookup? Yes, and you should. A single documented key definition makes both operations consistent. If invalidation uses a different key than lookup, purges will miss entries and stale content will persist. Treat the key as part of the interface between the cache and the application.