1 Overview
Hit ratio is the fraction of requests served without touching the origin. It is the single most quoted number in caching discussions and also the most misunderstood, because it is a ratio of two things that both move. Teams often treat a low hit ratio as a configuration bug and reach for a familiar list of fixes: cache everything, raise the TTL, add more cache nodes. Those fixes sometimes help and sometimes quietly make things worse by serving stale or incorrectly scoped content to the wrong users.
The myths in this article are not strawmen. They are pieces of advice that circulate because they are true under specific conditions, then get repeated without those conditions. The goal here is not to tell you to cache less. It is to separate the mechanism from the folklore so you can reason about your own traffic. A cache is a bet that a future request will resemble a past one. Every myth worth debunking is really a mistaken claim about which requests resemble which others, and how long that resemblance holds.
This piece covers the reverse-proxy and edge response cache layer, where the unit of storage is a full HTTP response keyed by a URL and a chosen set of request attributes. The same reasoning applies to object caches and database query caches, but the key construction and invalidation mechanics differ enough that the details below should be read as specific to HTTP response caching.
2 Where the common advice breaks down
The first myth is that a higher hit ratio is always better. A cache that ignores the Authorization header, or that keys only on path while ignoring query parameters, can post an excellent hit ratio by serving one user's private data to another. The ratio looks healthy while the system is broken. Correctness constraints cap the useful hit ratio, and any number above that cap is a defect rather than an achievement.
The second myth is that longer TTLs are free wins. TTL is a staleness budget, not a performance dial. Extending it reduces origin traffic only for content that is actually requested again within the window, and it increases the time during which a bad or outdated response can be served. For low-traffic URLs, a long TTL mostly stores bytes that expire unused, consuming memory that could hold genuinely hot objects.
The third myth is that adding cache nodes or capacity raises the hit ratio. Capacity helps only when eviction is the binding constraint. If the working set fits comfortably and the ratio is still low, the problem is key fragmentation, uncacheable responses, or traffic that is genuinely unique per request. Adding nodes in that situation spreads the same fragmented keys across more places, which can lower the ratio by giving each node a thinner slice of the request stream. Diagnose the constraint before scaling the tier.
3 How the mechanism actually works
A response cache is a dictionary. The lookup key is derived from the request, usually the method, host, path, and a selected subset of headers and query parameters. On a hit, the stored response is returned with its recorded freshness metadata. On a miss, the request goes upstream, and the response may be stored if it is deemed cacheable. Everything about hit ratio follows from two questions: how many distinct keys does your traffic produce, and how often is each key requested again before it is evicted or expires.
Key cardinality is the dominant factor and the one most often ignored. A URL with a cache-busting timestamp, a session identifier, or a full set of tracking parameters generates a near-unique key per request, so no two requests ever share a stored response. The cache is working exactly as configured and still yields almost no hits. This is why normalizing or stripping irrelevant query parameters, and moving volatile values into headers the cache ignores, can raise the ratio more than any TTL change.
Eviction is the second factor. Caches are finite, so they must discard entries, typically by some recency or frequency heuristic. When the working set of actively requested keys exceeds capacity, entries are evicted before they are reused, and the ratio collapses even though every individual response was cacheable. This is a sizing problem, not a configuration problem, and it is visible as a high miss rate concentrated on keys that were recently stored.
4 Trade-offs you are really making
Every caching decision trades freshness and correctness against origin load and latency. Broadening the key to share responses across more requests raises the hit ratio but also raises the chance that a response meant for one context is served in another. Narrowing the key protects correctness but fragments the cache and lowers the ratio. There is no setting that maximizes both; you are choosing where on that curve your application can tolerate being.
The second trade-off is memory against origin traffic. Holding more entries for longer costs edge storage and can push out hotter objects if eviction is not frequency-aware. Holding fewer entries for less time shifts load back to the origin and the database behind it. The right balance depends on the cost of a miss relative to the cost of storage, and those costs differ enormously between a static asset and an expensive rendered page.
The third trade-off is invalidation complexity against staleness risk. Purging precisely on every write keeps content fresh but requires the application to know every derived key, which is hard when one write affects many URLs. Relying on TTL alone is simple but leaves a window of staleness whose length you cannot control per event. Most systems end up with a hybrid: explicit purge for the keys the application can enumerate, and short TTLs as a backstop for the ones it cannot.
5 Implementing it correctly
Start by making cacheability explicit rather than assumed. Decide per route whether a response may be stored, who may share it, and for how long, and encode that in response headers so the cache and any intermediary agree. Responses that vary by user identity should be marked private or non-storable unless you have a deliberate shared-cache design with a key that captures the variation. Ambiguity here is what turns a hit-ratio optimization into an incident.
Next, control key cardinality deliberately. Strip parameters that do not change the response, sort the ones that remain so ordering does not create duplicate keys, and keep volatile values out of the key when they can be handled by varying on a header instead. For assets that change only when their content changes, embed a content hash in the filename so the URL itself is the version and the TTL can be very long without any purge logic at all.
Finally, treat invalidation as a first-class part of the write path. When data changes, publish a purge for the specific keys derived from it, and accept that some keys will only be corrected by TTL expiry. Keep the TTL short enough that the worst-case staleness is acceptable and long enough that the origin sees meaningful relief. Document which routes use which strategy so the next person changing a response does not silently break the assumptions the cache depends on.
6 Measuring whether it helped
Measure hit ratio by key class, not as a single global number. A blended ratio hides the fact that static assets may be hitting at a very high rate while dynamic pages miss almost always, which means the aggregate number is dominated by whichever class has the most requests. Break traffic into groups by route or content type, and track hits, misses, and evictions per group so you can see which part of the system is actually benefiting.
Watch the ratio alongside origin load and latency, not instead of them. A rising hit ratio that does not reduce origin requests or response times is not delivering value, which usually means the hits are on cheap responses while the expensive ones still miss. Conversely, a modest ratio improvement on a small number of high-cost routes can matter more than a large improvement on trivial ones. Attribute the benefit to the requests that were actually expensive.
Finally, measure staleness and correctness. Track how often purges are issued, how long content remains stale after a write, and whether any response was served to a context it did not belong to. A hit ratio is only meaningful within a correctness envelope; if you cannot measure whether the right content reached the right user, you cannot tell an optimization from a regression. Set the envelope first, then optimize inside it.
7 Frequently asked questions
Is a higher hit ratio always the goal? No. The goal is serving correct responses with less origin work. Once correctness constraints are met, the useful ceiling on hit ratio is set by how much of your traffic is genuinely repeatable. Chasing a number past that ceiling usually means weakening key scoping or extending TTLs beyond what the content tolerates.
Should I raise TTLs to improve the ratio? Only if the content can be stale for that long and is requested often enough to be reused within the window. For frequently requested, slowly changing content, longer TTLs help. For rarely requested or fast-changing content, they mostly consume memory and increase staleness risk without adding hits.
Why does adding cache capacity not help? Because capacity only matters when eviction is the constraint. If your keys are fragmented by volatile parameters or your responses are marked uncacheable, more memory stores the same non-reusable entries. Fix key construction and cacheability first, then size the tier to the working set that remains.
How do I know if my key is too broad or too narrow? Look for the two failure signatures. Too broad shows up as stale or cross-context content and correctness complaints. Too narrow shows up as a large number of keys requested once and never again. Both are visible in per-key request counts, which are worth sampling directly rather than inferring from the aggregate ratio.