1 Overview
A product catalog API looks like an easy caching target until you actually run one. The endpoint is read-heavy, the data changes slowly relative to how often it is requested, and the response is expensive to assemble because it joins products, variants, pricing, inventory, and category metadata. That profile is exactly what caching advice is written for, which is why the first instinct is to wrap the handler in a cache, watch the hit ratio climb, and declare victory.
What I found instead was that the headline number moved while the user-visible behavior barely did. The cache was absorbing a large share of requests, but the requests it absorbed were the cheap ones, and the expensive tail queries kept hammering the datastore. Worse, the moments when the catalog was most volatile, such as a bulk price update or a seasonal re-sort, were exactly the moments when the cache was least useful and most dangerous, because a stale response could show a wrong price or an out-of-stock item as available.
This article is about what actually changed the outcome. It is not a story about a clever library or a single configuration flag. It is about treating the cache as a correctness boundary rather than a throughput knob, restructuring keys around how clients really query, and building invalidation that follows the data model instead of guessing at expiration times. The hit ratio ended up being a lagging indicator, not the goal.
2 Where the common advice breaks down
The standard guidance says to cache the response, pick a time-to-live, and tune the key. That advice assumes requests are roughly uniform and that staleness is a smooth, tolerable gradient. Product catalog traffic is neither. A small number of filter combinations, often driven by a handful of landing pages and search facets, account for most of the volume, while a long tail of arbitrary sort and filter permutations accounts for most of the distinct keys. Caching by raw query string means the popular combinations hit and the long tail misses, so the hit ratio looks respectable while the datastore still sees nearly every distinct query.
The second breakdown is the assumption that time-to-live is a freshness control. In practice a time-to-live is a bet that nothing important changes within the window. For catalog data that bet fails precisely when it matters: a price correction, a stock-out, or a product takedown. When those events happen, a cache that only expires on a timer will serve wrong data for the remainder of the window, and the longer the window, the worse the exposure. Teams often respond by shortening the time-to-live, which quietly converts the cache back into a thin, low-value layer.
The third breakdown is measuring the wrong thing. A high hit ratio on a cache that stores cheap responses buys little, while a modest hit ratio on a cache that absorbs the most expensive queries can transform latency and datastore load. The ratio is a property of the cache, not of the system, and optimizing it directly tends to push teams toward caching things that are easy to cache rather than things that are expensive to compute.
3 How the mechanism actually works
The mechanism that worked was to stop caching whole responses and start caching the pieces that responses are assembled from. A catalog response is a composition: a set of product identities, each with attributes, pricing, and availability, plus ordering and pagination metadata. If you cache the composed response, every distinct ordering is a distinct entry and every underlying change invalidates all of them. If you cache the components and compose on read, a change to one product invalidates one component, and the many orderings that include it are rebuilt cheaply from mostly-warm parts.
This shifts the key design. Component keys are derived from stable entity identity, such as a product identifier plus a version or revision marker, rather than from the request. Request-level keys then become a thin layer that maps a normalized query to an ordered list of component keys, which is itself cheap to recompute or cache briefly. Normalization matters here: collapsing equivalent filters, canonicalizing sort order, and clamping pagination depth all reduce key cardinality without changing what the client sees.
Invalidation then follows the data model. A write to a product publishes an event carrying that product identifier and a new revision. The cache layer evicts the component for that identifier, and any request-level entry that references it is either versioned out or left to expire quickly because it is cheap to rebuild. The time-to-live becomes a backstop for missed events rather than the primary freshness mechanism, which means it can be generous without risking long-lived staleness.
4 Trade-offs you are really making
The first trade-off is freshness against reuse. Component caching with event-driven invalidation gives you both more reuse and tighter freshness than whole-response caching, but only if the event stream is reliable and ordered. If events can be dropped, delayed, or delivered out of order, a component can be marked fresh when it is not, and the failure is silent. You are trading a simple, self-healing timer for a correctness dependency on your write path and message infrastructure.
The second trade-off is memory against compute. Caching components separately means more entries and more bookkeeping, and composition on read costs CPU and serialization work that a single cached blob would avoid. For very small catalogs the composition overhead can exceed the savings, and a plain response cache with a short time-to-live is genuinely the better choice. The component approach pays off when responses are large, when the same entities appear across many orderings, and when change events are frequent enough that timer-based expiry would either be too slow or too aggressive.
The third trade-off is operational complexity against blast radius. Event-driven invalidation is harder to build, test, and reason about than a time-to-live. But it also lets you reason about exactly which responses a given change affects, which means you can make targeted decisions instead of global ones. The honest summary is that you are trading simplicity for control, and control is only worth it if you actually measure and operate the invalidation path rather than assuming it works.
5 Implementing it correctly
Start by separating the read path into two stages: resolving a request into a set of entity references, and materializing those entities into a response. The first stage should be deterministic and cheap, so it can run on every request without a cache. The second stage is where caching lives, keyed by entity identity and revision. This separation is what makes invalidation tractable, because a change to an entity only touches the second stage.
Next, make writes emit change events as part of the same transaction that commits the data, or use an outbox pattern so the event cannot be lost between commit and publish. Each event should carry the entity identifier and a monotonically increasing revision. Consumers apply events idempotently: evicting an already-evicted key is harmless, but applying an older revision after a newer one is not, so guard against out-of-order delivery by comparing revisions rather than trusting arrival order.
Finally, treat the time-to-live as a safety net and set it based on how bad a missed event would be, not on how fresh you want the data to be. Keep request-level composition caches very short-lived, and make sure the composition path degrades gracefully when a component is missing by falling back to the datastore for that component only. Add a way to force a full flush for emergencies, and rehearse it, because the moment you need it is the moment you cannot afford to discover that it is broken.
6 Measuring whether it helped
Hit ratio is the wrong primary metric because it does not distinguish between absorbing a cheap response and absorbing an expensive one. A better starting point is datastore read volume and the distribution of query cost, weighted by how often each query runs. If the cache is doing its job, the expensive queries should disappear from the datastore first, and the reduction in total read work should be visible even when the hit ratio is unremarkable.
Track latency at the tail, not just the median. Caching often helps the median modestly while dramatically improving the slowest requests, because those are the ones that hit the datastore hardest. Look at the ninety-fifth and ninety-ninth percentiles for the endpoint, and compare them against a baseline where the cache is disabled, not against a previous deploy that also changed other things. If the tail does not move, the cache is probably absorbing the wrong requests.
Measure staleness directly. Instrument how long a change takes to be reflected in served responses, and alert on the maximum, not the average. A cache that is usually fresh but occasionally serves a price from an hour ago is a correctness problem, and averages will hide it. Finally, watch the invalidation path itself: event lag, consumer errors, and eviction volume are leading indicators that the cache is drifting out of sync before users notice. If those signals are healthy and the tail latency improved, the change worked, regardless of what the hit ratio says.
7 Frequently asked questions
Is a high hit ratio ever a good sign? It can be, but only alongside a measure of what was avoided. A high ratio on cheap, highly repetitive requests is fine but low-stakes. A moderate ratio on the most expensive queries is usually more valuable. Treat the ratio as a diagnostic, not a target, and always pair it with datastore read volume and tail latency.
Do I need event-driven invalidation, or is a short time-to-live enough? It depends on how costly staleness is and how bursty your writes are. If wrong data is merely inconvenient and writes are rare, a short time-to-live with a plain response cache is simpler and often sufficient. If wrong data is a correctness or trust problem, or if writes are frequent, event-driven invalidation is worth the complexity because it decouples freshness from hit rate.
How do I choose a time-to-live? Set it from the cost of a missed invalidation, not from a desire for freshness. Ask how long the worst-case stale response could persist before someone notices and how bad that would be. Then make the time-to-live short enough to bound that exposure and rely on events for the common case.
What is the biggest failure mode? Silent divergence between the cache and the datastore, usually caused by dropped or reordered events. Guard against it with revisions, idempotent consumers, and alerts on event lag, and keep a tested full-flush path for the cases your guards miss.