The Cache & Edge JournalCache & edge engineering

CACHE AND EDGE, MAPPED

See where the request goes before you tune the cache.

A working map of caching, CDNs, and load balancing. Each policy card states the cache layer, the hit-rate levers, the invalidation strategy, the TTL guidance, the trade-off, and the cost impact up front.

Layer key

Browse the topology

11 entries Caching Strategies 8 entries Case Studies 5 entries Load Balancing Algorithms 6 entries Performance Tuning

Caching Strategies

All Caching Strategies →
Caching Strategies Stop Setting Max-Age: Use stale-while-revalidate for Cache Control LayerHTTP response caching at shared proxies, CDN edge nodes, and browser caches governed by Cache-Control directives TtlShort max-age for freshness plus a longer stale-while-revalidate window during which stale copies may be served while revalidation happens Hit rateAffected by how long responses remain usable, whether revalidation is required before reuse, how many distinct URLs and variants exist, and how often origin content changes Most cache control advice reduces to a single instruction: set a max-age and let the clock decide when content is fresh. Caching Strategies Stop Blaming the Load Balancer: Fix Your Cache Key First LayerHTTP response caching at the edge or reverse proxy layer, keyed by request attributes TtlSet TTL based on content volatility and tolerance for staleness; use short TTLs for frequently changing data, longer for stable assets, and consider stale-while-revalidate patterns Hit ratePrimarily determined by cache key design, request normalization, and the diversity of query parameters, headers, and cookies included in the key When latency spikes or origin load climbs, the load balancer is often the first suspect. Caching Strategies Cache Like a Pro: A Field Guide for the Overworked Architect LayerApplication-level read-through cache with a distributed key-value store, often paired with an in-process cache for hot keys. TtlShort TTLs (seconds to minutes) for volatile data, longer TTLs (hours to days) for stable reference data, tuned to freshness requirements. Hit rateKey distribution, request locality, cache size relative to working set, eviction policy, and TTL all influence the hit rate. Caching is the art of keeping a copy of data closer to where it is needed, trading memory and staleness for speed and reduced load on origin systems. Caching Strategies Five Caching Myths That Are Slowing Down Your Site LayerShared HTTP response caching at the edge and reverse-proxy layer, plus application-level object caching for computed data TtlLong TTLs for immutable, fingerprinted assets; short TTLs for personalized or fast-changing responses; add stale-while-revalidate where freshness can lag slightly Hit rateRequest diversity, key design, cacheable-response ratio, TTL length, eviction pressure, and how much of the working set fits in memory Caching advice travels well because it sounds universal, and that is exactly why so much of it is wrong in practice. Caching Strategies Cache-Aside vs. Write-Through: Which Caching Strategy Wins? LayerApplication-level cache in front of a primary data store, typically a key-value store such as Redis or an in-process cache, used for read-heavy object lookups. TtlUse a TTL as a safety net rather than the primary freshness mechanism: short TTLs for volatile or user-specific data, longer TTLs for stable reference data, plus jitter to avoid synchronized expiry storms. Hit rateDominated by access skew, key cardinality, working-set size versus cache capacity, and TTL length; write-through can lower hit rate for cold or rarely re-read keys because it populates entries that may never be read again. Cache-aside and write-through are two ways to keep a cache and a primary data store consistent, and they differ mainly in who is responsible for filling and updating the cache. Caching Strategies Why Your Cache Strategy Is Backwards: Rethink the Edge First LayerEdge caching layer, typically a reverse proxy or CDN point of presence, sitting in front of origin infrastructure. TtlUse short TTLs for frequently changing content and longer TTLs for stable assets, tuning based on observed change frequency and tolerance for staleness. Hit rateHit rate is influenced by request distribution, cache key design, TTL, content variability, and how well the edge can normalize and coalesce requests. Most caching advice starts at the origin and works outward. Caching Strategies Caching Myths That Are Killing Your Hit Ratio (And What to Do) LayerShared HTTP response cache (reverse proxy / edge layer) sitting in front of an origin application, keyed by request URL plus selected request headers TtlLong TTLs for content-addressed immutable assets; short TTLs with stale-while-revalidate for semi-dynamic pages; near-zero or no-store for per-user and authenticated responses Hit rateDriven by key cardinality, cacheability of responses, eviction pressure relative to working-set size, request distribution skew, and how much of the traffic is genuinely repeatable Hit ratio is the fraction of requests served without touching the origin. Caching Strategies Why Your Cache Isn't Hitting 95% and What to Do About It LayerShared reverse-proxy or application-level response cache sitting in front of origin services, keyed on request attributes such as URL, method, headers, and cookies. TtlSet TTL from object change frequency and tolerance for staleness, not from a target hit ratio; hot stable objects can live long, volatile or personalized ones should be short or uncacheable. Hit rateDriven by key cardinality, request distribution skew, cacheable response share, TTL length relative to request frequency, eviction pressure from working-set size, and how many requests are bypassed or personalized. A cache hit ratio is a symptom, not a goal. Caching Strategies Caching Isn't a Set-and-Forget Toggle: Rethink Your Edge Strategy LayerEdge caching layer that stores HTTP responses close to users, typically for static assets and cacheable API responses. TtlShort TTLs (seconds to minutes) for dynamic content; longer TTLs (hours to days) for static assets; balance freshness and hit rate. Hit rateAffected by content popularity, cache key design, TTL, request diversity, and geographic distribution of users. Caching at the edge is often treated as a simple switch: enable it, set a time-to-live, and walk away. Caching Strategies Stop Guessing Your Cache TTL: A Case for Stale-While-Revalidate LayerShared HTTP response caches sitting between clients and origin servers, including reverse proxies, edge caches, and browser caches that honor standard freshness and revalidation directives. TtlShort freshness windows for volatile content, longer windows for stable assets, with a stale-while-revalidate window sized to how long the origin can tolerate serving slightly old data. Hit rateDominated by how well the cache key matches real request variation, how long entries stay fresh, and how much of the traffic is genuinely repeatable rather than personalized or randomized. Most teams pick a time-to-live by feel. Caching Strategies Should You Cache That? A Decision Framework for HTTP Responses LayerHTTP response caching at shared reverse-proxy or edge layers, plus private browser caches, distinguished by cacheability and keying rules TtlShort for volatile or personalized-adjacent content, long for immutable fingerprinted assets; align TTL with how often the underlying data actually changes rather than with a default value Hit rateDriven by request distribution, URL and key cardinality, query-string and header variance, Vary correctness, personalization, and how well TTLs match content change frequency Caching an HTTP response is a bet that the same request will recur and that the answer will still be correct when it does.

Case Studies

All Case Studies →
Case Studies Myth-Busting: The Case for Putting Your Cache Key Before Your Load Balancer LayerShared edge or reverse-proxy cache for HTTP responses, keyed on request attributes such as URL, headers, and cookies. TtlShort TTLs for frequently changing content, longer TTLs for static assets; TTL should be set per content class rather than globally. Hit rateHit rate depends on cache key granularity, request diversity, TTL length, and how many distinct keys the workload generates. A common piece of architecture advice says to put the load balancer first and the cache behind it. Case Studies CDN Caching Isn't a Magic Switch: What Real Case Studies Teach Us LayerEdge CDN cache for HTTP responses TtlStatic assets: long (days to years) with versioning; dynamic content: short (seconds to minutes) Hit rateAffected by content popularity, TTL, cache key design, origin headers, and request patterns Content delivery networks are often pitched as a simple switch: turn on caching, and your site becomes faster and cheaper. Case Studies Stop Blaming the CDN: A Case-Study Guide to Cache Invalidation and Load Balancing LayerReverse-proxy HTTP cache at the edge and origin, plus application-level key-value caching for computed fragments TtlUse the shortest TTL that keeps origin load acceptable; seconds to minutes for frequently changing data, hours to days for stable assets, with stale-while-revalidate to absorb bursts Hit rateAffected by request diversity, query-string and header normalization, cache key design, TTL length, and how much content is safely shareable across users A common failure pattern in distributed systems is to treat the content delivery network as the source of every freshness problem. Case Studies Why Your Cache Hit Ratio Is a Lie (and How to Fix It) LayerHTTP response caching at shared reverse proxies, CDN edge nodes, and application-level object caches such as Redis or Memcached TtlShort TTLs for volatile or personalized data, longer TTLs for static assets, with stale-while-revalidate to mask origin latency Hit rateKey design, cache key normalization, request distribution, object popularity, TTL, origin response headers, and cacheability of responses Cache hit ratio is one of the most cited numbers in performance engineering, and one of the least trustworthy. Case Studies Caching Case Study: Fix Your Hit Rate Before Adding Load Balancers LayerHTTP response caching at the reverse proxy or edge layer, keyed by request attributes such as URL, headers, and cookies. TtlStart with short TTLs for dynamic content and longer TTLs for static assets; tune based on observed change frequency and tolerance for staleness. Hit rateAffected by cache key design, request variety, TTL length, origin response cacheability, and how much content is personalized or frequently changing. A team notices rising latency and origin load, so they add load balancers and more origin servers. Case Studies Why Your Cache Hit Rate Is a Lie (and How to Fix It) LayerShared reverse-proxy or application-level cache sitting in front of an origin service, keyed by request attributes such as URL, headers, query string, and method. TtlSet TTL from observed change frequency and tolerable staleness, not a single global default; hot, slow-changing objects can live longer, while volatile or personalized responses need short or zero TTL. Hit rateDriven by key design, request diversity, cacheable response share, eviction pressure, working-set size relative to cache capacity, and how long entries survive before being replaced or invalidated. A cache hit rate is one of the most quoted numbers in performance work, and one of the least trustworthy. Case Studies Stop Treating Your Cache Like Storage: Hard Lessons from Real Deployments LayerHTTP response caching at the edge and shared object caching in memory, treated as a disposable copy of origin data rather than a durable store TtlSet from measured change frequency per object class, not one global value; hot immutable assets can live long, while user-specific or rapidly mutating data should be short or uncacheable Hit rateDominated by key cardinality, query-string and header variance, TTL length relative to object churn, and how much of the working set fits in the allocated memory or edge footprint Most teams adopt a caching layer for one reason: the origin cannot absorb the traffic it is being asked to serve. Case Studies Forget Cache Hit Ratios: What Actually Worked When I Fixed a Product Catalog API LayerApplication-level response caching for a product catalog API, layered in front of the primary datastore and keyed by normalized request parameters rather than raw URLs. TtlShort-to-moderate time-to-live, on the order of minutes for volatile catalog slices and longer for stable reference data, always paired with explicit event-driven eviction so the time-to-live is a backstop rather than the primary freshness mechanism. Hit rateHit rate is governed by key cardinality, how many distinct filter and sort combinations clients actually request, whether the cached payload is segmented into reusable fragments, and how much of the catalog changes between requests. A product catalog API looks like an easy caching target until you actually run one.

Load Balancing Algorithms

All Load Balancing Algorithms →
Load Balancing Algorithms Stop Round-Robining Everything: A Load Balancer Algorithm Field Guide LayerLoad balancer connection and request distribution state (per-backend connection pools, health status, and routing tables), not a content cache TtlHealth check interval and weight decay window; typically seconds for liveness and tens of seconds to minutes for smoothed latency or load estimates Hit rateShare of requests routed to a backend that can serve them without queueing or retry; affected by backend capacity variance, request cost distribution, connection reuse, and how quickly the algorithm learns changing latency Round-robin is the default most teams reach for, and for good reason. Load Balancing Algorithms Choosing the Right Load Balancing Algorithm: A Practitioner's Guide LayerLoad balancer connection and session state, often stored in a shared cache or local to the balancer TtlShort TTLs for health state and connection pools, longer TTLs for session affinity when sticky sessions are required Hit rateAlgorithm choice, health-check accuracy, session persistence, and traffic skew affect how often requests reuse warm backend connections or cached session data Load balancing algorithms decide which backend receives each request. Load Balancing Algorithms Stop Round-Robin: Hash Your Way to a Smarter Load Balancer LayerApplication-layer cache sharding across load-balanced backends, including in-memory key-value stores and HTTP reverse-proxy caches. TtlSet TTLs based on data volatility and tolerance for staleness; use shorter TTLs for rapidly changing data and longer for stable content, with jitter to avoid synchronized expiry. Hit rateKey distribution uniformity, request skew, backend capacity, and whether the hash function maps hot keys evenly across nodes. Round-robin load balancing is often the first algorithm engineers reach for because it is simple, stateless, and easy to reason about. Load Balancing Algorithms Why Round Robin Fails Your API and What to Use Instead LayerLoad-balancer-level request distribution and connection pooling, not a content cache; any caching sits behind the balancer and is unaffected by the algorithm choice. TtlHealth-check interval and failure threshold act as the effective TTL; typical guidance is a few seconds between probes with two to three consecutive failures before eviction. Hit rateNot a cache metric here; the analogous metric is the fraction of requests routed to a backend that can serve them without queueing, which depends on request cost variance and backend capacity spread. Round robin is the default load balancing algorithm in most tutorials, and for good reason. Load Balancing Algorithms Load Balancing Algorithms: A Field Guide for Your First Production Outage LayerLoad balancing algorithms operate at the connection and request layer, not a data cache; they distribute traffic across backend servers, which may themselves cache responses. TtlNo TTL applies to load balancing algorithms; health check intervals and timeouts are the relevant timers, typically set based on application tolerance for stale server state. Hit rateHit rate is not directly applicable to load balancing; however, consistent hashing can improve cache hit rates on backends by routing the same keys to the same servers. Load balancing algorithms decide which backend server receives each incoming request.

Performance Tuning

All Performance Tuning →
Performance Tuning Is Your Cache Making You Slower? Tune Your Cache-Control Headers First LayerHTTP response caching at browser, shared proxy, and CDN edge layers controlled by Cache-Control headers TtlUse short TTLs for volatile or personalized content, longer TTLs for static assets; prefer stale-while-revalidate and stale-if-error to balance freshness and availability Hit rateAffected by freshness lifetime, revalidation support, request method, URL variance, Vary header correctness, and cacheability of responses Cache-Control headers are the primary control plane for HTTP caching, yet they are often set once and forgotten. Performance Tuning Why Your Cache Hit Rate Is Lying to You (and How to Fix It) LayerShared read-through cache in front of an origin data store, typically a key-value or HTTP response cache serving repeated reads. TtlShort TTLs for volatile data, longer TTLs for stable reference data; align TTL with the tolerance for stale reads rather than a single global value. Hit rateAffected by key cardinality, key normalization, request distribution skew, eviction pressure, TTL length, and whether writes invalidate or refresh entries. Cache hit rate is the most quoted number in performance tuning and one of the least trustworthy. Performance Tuning How to Tune Cache-Control Headers Without Breaking Your Load Balancer LayerShared HTTP response caching at the edge and in reverse proxies, coordinated with load balancer routing and health checks TtlShort for personalized or volatile responses, longer for static assets; use max-age with s-maxage for shared layers and revalidate via ETag or Last-Modified Hit rateDriven by URL and header normalization, Vary correctness, cache key design, freshness lifetime, and how much traffic repeats across nodes Cache-Control is a response header, but it is also a routing decision. Performance Tuning Cache or Load Balancer: Which Knob to Turn When Things Slow Down? LayerHTTP response caching at the edge and in shared reverse-proxy tiers, plus application-level object caching such as Redis-style key-value stores TtlShort for volatile data (seconds to minutes), long for immutable or fingerprinted assets (hours to days); always pair long TTLs with a versioned key or purge path Hit rateRequest pattern repetitiveness, key cardinality, cache key design, TTL length, eviction pressure, and how much of the working set fits in the allocated memory or disk When latency rises, the instinct is to reach for one of two controls: add caching, or add load balancing. Performance Tuning How Long Should Your Cache Live? A Practical Answer LayerShared HTTP response caches and in-process or remote key-value caches (object caches) sitting in front of origin application logic TtlShort for volatile, per-user, or rapidly changing data (seconds to minutes); long for stable, expensive-to-recompute, or shared content (hours to days), tuned per object class rather than globally Hit rateTraffic skew, key cardinality, request arrival rate, object size distribution, and how closely the TTL matches the natural change frequency of each object The question of how long a cache should live sounds like it has a single numeric answer, and almost every team eventually asks for one. Performance Tuning Can You Cache Dynamic Content at the CDN Edge? Yes, But You Gotta Watch the Details LayerReverse-proxy edge cache storing full HTTP responses for dynamic endpoints, keyed on URL plus selected request attributes such as cookies, headers, and query strings TtlStart short, often seconds to a couple of minutes, and extend only where measured staleness tolerance and upstream change frequency allow; vary TTL per endpoint class rather than applying one global value Hit rateDriven by response variability: how many distinct cache keys a URL fans out into, how personalized the payload is, how often upstream data changes, and how aggressively you normalize requests before lookup Dynamic content is usually defined by what it is not: it is not a static asset with a stable body and a long shelf life.