1 Overview
Content delivery networks are often pitched as a simple switch: turn on caching, and your site becomes faster and cheaper. In practice, real case studies show that CDN caching is a nuanced engineering discipline. The cache is not a single entity but a hierarchy of edge locations, each with its own storage and eviction policies. What works for one workload may fail for another because caching effectiveness depends on request patterns, content volatility, and how you configure cache keys and time-to-live values.
A common misconception is that enabling CDN caching automatically yields high hit rates. In reality, hit rate is a function of many variables: how many unique URLs are requested, how often they repeat, how long they stay fresh, and whether the cache key includes headers that fragment the cache. For example, if your cache key includes a user-specific cookie, every user gets a unique cache entry, and the hit rate collapses. Similarly, if your TTL is too short, objects expire before they can be reused. These are not vendor-specific issues; they are fundamental to how caching works.
This article examines what real-world case studies teach us about CDN caching. We will explore where common advice breaks down, how the mechanism actually works, the trade-offs you are really making, how to implement it correctly, and how to measure whether it helped. The goal is to provide evergreen engineering education, not to promote any specific product or promise performance numbers. By understanding the underlying principles, you can make informed decisions for your own workloads.
2 Where the common advice breaks down
The most frequent advice is to set a long TTL for static assets and a short TTL for dynamic content. While this is a reasonable starting point, it breaks down when content is not easily classified. Consider an API that returns product inventory. The response changes frequently, so a short TTL seems appropriate. But if the same product is requested thousands of times per minute, a short TTL means the origin is hit constantly, defeating the purpose of the CDN. Conversely, a long TTL risks serving stale inventory, which can lead to overselling. The advice ignores the trade-off between freshness and origin load.
Another common recommendation is to purge the cache when content changes. Purging seems straightforward: update the origin, then invalidate the CDN. However, purging is not instantaneous across all edge locations. It can take time to propagate, and during that window, some users may see stale content while others see fresh content. Moreover, frequent purges can reduce hit rate because objects are evicted before they can be reused. In high-write environments, a purge-heavy strategy can make the CDN almost useless, as the cache is constantly emptied.
A third piece of advice is to use cache keys that include all relevant request variations. This is intended to avoid serving the wrong content to the wrong user. But over-including headers or cookies fragments the cache, leading to many low-traffic entries that are evicted before they are reused. For example, including a language header is necessary if you serve localized content, but including a user-agent string can create thousands of variants for the same resource. The key must balance correctness with cache efficiency.
3 How the mechanism actually works
At its core, a CDN cache is a distributed key-value store. When a request arrives at an edge location, the cache computes a cache key from the request, typically based on the URL and selected headers. It then looks up that key. If the object is present and fresh, it is served directly. If not, the edge forwards the request to the origin or to a parent cache, stores the response if it is cacheable, and then serves it. The cacheability is determined by HTTP headers like Cache-Control and Expires, as well as the CDN's configuration. The TTL is the duration for which the object is considered fresh. After that, the edge may revalidate with the origin using conditional requests, or it may serve stale content while fetching a fresh copy in the background, depending on the stale-while-revalidate directive.
Eviction is a critical part of the mechanism. Each edge location has limited storage, so it must decide which objects to keep and which to evict. Most caches use a least-recently-used or similar algorithm, but the exact policy varies. When an object is evicted, the next request for it will be a miss, requiring a fetch from the origin. This means that even with a long TTL, an object may not stay cached if it is not requested frequently enough. The effective hit rate is therefore a function of both TTL and request frequency. Objects that are popular but infrequently requested may still be evicted.
Invalidation is another mechanism. When content changes, you can either wait for the TTL to expire or actively purge the object. Purging sends a signal to all edge locations to remove the object. However, purging can be slow and may not be atomic. Some CDNs support soft purge, which marks the object as stale but still serves it while revalidating. This can reduce origin load during purges. Understanding these mechanisms helps you design a caching strategy that aligns with your content's lifecycle.
4 Trade-offs you are really making
Every caching decision involves trade-offs. The most obvious is between hit rate and freshness. A longer TTL increases the chance that an object is still in cache when requested, raising the hit rate and reducing origin load. But it also increases the risk of serving stale content. For content that changes infrequently, such as images or stylesheets, this trade-off is easy: use a long TTL and version the URL when content changes. For dynamic content, such as a stock ticker or a user's shopping cart, freshness is paramount, so you might use a very short TTL or no caching at all. The key is to classify content by its volatility and business impact.
Another trade-off is between cache efficiency and correctness. Including more headers in the cache key ensures that different users get the correct content, but it also fragments the cache. For example, if you include the Accept-Encoding header, you might have separate entries for gzip and brotli, which is necessary for correct content encoding. But if you include a header that varies per user, such as a session cookie, you effectively disable caching. The goal is to include only those headers that truly affect the response body. This often requires coordination between application and CDN configuration.
A third trade-off is between invalidation speed and origin load. Purging the cache immediately after a content update ensures users see fresh content quickly, but it also means the next requests will be misses, increasing origin load. Waiting for TTL expiry avoids purges but delays freshness. Some systems use stale-while-revalidate to serve stale content while fetching a fresh copy, which balances both: users get a fast response, and the origin is hit only once per revalidation. However, this can serve stale content for a short period, which may be unacceptable for some applications. The right choice depends on your tolerance for staleness and your origin's capacity.
5 Implementing it correctly
Implementing CDN caching correctly starts with a clear content classification. Categorize your content into immutable, semi-static, and dynamic. Immutable content, like versioned JavaScript bundles, can have a very long TTL, often a year or more, because the URL changes when the content changes. Semi-static content, like product images that change occasionally, can have a moderate TTL, such as a day, combined with a purge when updated. Dynamic content, like API responses, may use a short TTL, stale-while-revalidate, or no caching at all. This classification should be based on how often the content changes and how costly it is to serve from the origin.
Next, design your cache keys carefully. The cache key should include only the request attributes that affect the response. Typically, this is the URL and the Accept-Encoding header. If your application serves different content based on language, include the Accept-Language header. Avoid including cookies unless absolutely necessary, as they are unique per user. If you must vary by user, consider using a token in the URL or a custom header that represents a user segment rather than an individual. Also, normalize the URL: strip query parameters that do not affect the response, such as tracking parameters. This reduces cache fragmentation.
Finally, configure invalidation and revalidation. Use versioned URLs for immutable assets so you never need to purge. For semi-static content, set up a purge mechanism that triggers when content changes, but be aware of propagation delays. For dynamic content, use stale-while-revalidate to serve stale content while fetching a fresh copy, which reduces origin load and improves latency. Also, set appropriate Cache-Control headers: public for shared caches, private for user-specific content, and no-store for sensitive data. Test your configuration with real traffic patterns to ensure it behaves as expected.
6 Measuring whether it helped
To know if your CDN caching strategy is effective, you must measure the right metrics. The most important is hit rate, which is the percentage of requests served from cache. However, hit rate alone can be misleading. A high hit rate on low-value content may not translate to cost savings or performance gains. You should also measure origin offload, which is the reduction in requests to your origin. This directly impacts cost and origin load. Additionally, measure latency for cache hits versus misses. If cache hits are not significantly faster, the CDN may not be providing value. Finally, track error rates and stale content incidents to ensure correctness.
Segment your metrics by content type, edge location, and time. A global hit rate might hide poor performance in specific regions or for specific content. For example, a video streaming service might have a high hit rate for popular videos but a low hit rate for long-tail content. By segmenting, you can identify opportunities to improve caching for specific segments. Also, monitor cache eviction rates. If objects are evicted before they are reused, your TTL or cache size may be inadequate. Some CDNs provide insights into cache efficiency, such as the ratio of bytes served from cache to total bytes.
Conduct controlled experiments when possible. For instance, you can A/B test different TTLs or cache key configurations and measure the impact on hit rate, origin load, and user-perceived latency. Be cautious about drawing conclusions from small samples; caching effects can be subtle and vary with traffic patterns. Also, consider the cost of the CDN itself. A higher hit rate may reduce origin costs but increase CDN costs if the CDN charges for requests. The net cost impact depends on your pricing model. Ultimately, the goal is to optimize for your specific objectives, whether that is reducing cost, improving latency, or both.
7 Frequently asked questions
What is a good hit rate? There is no universal answer. A hit rate of 90% might be excellent for an API but poor for a site serving mostly unique user-generated content. The right target depends on your content and goals. Focus on trends and origin offload rather than a specific number. If increasing TTL or adjusting cache keys improves hit rate without harming freshness, it is likely beneficial. But if you are already serving most requests from cache, further optimization may yield diminishing returns.
How often should I purge the cache? Purge only when necessary. Frequent purges reduce hit rate and increase origin load. For immutable content, use versioned URLs and never purge. For semi-static content, purge when content changes, but consider using stale-while-revalidate to avoid a thundering herd. For dynamic content, rely on short TTLs or revalidation rather than purging. If you must purge frequently, evaluate whether your content classification is correct; perhaps some content should not be cached at all.
Can I cache personalized content? Yes, but with care. If the personalization is based on a small number of segments, you can include a segment identifier in the cache key. For example, cache separate versions for each language or currency. If personalization is per-user, caching at the edge is generally not feasible because it would create a unique entry per user, leading to a near-zero hit rate. In that case, consider caching at a layer closer to the user, such as in the browser, or using edge compute to assemble personalized responses from cached fragments. Always respect privacy and security by not caching sensitive data in shared caches.