Stale-While-Revalidate for Cache Stampede Prevention

Stale-while-revalidate (SWR) returns an existing cached value immediately, even after its freshness period, while triggering a background refresh. This keeps latency low and prevents many concurrent requests from simultaneously rebuilding the same expired value.

An SWR cache typically tracks both the cached value and its freshness state. While the entry is fresh, requests return it normally; after it becomes stale but remains within a permitted stale window, requests can still receive the old value while a refresh runs asynchronously.

The important concurrency property is that the refresh should be coordinated so that one logical refresh is normally in flight for a key at a time. Without coordination, many requests that observe the same stale entry can each start an expensive backend operation, recreating the cache-stampede problem.

SWR is useful when slightly old data is acceptable for a bounded period. The refresh path should update the cache atomically after obtaining the new value, and failures should be handled separately from serving the existing stale value when the application's consistency requirements permit that behavior.

In a distributed Java service, a local `ConcurrentHashMap` can coordinate refreshes only within one JVM. If multiple application instances can refresh the same key concurrently and that matters operationally, coordination may require a distributed mechanism such as a lease, distributed lock, or cache-system feature.

SWR is a latency and load-management technique, not a guarantee that data is always current. The design must explicitly define freshness limits, stale-data tolerance, refresh failure behavior, and what happens when no cached value exists.

Key points

• Serve a stale-but-usable value immediately during the stale window.
• Run the refresh asynchronously rather than making the current request wait.
• Coordinate refreshes per key to avoid duplicate backend work.
• Use a bounded stale window so stale data does not persist indefinitely.
• For multiple JVMs, local synchronization alone does not coordinate all instances.

Interview questions

What problem does stale-while-revalidate solve?

It reduces latency and backend load when a cached value has become stale but is still acceptable to serve. Instead of making the request wait for a fresh backend response, the service returns the existing value and starts a refresh. This is particularly useful when many requests arrive around the same expiration time.

How would you prevent multiple threads from refreshing the same stale key?

I would maintain per-key refresh state and allow only one caller to acquire the refresh role. In Java, an atomic operation such as `compareAndSet` can be used for a simple local implementation, or a keyed future can represent the shared refresh operation. Other callers continue receiving the stale value rather than launching additional backend calls. In a multi-instance deployment, I would consider distributed coordination if duplicate refreshes are costly.

What happens if the background refresh fails?

The answer depends on the application's tolerance for stale data. If stale serving is allowed, the old value can remain available for a bounded stale period while the system retries later. The implementation should record refresh failures and avoid an uncontrolled retry loop. Once the maximum stale period is exceeded, the service may need to block, return an error, or use another fallback.

How is stale-while-revalidate different from request coalescing?

Request coalescing coordinates concurrent cache loads so that multiple callers share one in-flight operation. SWR changes the read behavior as well: callers can receive an existing stale value while the refresh proceeds. They are complementary techniques, so a robust cache can use SWR to avoid waiting and single-flight coordination to avoid duplicate refresh work. The exact combination depends on consistency requirements and deployment topology.

Design an SWR cache for a high-traffic Java service running multiple instances.

I would define separate freshness and maximum-staleness windows, then return fresh data normally and stale-but-valid data while scheduling one refresh. Within each instance, I would deduplicate refreshes by key using an atomic state or shared future. If duplicate cross-instance refreshes create significant backend load, I would add distributed coordination or use cache infrastructure that supports it. I would also instrument refresh latency, failures, stale-hit rate, backend load, and the age of served values.