Cache Stampede in Distributed Caching

A cache stampede occurs when many requests simultaneously regenerate an expired or missing cache entry. In a distributed Java backend, the resulting burst can overload the database or downstream service and amplify latency and failures.

A stampede commonly begins when a popular cache entry expires while traffic remains high. Requests that would normally be served from the cache instead discover a miss and attempt to rebuild the same value concurrently.

The main danger is duplicate work: hundreds or thousands of requests can independently query a database or call another service for identical data. The backend may become overloaded precisely when the cache is least able to protect it, creating a feedback loop of higher latency, timeouts, and more misses.

A common mitigation is request coalescing, where only one request performs the regeneration while others wait for its result. In a distributed deployment, this can be implemented with a distributed lock, provided the lock has an appropriate expiration and failure-handling strategy.

Other defenses include randomized TTLs to avoid synchronized expirations, serving stale data while refreshing asynchronously, and using early refresh before an entry actually expires. The appropriate combination depends on freshness requirements, failure tolerance, traffic patterns, and the cost of regenerating the value.

In Java, synchronization confined to one JVM does not coordinate requests handled by other application instances. For a multi-instance service, the coordination mechanism must therefore be shared across instances, such as a Redis-based lock or another distributed coordination facility.

Key points

• A stampede turns one cache miss into many concurrent regenerations.
• The primary risk is overload of the backing database or service.
• Request coalescing ensures one regeneration supplies many waiting requests.
• Randomized expiration reduces synchronized cache misses.
• Stale-while-revalidate can preserve availability while refreshing asynchronously.
• Single-process locks are insufficient for coordination across multiple service instances.

Interview questions

What is a cache stampede, and why is it dangerous?

A cache stampede happens when many requests encounter an expired or missing popular cache entry at about the same time. They may all perform the expensive operation needed to rebuild that entry. This can overload a database or downstream service and increase latency. The extra load can cause failures that produce even more cache misses.

How would you prevent a cache stampede in a multi-instance Java service?

I would first identify whether the value can tolerate stale reads and how expensive regeneration is. For strict freshness, I could use request coalescing so one request regenerates the value while others wait, using shared coordination such as a distributed lock. The lock needs an expiration and careful handling of failures so one crashed worker does not block regeneration indefinitely. If stale data is acceptable, stale-while-revalidate can often reduce request contention and protect the backing service.

Why can randomized TTLs help with cache stampedes?

If many related entries are inserted at the same time with identical TTLs, they can expire together and create synchronized misses. Adding a random component spreads their expiration times over a window. That reduces the probability of a large synchronized regeneration burst. It does not prevent a stampede for a single extremely popular key, so it is usually complementary to request coalescing or refresh strategies.

How would you handle a cache entry that is expensive to rebuild but can be slightly stale?

I would consider stale-while-revalidate: continue serving the existing value while a background operation refreshes it. The refresh should be coalesced so only one worker performs the expensive regeneration for a given key. I would also define bounds for staleness and behavior when refresh repeatedly fails. Metrics should track refresh duration, stale-serving frequency, failures, and backing-service load.

What distributed-lock problems would you consider when using Redis to prevent a stampede?

The lock should have bounded ownership so a failed process cannot hold it forever. The implementation must also avoid allowing an expired lock to be released by a different owner, typically by associating the lock with an owner token. A lock alone does not guarantee that the cached value is correct, so regeneration and cache publication still need careful failure handling. For high-value systems, I would evaluate lock contention, timeout behavior, and whether stale serving or another coordination design is safer.