Cache Locking for Cache Stampede Prevention

Cache locking coordinates concurrent cache misses so that only one request computes and stores a missing value for a given key at a time. Other requests can wait briefly, retry the cache, or use a fallback instead of performing the same expensive work concurrently.

A cache stampede occurs when a popular cached entry expires or disappears and many requests discover the miss simultaneously. Without coordination, each request may query the database or call a downstream service, creating a burst of duplicate work precisely when the cache is least able to absorb it.

Cache locking associates a lock with the cache key being populated. The first request that successfully acquires the lock becomes the loader; it checks the cache again, computes the value if it is still absent, writes the value, and releases the lock. Requests that fail to acquire the lock can wait briefly and then retry the cache, or return a fallback when waiting is undesirable.

In a single JVM, a keyed lock can be implemented with Java concurrency primitives. In a distributed application with multiple application instances, the lock must be coordinated through a shared system such as Redis or a database; a local `synchronized` block cannot prevent two different instances from loading the same key.

A distributed lock needs failure handling. Use a bounded lease or expiration so a crashed loader does not hold the lock forever, and make lock release conditional on ownership so one request cannot accidentally release another request's lock. The lock timeout, wait duration, and fallback strategy should be chosen according to the latency and consistency requirements of the application.

Cache locking reduces duplicate population work, but it does not by itself guarantee perfect protection. A slow loader can outlive its lease, and lock contention can increase request latency. Techniques such as request coalescing, stale-while-revalidate, randomized TTLs, refresh-ahead, and negative caching can complement locking.

Key points

• Lock per cache key rather than globally when independent keys can be loaded concurrently.
• The lock holder should recheck the cache after acquiring the lock.
• A local JVM lock does not coordinate requests across application instances.
• Distributed locks need bounded leases and ownership-safe release.
• Waiting requests can retry, serve stale data, or use a fallback instead of duplicating the expensive load.

Interview questions

Why doesn't a Java `ReentrantLock` solve cache stampedes in a distributed service?

A `ReentrantLock` coordinates threads that share the same JVM and lock object. If the service has several instances, each instance has its own lock state, so two instances can acquire their local locks simultaneously. A distributed lock requires shared coordination, such as a Redis-based mechanism or another suitable distributed coordination system. The implementation also needs to account for expiration, ownership, and failure.

What problem does cache locking solve?

It addresses the cache stampede that can happen when many requests miss the same key at once. Instead of every request performing the expensive load, one request obtains the lock and populates the key. Other requests wait briefly, retry the cache, or use a fallback. This reduces duplicate database or downstream-service work.

Why is the second cache check necessary after acquiring a per-key lock?

Several requests can observe the same initial miss before any one of them acquires the lock. The first request then loads and stores the value while the others remain blocked. When a waiting request obtains the lock, the value may already exist, so it should read the cache again rather than execute the expensive loader. This pattern is often called double-checked locking in this cache-loading context.

What failure modes must you consider when using a distributed cache lock?

A crashed loader can leave a lock held indefinitely unless the lock has a bounded lease or expiration. A loader can also run longer than its lease, allowing another process to acquire the lock while the original loader is still working. Release must therefore be ownership-aware so an old owner cannot delete a lock belonging to a newer owner. Production designs should also define behavior for lock acquisition timeouts, cache failures, and downstream failures.

When would you prefer stale-while-revalidate over making every request wait for a cache lock?

Stale-while-revalidate is useful when serving slightly outdated data is acceptable and request latency is more important than immediate freshness. A request can receive the existing stale value while one background operation refreshes the cache. Cache locking is more appropriate when callers require fresh data and regeneration must be serialized. The choice depends on freshness requirements, load cost, and acceptable latency.