Cache Jitter for Cache Stampede Prevention

Cache jitter randomizes cache expiration times so a large group of entries does not become invalid at once. This reduces synchronized cache misses and the resulting burst of backend requests.

A cache stampede occurs when many cached values expire around the same time and requests simultaneously fall through to the underlying database or service. If a popular key is missing, many concurrent requests may independently try to rebuild it, producing a sudden load spike.

Cache jitter addresses synchronized expiration by adding controlled randomness to each entry's TTL. Instead of assigning every entry exactly `300` seconds, an application might choose a TTL somewhere between `270` and `330` seconds. Entries created at approximately the same time therefore tend to expire at different moments.

In Java, the randomization can be applied when constructing the cache entry or calculating its expiration duration. The exact mechanism depends on the cache technology, but the important property is that expiration is distributed over time rather than concentrated at one instant.

Jitter reduces synchronization, but it does not by itself guarantee that only one request rebuilds a missing value. For hot keys, combine it with techniques such as request coalescing, distributed locking, stale-while-revalidate, or background refresh when stronger stampede protection is required.

Key points

• Randomize TTLs to spread cache expirations over time.
• Use jitter when many entries are created or refreshed together.
• Jitter reduces synchronized misses but does not prevent duplicate recomputation by itself.
• Combine jitter with request coalescing or locking for high-contention hot keys.
• Choose jitter bounds carefully so cache freshness remains within acceptable limits.

Interview questions

How do you choose an appropriate jitter range?

The range should be large enough to decorrelate expiration events but small enough to preserve the application's freshness requirements. I would start from the acceptable TTL variation and consider how many entries are refreshed together and how much backend load the system can absorb. A fixed percentage or bounded absolute offset can work, but the choice should be validated using traffic and expiration metrics. I would also check whether scheduled refresh jobs or batch writes introduce new synchronization patterns.

What is cache jitter, and why does it help prevent a cache stampede?

Cache jitter adds a random offset to an otherwise fixed cache TTL. This means entries created at roughly the same time do not all expire simultaneously. As a result, cache misses and backend recomputation are spread over a wider period. It reduces the probability of a sudden synchronized request surge.

How would you implement TTL jitter in a Java backend?

I would calculate a base TTL and add a bounded random offset before storing the entry. For example, a 300-second TTL could receive a random offset between -30 and +30 seconds. In Java, `ThreadLocalRandom` is convenient for generating the offset without sharing mutable random state between request threads. The cache library then receives the resulting TTL when the entry is written.

Is cache jitter sufficient to solve a cache stampede for a very hot key?

No. Jitter reduces the chance that many keys expire together, but it does not coordinate requests that encounter the same missing hot key. Several threads can still observe a miss and independently regenerate the value. For that case, I would consider request coalescing, a distributed lock, stale-while-revalidate, or another single-flight mechanism. The appropriate choice depends on latency, consistency, and infrastructure requirements.

What other techniques would you combine with cache jitter in a high-traffic Java service?

For hot keys, I would combine jitter with request coalescing or a lock so concurrent misses do not trigger duplicate backend work. Serving stale data during refresh can further protect latency when slightly old values are acceptable. Background refresh is another option when the application can proactively update popular entries. I would select the combination based on consistency requirements, failure behavior, and the cost of recomputation.