Four Ways Your Cache Silently Turns Into a Database-Killer
Thundering herd, cache penetration, hot-key breakdown, and full cache crashes are four distinct failure modes, each with a different fix.
On this page
A cache’s entire job is to stand between your application and a database that can’t handle full traffic directly. That means every cache failure mode has the same shape: the cache, for one specific reason, stops absorbing traffic, and the database receives a spike it was never sized for. Four distinct mechanisms produce that shape, and — this is the part worth sitting with — none of them share a fix. Diagnosing which one you’re looking at is most of the incident response.
Failure 1: thundering herd — everything expires at once
If a batch of keys is all written with the same TTL — a common pattern when data is loaded or refreshed in bulk — they all expire at the same moment too. The next request for any of them is a cache miss, and if that’s thousands of keys expiring within the same second, thousands of requests hit the database simultaneously, for data that was cached and fine a moment earlier.
t=0: keys A, B, C, ... Z all cached, TTL=300s
t=300: A, B, C, ... Z all expire in the same instant
t=300+ε: every request for A..Z becomes a cache miss → DB hit
→ thousands of simultaneous queries hit a DB sized for cache-assisted load
The fix is distributing expiry, not eliminating it: add a small random jitter to each key’s TTL (300s ± random(0, 30s)) so expirations spread out over a window instead of landing on one instant. A complementary approach: let only core, business-critical keys fall through to the database on a miss, and serve a slightly stale or degraded response for non-critical keys until the cache catches back up.
Failure 2: cache penetration — querying data that doesn’t exist anywhere
A normal cache miss still ends with the cache populated — fetch from the database, write to the cache, done, and the next request for that key is fast. Cache penetration breaks that assumption: the key doesn’t exist in the database either, so there’s nothing to cache. Every single request for that key repeats the full miss-then-database-query cycle, indefinitely.
request(id=99999999) → cache: MISS
→ database: NOT FOUND
→ nothing gets cached
→ next identical request repeats the entire cycle
This isn’t just inefficient — it’s a real attack surface. A malicious actor querying a stream of nonexistent IDs can force every one of those queries straight through to the database, bypassing the cache entirely by construction.
Failure 3: cache breakdown — one hot key, not many
Structurally the same mechanism as thundering herd — a cache miss sends traffic to the database — but concentrated on a single key that alone accounts for a disproportionate share of total read traffic. When that one key expires, its entire traffic share redirects to the database at once, which can be just as damaging as thousands of ordinary keys expiring together.
The fix follows directly from the diagnosis: don’t let hot keys expire on a fixed TTL at all. Identify keys carrying an outsized share of read traffic and either exempt them from expiration, or refresh them proactively in the background before they would expire — so the traffic never sees a miss in the first place.
It’s tempting to apply a uniform TTL policy across every key for simplicity. That’s backwards from a risk standpoint — a cold key expiring affects almost no traffic; a hot key expiring redirects a large share of your total load onto the database in one instant. Protection should scale with traffic share, not be uniform.
Failure 4: the cache itself goes down
The first three failures are about specific keys misbehaving. This one is structural: the cache layer becomes entirely unreachable, and by default, every single request that would have hit it now falls through to the database — 100% of traffic landing on a system that was explicitly sized assuming the cache would absorb most of it.
- Detect the outage fastA circuit breaker pattern watches for repeated cache failures and trips open once a threshold is crossed, rather than letting every request individually discover the cache is down and fall through one at a time.
- Fail fast, don't fail throughOnce the breaker is open, stop routing traffic to the database as a fallback — serve a degraded response, queue the request, or reject it outright. Letting every request cascade to the database is exactly the failure a circuit breaker exists to prevent.
- Build for cache-layer redundancy, not just application redundancyA clustered cache deployment, rather than a single instance, means losing one node degrades capacity instead of taking the entire caching layer offline at once.
The diagnostic table
| Symptom | Failure mode | Root fix |
|---|---|---|
| Many unrelated keys expire together, DB spike | Thundering herd | Jitter TTLs; protect core-only data during recovery |
| Repeated queries for IDs that don’t exist | Cache penetration | Cache null results; bloom filter in front |
| One specific key’s expiry causes a DB spike | Cache breakdown | No TTL, or proactive refresh, on hot keys |
| Cache is entirely unreachable, DB gets 100% of traffic | Cache crash | Circuit breaker; clustered cache deployment |
Takeaway
“Add caching” solves a performance problem. It also creates four new, specific failure modes that a database-only architecture never had — and the mistake that turns a caching layer into an outage generator is treating all four as one generic problem with one generic fix. Thundering herd needs jitter. Penetration needs a bloom filter or null-caching. Breakdown needs your hottest keys treated as the ones that expire least. And a cache crash needs a circuit breaker that admits the cache is down instead of quietly forwarding its entire load to a database that was never meant to carry it alone.
Frequently asked questions
What's the actual difference between thundering herd and cache breakdown?
Thundering herd happens when many different keys expire at roughly the same time, sending a broad wave of requests to the database all at once. Cache breakdown is the same underlying mechanism — a cache miss triggering a database hit — but concentrated on a single hot key that alone accounts for a large share of traffic. Thundering herd needs distributed expiry times; breakdown on one hot key needs that specific key to simply never expire.
Why is cache penetration more dangerous than a normal cache miss?
A normal cache miss still populates the cache after fetching from the database, so the next request for that key is fast. Cache penetration happens when the requested key doesn't exist in the database either — there's nothing to cache, so every single request for that key hits the database again, indefinitely. A malicious actor querying nonexistent IDs in a loop can turn this into a real denial-of-service vector.
How does a bloom filter prevent cache penetration?
A bloom filter is a compact, probabilistic structure that can say 'this key definitely does not exist' with certainty, or 'this key probably exists' (with a small false-positive rate). Checking the bloom filter before touching the database lets you reject requests for genuinely nonexistent keys immediately, without ever reaching the database — closing off the attack surface that repeated queries for nonexistent data would otherwise open.
Why not just add an expiration time to your hottest keys to be safe?
A key responsible for a large share of total read traffic is exactly the key you want to never expire, or to refresh proactively in the background before it would expire — because the moment it does expire, every one of those reads redirects to the database simultaneously. The keys carrying the most traffic are the ones where an expiry-driven cache miss does the most damage, which is the opposite of where a uniform TTL policy should apply.
What's the actual purpose of a circuit breaker when the cache itself goes down?
Without a circuit breaker, a cache outage sends 100% of traffic straight to the database — a system that was never sized to handle that load, since the cache was specifically there to absorb most of it. A circuit breaker detects the cache is unreachable and fails fast (serving a degraded response, queuing, or rejecting) instead of letting every request pile onto a database that will fall over under the sudden, unplanned full load.

/* Comments */
Comments are offline right now — we reconnect automatically, nothing is lost.