---
topic: system-design
author: Crashtech Editorial
published: Nov 17, 2026 · read: 5 min
---

Four Ways Your Cache Silently Turns Into a Database-Killer

Thundering herd, cache penetration, hot-key breakdown, and full cache crashes are four distinct failure modes, each with a different fix.

– –

A cache’s entire job is to stand between your application and a database that can’t handle full traffic directly. That means every cache failure mode has the same shape: the cache, for one specific reason, stops absorbing traffic, and the database receives a spike it was never sized for. Four distinct mechanisms produce that shape, and — this is the part worth sitting with — none of them share a fix. Diagnosing which one you’re looking at is most of the incident response.

Failure 1: thundering herd — everything expires at once

If a batch of keys is all written with the same TTL — a common pattern when data is loaded or refreshed in bulk — they all expire at the same moment too. The next request for any of them is a cache miss, and if that’s thousands of keys expiring within the same second, thousands of requests hit the database simultaneously, for data that was cached and fine a moment earlier.

t=0:      keys A, B, C, ... Z all cached, TTL=300s
t=300:    A, B, C, ... Z all expire in the same instant
t=300+ε:  every request for A..Z becomes a cache miss → DB hit
          → thousands of simultaneous queries hit a DB sized for cache-assisted load

The fix is distributing expiry, not eliminating it: add a small random jitter to each key’s TTL (300s ± random(0, 30s)) so expirations spread out over a window instead of landing on one instant. A complementary approach: let only core, business-critical keys fall through to the database on a miss, and serve a slightly stale or degraded response for non-critical keys until the cache catches back up.

Failure 2: cache penetration — querying data that doesn’t exist anywhere

A normal cache miss still ends with the cache populated — fetch from the database, write to the cache, done, and the next request for that key is fast. Cache penetration breaks that assumption: the key doesn’t exist in the database either, so there’s nothing to cache. Every single request for that key repeats the full miss-then-database-query cycle, indefinitely.

request(id=99999999)  → cache: MISS
                       → database: NOT FOUND
                       → nothing gets cached
                       → next identical request repeats the entire cycle

This isn’t just inefficient — it’s a real attack surface. A malicious actor querying a stream of nonexistent IDs can force every one of those queries straight through to the database, bypassing the cache entirely by construction.

Fix 1: cache the null result simple
When a database lookup comes back empty, cache that “not found” result too (with a short TTL). The next request for the same nonexistent key hits the cache and returns immediately, instead of re-querying the database.
Fix 2: bloom filter in front stronger
A bloom filter can say “this key definitely doesn’t exist” with certainty before ever touching the database. Reject provably nonexistent keys at the filter, closing off the entire class of repeated-nonexistent-key traffic.
Advertisement

Failure 3: cache breakdown — one hot key, not many

Structurally the same mechanism as thundering herd — a cache miss sends traffic to the database — but concentrated on a single key that alone accounts for a disproportionate share of total read traffic. When that one key expires, its entire traffic share redirects to the database at once, which can be just as damaging as thousands of ordinary keys expiring together.

The fix follows directly from the diagnosis: don’t let hot keys expire on a fixed TTL at all. Identify keys carrying an outsized share of read traffic and either exempt them from expiration, or refresh them proactively in the background before they would expire — so the traffic never sees a miss in the first place.

The keys most protected should be the hottest ones, not the coldest

It’s tempting to apply a uniform TTL policy across every key for simplicity. That’s backwards from a risk standpoint — a cold key expiring affects almost no traffic; a hot key expiring redirects a large share of your total load onto the database in one instant. Protection should scale with traffic share, not be uniform.

Failure 4: the cache itself goes down

The first three failures are about specific keys misbehaving. This one is structural: the cache layer becomes entirely unreachable, and by default, every single request that would have hit it now falls through to the database — 100% of traffic landing on a system that was explicitly sized assuming the cache would absorb most of it.

  1. Detect the outage fast
    A circuit breaker pattern watches for repeated cache failures and trips open once a threshold is crossed, rather than letting every request individually discover the cache is down and fall through one at a time.
  2. Fail fast, don't fail through
    Once the breaker is open, stop routing traffic to the database as a fallback — serve a degraded response, queue the request, or reject it outright. Letting every request cascade to the database is exactly the failure a circuit breaker exists to prevent.
  3. Build for cache-layer redundancy, not just application redundancy
    A clustered cache deployment, rather than a single instance, means losing one node degrades capacity instead of taking the entire caching layer offline at once.

The diagnostic table

SymptomFailure modeRoot fix
Many unrelated keys expire together, DB spikeThundering herdJitter TTLs; protect core-only data during recovery
Repeated queries for IDs that don’t existCache penetrationCache null results; bloom filter in front
One specific key’s expiry causes a DB spikeCache breakdownNo TTL, or proactive refresh, on hot keys
Cache is entirely unreachable, DB gets 100% of trafficCache crashCircuit breaker; clustered cache deployment

Takeaway

“Add caching” solves a performance problem. It also creates four new, specific failure modes that a database-only architecture never had — and the mistake that turns a caching layer into an outage generator is treating all four as one generic problem with one generic fix. Thundering herd needs jitter. Penetration needs a bloom filter or null-caching. Breakdown needs your hottest keys treated as the ones that expire least. And a cache crash needs a circuit breaker that admits the cache is down instead of quietly forwarding its entire load to a database that was never meant to carry it alone.

Advertisement

Frequently asked questions

What's the actual difference between thundering herd and cache breakdown?

Thundering herd happens when many different keys expire at roughly the same time, sending a broad wave of requests to the database all at once. Cache breakdown is the same underlying mechanism — a cache miss triggering a database hit — but concentrated on a single hot key that alone accounts for a large share of traffic. Thundering herd needs distributed expiry times; breakdown on one hot key needs that specific key to simply never expire.

Why is cache penetration more dangerous than a normal cache miss?

A normal cache miss still populates the cache after fetching from the database, so the next request for that key is fast. Cache penetration happens when the requested key doesn't exist in the database either — there's nothing to cache, so every single request for that key hits the database again, indefinitely. A malicious actor querying nonexistent IDs in a loop can turn this into a real denial-of-service vector.

How does a bloom filter prevent cache penetration?

A bloom filter is a compact, probabilistic structure that can say 'this key definitely does not exist' with certainty, or 'this key probably exists' (with a small false-positive rate). Checking the bloom filter before touching the database lets you reject requests for genuinely nonexistent keys immediately, without ever reaching the database — closing off the attack surface that repeated queries for nonexistent data would otherwise open.

Why not just add an expiration time to your hottest keys to be safe?

A key responsible for a large share of total read traffic is exactly the key you want to never expire, or to refresh proactively in the background before it would expire — because the moment it does expire, every one of those reads redirects to the database simultaneously. The keys carrying the most traffic are the ones where an expiry-driven cache miss does the most damage, which is the opposite of where a uniform TTL policy should apply.

What's the actual purpose of a circuit breaker when the cache itself goes down?

Without a circuit breaker, a cache outage sends 100% of traffic straight to the database — a system that was never sized to handle that load, since the cache was specifically there to absorb most of it. A circuit breaker detects the cache is unreachable and fails fast (serving a degraded response, queuing, or rejecting) instead of letting every request pile onto a database that will fall over under the sudden, unplanned full load.

/* Comments */