How Uber’s CacheFront Serves 150 Million Reads per Second
Uber's CacheFront integrates Redis with Docstore and uses Flux CDC, TTLs, and cache inspection to manage consistency at high read volume.
On this page
In an engineering post published in 2025, Uber reports CacheFront served more than 150 million rows per second during peak hours. The post’s central engineering issue is cache freshness: asynchronous invalidations can fail, and a short TTL does not necessarily mean the cached value itself is only slightly out of date.
+--------------------------------------------------------------------------+
| Uber Docstore & CacheFront Architecture |
+--------------------------------------------------------------------------+
| Application client |
+--------------------------------------------------------------------------+
|
v
+--------------------------------------------------------------------------+
| Stateless Query Engine Tier |
| +--------------------------------------------------------------------+ |
| | CacheFront: integrated cache for Docstore reads | |
| +--------------------------------------------------------------------+ |
+--------------------------------------------------------------------------+
| |
| cache hit | cache miss / write
v v
+------------------+ +------------------+
| Redis Cache Tier | | MySQL Storage |
| (In-Memory RAM) | <===[Write Invalidation]=== | Engine Nodes |
+------------------+ +------------------+
The Architecture: Three Decoupled Tiers
The post describes an integrated cache for Docstore and a Flux pipeline that tails MySQL binlogs. At a high level:
- CacheFront handles integrated cache reads.
- Redis holds cached row values.
- Docstore storage persists rows; Flux tails MySQL binlogs for invalidation and inspection.
The Write Path Challenge: Point Writes vs Conditional Updates
When data mutates in the database, invalidating the cached representation in Redis is mandatory:
+---------------------+-------------------------+--------------------------+
| Mutation Type | SQL Query Example | Invalidation Complexity |
+---------------------+-------------------------+--------------------------+
| Point Write | UPDATE trips | Trivial: The primary key |
| | SET status = 'ended' | (`trip_uuid`) is known |
| | WHERE id = 'trip_99182' | before query executes. |
+---------------------+-------------------------+--------------------------+
| Conditional Update | UPDATE drivers | High: Which driver IDs |
| | SET active = false | matched the WHERE filter |
| | WHERE idle_mins > 30 | is unknown prior to SQL. |
+---------------------+-------------------------+--------------------------+
Because conditional updates do not declare their matching row keys upfront, the Query Engine historically could not know which Redis keys to invalidate during the write request.
The Staleness Beyond TTL Trap
Prior to CacheFront’s latest architecture, Uber relied on two mechanisms:
- Flux CDC: Tails MySQL binlogs and can invalidate cache entries asynchronously.
- Time-to-live (TTL): Expires cached entries after a configured duration; Uber says its recommended default on onboarding is five minutes.
This design concealed a dangerous distributed consistency flaw: Staleness Beyond TTL.
[Timeline of a Stale Data Outage]
T = 0: Row written to MySQL one year ago (Value: 'NYC').
T = Day: User reads row; CacheFront loads 'NYC' into Redis with 1-hour TTL.
T = Day+1s: User updates row in MySQL (New Value: 'SFO').
Flux CDC attempts to invalidate Redis, but network blip drops it!
T = Day+2s: User reads row back.
CacheFront returns 'NYC' from Redis!
Even though the TTL was set to only 1 hour, the user was served data that was one year out of date. The TTL only controls how long the entry resides in RAM, not the historical age of the underlying data.
The Breakthrough: Soft Deletes and Microsecond Timestamps
Uber says two storage changes made it possible to identify affected rows and invalidate them on the write path:
+---------------------------------------------------------------+
| 1. Universal Soft Deletes: |
| Deletes are soft, with tombstones later garbage-collected. |
| |
| 2. Strictly Monotonic Microsecond Commit Timestamps: |
| Session timestamps are strictly monotonic at microsecond |
| precision within a storage node and its Raft group. |
+---------------------------------------------------------------+
With these guarantees in place, Docstore can determine the exact row keys touched by any conditional update:
[Write Transaction: UPDATE drivers SET active=false WHERE idle_mins>30]
|
v
+---------------------------------------------------------------+
| Transaction exposes the rows affected by its update |
+---------------------------------------------------------------+
|
v
+---------------------------------------------------------------+
| CacheFront can invalidate the corresponding cache entries |
+---------------------------------------------------------------+
|
v
+---------------------------------------------------------------+
| Changed rows are identified for invalidation |
+---------------------------------------------------------------+
|
v
+---------------------------------------------------------------+
| Cache values are invalidated |
+---------------------------------------------------------------+
The Triple Defense Strategy
Uber describes three freshness mechanisms:
+--------------------------------------------------------------------------+
| Uber's Triple Defense Resilience Stack |
+--------------------------------------------------------------------------+
| Write-path invalidation |
| - Uses identified changed rows to invalidate cache entries. |
| |
| Flux CDC |
| - Tails MySQL binlogs and supports invalidation and inspection. |
| |
| Defense 3: Time-To-Live (TTL) Safety Net |
| - Automatically purges cold entries after their configured duration. |
+--------------------------------------------------------------------------+
Verification: The Cache Inspector
Uber’s Cache Inspector uses Flux to read the same MySQL binlogs with an induced one-minute delay, then compares those values with entries in the cache and exports mismatch and staleness metrics.
Uber reports that stale values were negligible relative to total reads and writes for one inspected table over a week. For that table, it says a 24-hour TTL raised the cache hit rate above 99.9%. Separately, the post reports more than 150 million rows served per second across CacheFront during peak hours; these are vendor-reported figures, not independent measurements.
Frequently asked questions
What is CacheFront?
Uber describes CacheFront as an integrated cache for Docstore. It serves row reads from Redis and uses its storage and change-data-capture systems to manage cache freshness.
What is the 'Staleness Beyond TTL' paradox in distributed caches?
TTL bounds how long a cached entry remains after it is populated; it does not bound the age of the value itself. Uber gives an example where a failed invalidation can leave an old value cached until that TTL expires.
How did soft deletes and monotonic timestamps unlock synchronous cache invalidation at Uber?
Uber says soft deletes and strictly monotonic microsecond session timestamps helped the storage layer identify the rows changed by transactions. This enabled write-path invalidation for writes whose affected rows were previously difficult to identify.
What constitutes Uber's Triple Defense caching strategy?
The article describes write-path invalidations, Flux consuming MySQL binlog changes, and TTL expiry. Cache Inspector compares cache contents with binlog events after a delay to measure remaining staleness.



/* Comments */
Comments are offline right now — we reconnect automatically, nothing is lost.