How Meta Scaled FOQS, a Priority Queue Built on Sharded MySQL
Meta's FOQS combines MySQL-backed persistence, per-shard ready-item indexes, prefetching, leases, and tenant capacity controls.
On this page
The 2021 engineering post says FOQS supports hundreds of Facebook use cases, including asynchronous computing, video encoding, and language translation. It documents the service’s API and persistence path rather than claiming a current throughput benchmark.
The Scale Problem: Workload Diversity at Meta
The documented use cases include work that can be deferred, such as video encoding and translation, as well as broader asynchronous platform tasks.
FOQS lets producers attach priority and optional delivery delay to work. Namespaces separate use cases and have guaranteed enqueue capacity; topics within a namespace are logical priority queues.
[Client Services (Publishers)]
|
v
+-------------------------------------------------------------+
| FOQS Ingress Gateway |
| - Rate Limiting & Namespace Quota Verification |
| - Shard Router (Hash Partitioning by Namespace / Key) |
+-------------------------------------------------------------+
| |
v (Batch Writes) v (Batch Writes)
+---------------------------+ +---------------------------+
| MySQL Shard 1 | | MySQL Shard 2 |
| - InnoDB Table (Items) | | - InnoDB Table (Items) |
| - Active Memory B-Tree | | - Active Memory B-Tree |
+---------------------------+ +---------------------------+
| |
v (Prefetch Workers) v (Prefetch Workers)
+-------------------------------------------------------------+
| FOQS Dequeue Prefetch Engine |
| - In-Memory K-Way Priority Merge Heap |
| - Lease Management & Visibility Timeout Clock |
+-------------------------------------------------------------+
|
v
[Worker Pool (Consumers: Acks / Nacks)]
The Data Model: Item Lifecycle on Sharded MySQL
Meta’s post describes items with a namespace, topic, user-specified 32-bit priority (lower numbers are higher priority), payload, optional dequeue delay, lease duration, unique ID, and TTL. Each item maps to one MySQL row.
Schema Indexing Strategy
The published design does not specify the composite SQL index or SKIP LOCKED query asserted in an earlier draft. It describes a ready-item index maintained in memory and database row state updated to prevent double delivery.
Enqueue vs Dequeue Optimizations
1. High-Throughput Enqueue Batching
An FOQS host buffers incoming enqueue requests and has a worker per MySQL shard insert queued items. The post does not publish a fixed batch interval or batch size.
2. K-Way Priority Merge Prefetching
To avoid making a consumer request synchronously search all shards, FOQS uses an intermediate Prefetch Buffer:
- A background prefetch buffer uses client demand to replenish work from shards.
- The host merges shard candidates by priority and stores them for dequeue calls.
- The post does not claim a sub-five-millisecond latency.
Fault Tolerance: Leases and At-Least-Once Delivery
Distributed queues must handle worker crashes cleanly. When a worker dequeues an item:
- FOQS starts the item’s configured lease when it is dequeued.
- The worker executes the task.
- If successful, the worker sends an
ACK; otherwise it can send aNACKwith a delay. - If it sends neither, expiry makes the item eligible for delivery again according to the selected at-least-once or at-most-once policy.
Because network partitions can cause duplicate deliveries after lease timeouts, FOQS requires worker consumer handlers to be strictly idempotent.
Do
- Design worker consumer tasks to be strictly idempotent, using unique transaction IDs to discard duplicate deliveries.
- Implement memory prefetch buffers with k-way priority merging to eliminate database query stalls on dequeue.
- Isolate tenant workloads with namespace rate limits to prevent bulk batch backfills from starving real-time tasks.
Don't
- Do not poll database tables continuously without indexing
(priority, deliver_after). - Avoid holding long-lived synchronous database transactions while external worker tasks execute.
- Never use unbound in-memory queues without backpressure mechanisms and circuit breakers.
FOQS demonstrates one trade-off: durable queue items can live in sharded relational storage while host-side indexes and prefetch reduce work on the consumer path. Its leases and retry policy make duplicate handling an application concern; capacity guarantees and shard health controls address service isolation. These are mechanisms described in Meta’s post, not a claim about today’s scale or latency.
Frequently asked questions
What is FOQS and what does it use MySQL for?
Meta describes FOQS as a persistent, multitenant distributed priority queue built on sharded MySQL. Each queue item corresponds to a row, while FOQS hosts manage API traffic and dequeue optimization.
How does FOQS handle priority scheduling without slow full-table scans?
Each shard keeps an in-memory index of ready item primary keys ordered by priority. A prefetch buffer combines candidates across shards with a k-way merge, then retrieves rows from MySQL.
What is lease-based visibility in distributed priority queues?
A dequeued item has a lease. If the consumer neither acknowledges nor negatively acknowledges it before the lease expires, FOQS can redeliver it according to the configured retry and delivery policy.
How does FOQS achieve multi-tenant isolation across thousands of internal workloads?
Namespaces are FOQS's multitenancy unit and receive guaranteed capacity measured in enqueues per minute. Meta says namespaces sharing a tier should not affect one another within that capacity model.



/* Comments */
Comments are offline right now — we reconnect automatically, nothing is lost.