The Database Scaling Ladder: What to Try First and What Each Rung Costs
Indexes, bigger box, pooling, replicas, caching, vertical split, then sharding. Climb in order, because each rung buys time and adds a new failure mode.
On this page
“Our database can’t keep up” gets answered with “shard it” far too often. Sharding is the last rung of a ladder, and most products never need to reach it. The skill is knowing which rung your bottleneck actually lives on.
Diagnose before you climb
Three different problems look identical from the dashboard (“the database is slow”):
| Bottleneck | Symptom | Rung that helps |
|---|---|---|
| Bad queries | A few statements dominate time; sequential scans | Indexes, query rewrites |
| Too many connections | Memory pressure, connection errors as app servers scale out | Pooling |
| Read volume | CPU and IO saturated by SELECTs | Replicas, cache |
| Write volume or table size | Primary saturated, vacuum and maintenance struggling, IOPS ceiling | Partitioning, sharding |
Measure first. The slow query log and your database’s statistics views tell you which row of this table you are on.
The ladder
- 1. Indexes and query fixes
The cheapest and highest-leverage rung. A correct index turns a scan into a seek. The trade: every index speeds some reads and slows every write, and unused indexes are pure cost. Remove the ones nothing uses.
- 2. A bigger machine
Vertical scaling: more CPU, memory, faster disks. It needs no code changes and buys real time. Its limit is that the biggest instance your provider sells is a hard ceiling, and the price curve gets steep near the top.
- 3. Connection pooling
A Postgres connection is a process with its own memory. Hundreds of app instances each opening dozens of connections will hurt the server before queries do. A pooler such as PgBouncer sits in front and multiplexes many clients onto few server connections. Watch for features that rely on session state, which poolers in transaction mode break.
- 4. Read replicas
Copies of the primary that serve reads. This scales read throughput and adds redundancy. The cost is replication lag: a user who just wrote something may not see it on a replica. Route reads that need freshness to the primary.
- 5. Caching
Keep hot, repeatable reads in memory in front of the database. The biggest win per dollar for read-heavy workloads, and the biggest source of stale-data bugs. Decide invalidation before you ship.
- 6. Vertical partitioning
Move whole tables or table groups, such as files and organizations, into their own databases, each with its own pooler. Each database now carries a slice of the load. It is a modest change if tables do not join across the boundary, and it ends where your single largest table outgrows one machine.
- 7. Horizontal partitioning (sharding)
Split the rows of one big table across several physical databases by a key. Now writes scale too. Now you also own routing, cross-shard queries, rebalancing and operational complexity.
Figma has publicly described growing Postgres roughly a hundredfold. They started on one large RDS instance, moved to the biggest instance size, added read replicas and PgBouncer, then split high-traffic tables such as files and organizations into separate databases, and only later horizontally partitioned the largest tables behind a routing proxy once table size, vacuum behaviour and IOPS limits became the problem. Each step answered a measured bottleneck.
Why sharding is last
Sharding removes the ceiling and adds permanent costs.
Postgres’s own table partitioning splits one logical table into pieces inside one database. It is a useful intermediate step for maintenance (dropping an old partition instead of deleting rows) and for keeping vacuum manageable, but it does not give you more machines.
The application tier tells the same story
The data layer is only part of scaling. Systems that grow gracefully also make the stateless tier easy to multiply.
- Stateless services keep no per-user data in memory, so any instance can serve any request.
- Horizontal scaling and load balancing add instances and spread requests across them.
- Auto scaling adjusts instance counts to real traffic instead of a guess.
- Async processing pushes slow work (emails, exports, image processing) onto queues and background workers so request latency stays flat.
Each of these pushes load toward the database faster, which is why the database ladder matters.
Do
Write down the measured bottleneck before each change, pick the cheapest rung that addresses it, and record what you will watch afterwards: replica lag, pool saturation, p99 latency.
Don't
Add replicas to fix a write bottleneck, or shard to fix a missing index. The new infrastructure will work, cost a fortune, and leave the original problem standing.
Takeaway
Treat scaling a database as a ladder of increasing cost and decreasing reversibility. Fix queries, buy headroom, pool, replicate, cache, split by table, and shard only when a single table’s size or write rate demands it. Each rung is a trade, and the right one is whichever answers the bottleneck you measured this week.
Frequently asked questions
What should I try first when my database is slow?
Look at the queries before the hardware. Find the slowest and most frequent statements, check their plans, and add or fix indexes. A missing index can turn a millisecond lookup into a full table scan, and no amount of replicas or extra servers fixes that cheaply.
When do read replicas stop helping?
When writes, not reads, are the bottleneck, or when the primary cannot keep up with the volume of changes it must ship to replicas. Replicas scale reads and add redundancy. They also lag, so anything needing read-your-own-writes must still hit the primary.
What is the difference between vertical partitioning and sharding?
Vertical partitioning moves whole tables, or groups of tables, to separate databases, so each handles a subset of the workload. Sharding splits the rows of one table across databases by a key. The first is easier but caps out at the size of your largest table.
Why use a connection pooler like PgBouncer?
Each Postgres connection is a process with real memory cost, so thousands of application connections degrade the server. A pooler multiplexes many client connections onto a small number of server connections, keeping the database responsive as the app tier scales out.
/* Comments */
Comments are offline right now — we reconnect automatically, nothing is lost.