Designing a Subscription Platform for Hundreds of Millions of Members
Catalog, state and history are three different data problems. How a global subscription platform splits them across databases and a change log.
On this page
“We have users, plans and payments” sounds like one schema. It is not. The day a support agent asks “what exactly happened to this account in March?”, or a price change for one country has to go live without touching anyone else, you find out the schema was three problems glued together.
The lifecycle you actually have to model
Every subscription product walks the same states, whether it has ten members or hundreds of millions.
- Sign up
Directly, or through a partner channel such as a mobile carrier or an app store. The plan shown depends on country, currency and device.
- Change plan
Upgrades, downgrades, extra members. Price rules differ by region and by when the member joined.
- Renew
An automatic charge against a stored payment method, on a schedule, forever.
- Recover from payment failure
Hold, grace period, retry, then lapse. The member may be using the service the whole time.
- Pause or cancel
Temporary or permanent, with a path back (the “rejoin” experience) that matters commercially.
Three data problems, three shapes
| Catalog | Member state | History | |
|---|---|---|---|
| What it holds | Plans, prices, rules per region and device | Current plan, price tier, billing partner, status | Every change ever made to a member |
| Size | Small | Huge (one record per member) | Larger still, append-only |
| Access | Read constantly, change rarely | Read per request, write per event | Write per change, read rarely |
| Needs most | Exact correctness, strong consistency | Scale and availability | Ordering and completeness |
| Natural fit | Strongly consistent SQL store | Wide-column or sharded key-value store | Append-only log |
A price that is wrong for a minute is an incident with legal weight. A member profile that is stale for a second is nothing. Giving those two different consistency guarantees is the point of splitting them.
A strongly consistent distributed SQL store such as CockroachDB makes every write agree across replicas through a transaction layer. That is exactly right for a price table and exactly wasteful for hundreds of millions of per-member rows. A Cassandra-style store gives up cross-row transactions to scale writes and stay available when nodes fail.
The hot path nobody draws on the diagram
The largest traffic is not signup or billing. It is the entitlement check: when a member presses play, or opens a page, the product asks “what does this plan allow?” Concurrent streams, devices, quality tier. That call happens on every session of every member, so it dwarfs everything else combined.
Two rules fall out of that.
- Cache the plan record with the caller. Keep a small, complete copy close to where it is used, with a short lifetime, so the check costs a memory lookup.
- Keep the check independent of the catalog. If the pricing database has a bad hour, paying members must still be able to use what they paid for. Serving the whole record from cache, rather than rebuilding it from catalog rules per request, is what prevents the strongly consistent store from becoming a single point of failure.
Signup, step by step
The discipline: the write path does the minimum to make the membership real, and everything else reacts to an event.
History: stop asking applications to remember
The first version of history is usually application events: code logs “plan changed” when it changes a plan. It works until a code path forgets, a migration updates rows directly, or two services write the same field.
The sturdier design is change data capture. The database’s own change log becomes the source: every insert, update and delete is read as an ordered event and appended to a history store. A tool like Debezium reads the Postgres write-ahead log this way, which is why it cannot miss a change the database accepted.
update billing partner
|
v
subscription service --> member state table
| (change log)
v
history service --> append-only member history
|
v
"membership updated" event --> downstream consumers
What you get for free:
- Debugging. You can read the exact sequence that produced today’s state.
- Replay and reconciliation. If a derived table is wrong, rebuild it from a known good point by replaying the log.
- Support answers. “Why was I charged this?” becomes a lookup instead of an investigation.
- Offline reconciliation. Batch jobs compare systems of record, such as subscriptions against history against billing, and flag drift before customers do.
Append-only only holds if nothing can update it. Give the history writer its own credentials and no update or delete rights, or the replay guarantee is a hope, not a property.
Start small, keep the seams
You do not need three databases on day one. You need three seams.
Do
Keep catalog tables, member state and history in separate schemas with separate access paths. Read the catalog through one module, write history from the change log or an outbox, and make the entitlement check cache-friendly. Splitting stores later is then a move, not a rewrite.
Don't
Join the live pricing table into every member read, or let application code decide what gets logged to history. Both work in the demo and both become the outage and the audit gap at scale.
Takeaway
Treat a subscription platform as catalog, state and history. Give the catalog strong consistency, give member state scale, and derive history from the change log rather than from good intentions. Protect the entitlement check as the hottest path in the product, and everything around it gets easier to reason about.
Frequently asked questions
Why split a subscription system into catalog, state and history?
They have different access patterns. The pricing catalog is small, read constantly and must be exactly right. Member state is huge and updated per user. History is append-only and read rarely, mostly for debugging and support. One database tuned for all three is tuned well for none.
What is change data capture and why use it for member history?
Change data capture records every row-level change in a database as an ordered stream of events. Writing history from that stream, instead of from application code that must remember to log, means no change can be missed, and the log can be replayed to rebuild or audit state.
Do you need two databases for a subscription product?
Not at small scale. One Postgres instance handles catalog, state and history comfortably for years. Separate stores pay off when one workload, usually the per-member read path on every play or page load, outgrows what the strongly consistent store should serve.
How should entitlement checks be kept fast?
Treat them as the hottest path in the product. Cache the plan record close to the caller, keep the cached copy small and complete, and make the check independent of the pricing catalog so a catalog outage never blocks a paying member from using the service.



/* Comments */
Comments are offline right now — we reconnect automatically, nothing is lost.