How do you migrate data between shards without downtime?
Copy a snapshot in bulk, then stream every change made after the snapshot from the source's operation log until the target has caught up. Verify both sides match, briefly block writes on the source, finish replication, and switch routing. The write pause lasts seconds, and clients retry through it.
Answered in
Moving Live Data Between Shards With Zero Downtime: The Six-Step ProtocolRegister, bulk copy, replicate the tail, verify, flip traffic behind a version gate, clean up. How online shard migrations work without losing a write.
Read the full analysisOther questions this article answers
More system design & architecture questions
- What is the architectural difference between an API Gateway and a Reverse Proxy?
- When should an architecture use Layer 4 instead of Layer 7 load balancing?
- How do edge ingress layers prevent backend cascading thundering herds?
- What algorithm is best for sticky sessions without breaking even distribution?
- How can a CDN improve performance for dynamic requests?
- What is the hardest part of operating behind a CDN?
- How can a CDN help with traffic surges or DDoS attacks?
- What is the difference between a Push CDN and a Pull CDN?
Every answer on Crashtech is written by the editor of the article it comes from — never auto-summarised. Browse all answers or the System Design & Architecture beat.