How much spare capacity should a gateway keep?
There is no universal utilization target. Size the fleet against explicit failure scenarios such as losing the largest unit, region or dependency, then load-test at the resulting N-1 traffic mix. Keep a measured reserve for bursts, control-plane work and recovery rather than relying on average utilization.
Answered in
Designing Gateways That Fail Without Taking the Fleet DownA technical blueprint for gateway overload control: N-1 capacity, bounded queues, load shedding, isolated failover and controlled recovery.
Read the full analysisOther questions this article answers
More system design & architecture questions
- Why are bounded queues important for availability?
- Should a load balancer always move requests away from an unhealthy server?
- Can a circuit breaker prevent every cascading failure?
- Why does pagination improve API performance beyond just 'smaller responses'?
- Why is asynchronous logging faster than writing logs synchronously?
- When does caching actually help API performance, and when doesn't it?
- Why does payload compression trade CPU time for network time, and when is that worth it?
- What specific overhead does connection pooling eliminate?
Every answer on Crashtech is written by the editor of the article it comes from — never auto-summarised. Browse all answers or the System Design & Architecture beat.