---
topic: system-design
author: Crashtech Editorial
published: Nov 15, 2026 · read: 3 min
---

Rate Limiting Strategies Explained: Token Bucket, Leaky Bucket and Sliding Windows

How rate limiters protect backend systems from cascading collapse, compare algorithms for burst tolerance and fairness, and handle distributed Redis state.

– –

Without rate limits, a single misconfigured cron job, malicious scraping script, or partner retry storm can cascade across your architecture, exhausting connection pools and taking down entire services. Rate limiting is not an edge-case security feature; it is core infrastructure for survivability.

The Four Core Rate Limiting Algorithms

Every rate limiting algorithm balances memory footprint, computational overhead, and burst accommodation:

AlgorithmBurst ToleranceMemory CostSmoothing BehaviorIdeal Production Use Case
Token BucketHigh (up to capacity)Minimal (2 variables)Bursty up to limitUser-facing APIs, webhook dispatchers
Leaky BucketNone (constant drain)Minimal (queue/variables)Perfect smoothingDownstream payment gateways, slow third-party sinks
Fixed WindowHigh (2x at boundary)Lowest (1 counter)UnevenBasic internal microservice throttling
Sliding Window LogPerfect accuracyHigh (O(requests))ContinuousHigh-value, low-throughput financial transactions
Sliding Window CounterGood approximationLow (2 counters)Smooth rolling curveHigh-throughput distributed API gateways

Token Bucket: Accommodating Real-World Bursts

The Token Bucket algorithm models a bucket of fixed capacity $C$ refilled with tokens at a constant rate $r$ tokens per second.

Incoming Tokens (Rate r/sec) ---> [  Bucket (Capacity C)  ]
                                       |
Incoming Request ----------------> Consumes Token?
                                       |---> Yes: Request Allowed
                                       |---> No:  HTTP 429 Rejected

When a request arrives:

  • If tokens are available, one token is decremented and the request passes.
  • If the bucket is empty, the request is immediately rejected with HTTP 429.

Instead of running background background timers to add tokens, implementations calculate tokens lazily upon each request:

current_tokens = min(C, last_tokens + (now - last_updated) * r)

This requires storing only two numbers per user key: the timestamp of the last request and the remaining token count.

The Sliding Window Counter: Best of Both Worlds

Fixed windows suffer from boundary spikes. Sliding window logs maintain an in-memory timestamp for every request, which explodes memory under high throughput.

The Sliding Window Counter hybrid solves this by weighting previous and current window counts:

Estimated Count = Current Window Count + Previous Window Count * (1 - (Time Elapsed in Current Window) / (Window Size))

If the window is 60 seconds, 15 seconds have passed in the current minute (25% elapsed), the previous minute had 100 requests, and the current minute has 20 requests:

Estimated Count = 20 + 100 * (1 - 0.25) = 20 + 75 = 95

If the limit is 100, the request is admitted. If it were 94, it would be rejected. Memory remains constant at two counters per key.

Distributed Rate Limiting: Solving Race Conditions in Redis

In a cluster of API gateway nodes, rate limit counters must be synchronized. A naive implementation does:

  1. GET user:102:tokens
  2. Increment or decrement in application memory
  3. SET user:102:tokens

Under high concurrency, two requests reading simultaneously will both observe valid tokens, allowing double the allowed quota.

The Lua Script Fix

To guarantee atomicity, all calculations must run within a single Redis Lua script:

-- KEYS[1]: User rate limit key
-- ARGV[1]: Bucket capacity
-- ARGV[2]: Refill rate per millisecond
-- ARGV[3]: Current timestamp (ms)

local data = redis.call('HMGET', KEYS[1], 'tokens', 'last_updated')
local capacity = tonumber(ARGV[1])
local rate = tonumber(ARGV[2])
local now = tonumber(ARGV[3])

local tokens = tonumber(data[1])
local last_updated = tonumber(data[2])

if not tokens then
    tokens = capacity
    last_updated = now
else
    local delta = math.max(0, now - last_updated)
    tokens = math.min(capacity, tokens + delta * rate)
    last_updated = now
end

if tokens >= 1 then
    tokens = tokens - 1
    redis.call('HMSET', KEYS[1], 'tokens', tokens, 'last_updated', last_updated)
    redis.call('EXPIRE', KEYS[1], 3600)
    return 1 -- Allowed
else
    redis.call('HMSET', KEYS[1], 'tokens', tokens, 'last_updated', last_updated)
    return 0 -- Rejected
end

When Redis is degraded or partitioned, decide your failure mode in advance. Fail-open (admit traffic when rate limiter errors) prioritizes user experience during outages. Fail-closed (reject requests when rate limiter is down) protects sensitive backends from catastrophic overload.

Client Communication Contract

When returning HTTP 429, respect standard header contracts:

HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Retry-After: 30
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1731604500

{
  "error": {
    "code": "rate_limit_exceeded",
    "message": "Too many requests. Please retry after 30 seconds."
  }
}

Clear response headers allow well-behaved SDKs and clients to implement exponential backoff and jitter, turning uncontrolled traffic storms into orderly, predictable queues.

Advertisement

Frequently asked questions

What is the difference between Token Bucket and Leaky Bucket?

Token Bucket allows temporary traffic bursts up to bucket capacity while enforcing an average fill rate. Leaky Bucket processes requests at a strictly constant output rate regardless of ingress bursts, smoothing traffic out like a FIFO queue.

Why is Fixed Window Counter vulnerable to traffic spikes?

If the window boundary is 1 minute and the limit is 100 requests, a client can send 100 requests at 00:59 and another 100 requests at 01:01. Over a two-second interval, 200 requests pass through, effectively doubling the permitted peak capacity.

How do distributed rate limiters avoid race conditions in Redis?

Simple read-modify-write patterns cause race conditions under concurrent requests. Production systems execute rate limiting logic inside atomic Redis Lua scripts or use Redis cell modules, ensuring counters increment and compare in a single round-trip.

What HTTP headers should an API return when rate limiting?

Return HTTP 429 Too Many Requests along with standard headers: RateLimit-Limit (quota per window), RateLimit-Remaining (tokens left), and Retry-After (seconds until the client can retry safely).

Sources & further reading

/* Comments */