A token bucket limits the long-run request rate while allowing a bounded burst. Tokens accumulate in a bucket of capacity B at rate r; each request consumes its cost. If too few tokens remain, the service rejects the request or applies a separate queueing policy. The Redis rate-limiter guide demonstrates a Redis token bucket, while RFC 6585 §4 defines the meaning of HTTP 429.

Read capacity and refill rate as separate controls

Assume a bucket holds five tokens and refills at two tokens per second. After enough idle time, a burst of up to five one-token requests can pass quickly. Once empty, the bucket recovers one token every half-second on average. “Two per second” therefore does not mean exactly two requests may arrive in every instant-sized interval: capacity bounds the immediate burst, while refill rate controls the sustainable average.

Let T be the previous token count, t₀ the last calculation time, and t₁ the current time. Before deciding a request, compute min(B, T + (t₁−t₀)×r). If the result covers the request cost c, allow the request and subtract c; otherwise reject it. Calculating on arrival avoids a timer that continuously writes replenished tokens.

Keep shared state atomic across instances

If every gateway has its own bucket, the effective allowance can multiply with the number of gateways. For a global limit, store shared state under a deliberate identity such as an account or API key, and make refill calculation plus deduction atomic so concurrent requests cannot spend the same token twice. Redis demonstrates Lua scripts for this operation, but Lua is one implementation pattern; suitable atomic commands, transactions, or another coordination store may fit a different design.

Choose identity with the traffic model in mind. An account key may suit authenticated usage; IP-based limits can help with anonymous traffic, but NAT can group unrelated people and distributed clients can use multiple addresses. A single identifier is not a complete abuse-control strategy. Cost-weighted requests can also reflect that one report-generation call uses more resources than a small lookup.

Token bucket versus a leaking queue

A token bucket can pass a burst that accumulated while the system was quiet. A leaky-bucket design backed by a queue instead releases work at a steadier pace, which smooths downstream load but introduces queue length and waiting time. Immediate rejection and delayed processing are different product behaviors. Choose based on whether the backend can absorb bursts and whether the work remains useful after a delay.

Return 429 without promising a mandatory header

A server can use HTTP 429 when a client has sent too many requests. RFC 6585 says the response may explain the condition and that the server MAY include Retry-After; the header is not mandatory on every 429. A client should honor it when present. If it is absent, use the documented client policy, such as bounded exponential backoff with jitter, and account for whether the operation is safe to repeat. Uncoordinated retries can add load precisely when the service is constrained.

Sources