System Design · core
Rate Limiter Design
Token bucket versus sliding window, and making a limiter work across many nodes.
Mental model
Fixed windows are simple and wrong at the boundary — a client can send a full quota at the end of one window and again at the start of the next, so the observed rate doubles. Token bucket fixes that by refilling continuously, which also permits a deliberate burst up to the bucket size. The hard part is distribution: a per-node limit multiplies by node count, and a shared counter adds a round trip to the hot path, so real systems either shard the budget per node or accept approximate global limits with periodic reconciliation.
How to study Rate Limiter Design
Begin by restating the mental model in your own words, then connect it to a concrete system you have built or operated. Name the mechanism, the constraint it addresses, and the trade-off it introduces. Use the linked roadmap and primary implementation references to check details, but close the source before writing your explanation. Retrieval is the learning step; rereading is only preparation.
Next, compare Rate Limiter Design with Retries & Circuit Breakers, Consistent Hashing. Ask what changes in correctness, latency, resource use, operability, and failure recovery. Complete Token bucket rate limiter and preserve the command, input, output, and one failed attempt as evidence. Finish by explaining the idea without jargon to someone who has not studied the track.
Proof of understanding
- Explain the mechanism from first principles and identify the state it reads or changes.
- Give one situation where the concept is the right choice and one where it is not.
- Predict a realistic failure mode before running the drill, then compare the prediction with evidence.
- Connect the result to a roadmap or build artifact instead of treating the concept as isolated trivia.
Where it matters
Every public API, and the control that keeps one noisy tenant from consuming a shared quota.
Common mistakes
- Fixed windows without smoothing, allowing 2x the intended rate across a boundary
- Enforcing per-node limits and being surprised the global rate is N times higher
- A synchronous shared counter on the request path, making the limiter itself the bottleneck
- Returning 429 without Retry-After, so well-behaved clients retry immediately and make it worse
Learn from primary sources
Use the linked roadmap context and practice prompt.
Practice and explain it back
Token bucket rate limiter
Implement makeLimiter(capacity, refillPerSecond) returning allow(nowMs) -> boolean. Tokens refill continuously at refillPerSecond, capped at capacity; the bucket starts full. Each allowed call consumes one token. Do not refill in discrete ticks.
Expected evidence: capacity 2, refill 1/s: two immediate calls allowed, third denied, allowed again one second later.
Open the interactive drill →Review prompts
- A fixed-window limiter allows 100 requests per minute, yet a client sustains 200 in a sixty-second span. Explain the mechanism and what fixes it.
Build evidence
Use a roadmap capstone to turn this concept into working evidence.