System Design · Lesson 8 of 11

Rate Limiting

Decide who gets turned away, before load decides it for you.

The idea

Rate limiting decides, deliberately, who gets turned away. Without it the decision still gets made — by whichever resource exhausts first, and usually in the worst possible way, with every user degraded rather than the abusive one stopped.

The algorithms differ mainly in how they treat bursts:

Fixed window counts requests per calendar minute. It is the easiest to implement and has a real flaw: a client can send the full allowance at 11:59:59 and again at 12:00:00, so the actual peak is double the limit.

Sliding window log keeps timestamps and is exactly correct, at the cost of memory proportional to the request rate.

Sliding window counter blends the current and previous window by how far through you are. It is very nearly exact and costs two numbers, which is why it is the common production choice.

Token bucket refills tokens at a fixed rate up to a maximum. A request costs a token. This permits bursts up to the bucket size, which is usually what you want — a client that has been idle should be allowed to catch up.

Leaky bucket drains at a constant rate and permits no burst at all. Use it when the thing you are protecting genuinely cannot absorb a spike.

Two design questions matter as much as the algorithm. What is the key? Per IP punishes everyone behind a NAT; per API key is precise but requires authentication; per user is right for logged-in traffic. Most real systems layer several. And where does the state live? In-memory per instance is fast and wrong — with ten instances the effective limit is ten times what you configured. Shared state in Redis is the standard answer.

When you reject, say so properly: 429, with Retry-After. A client that knows when to come back stops hammering; one that gets a generic 500 retries immediately and makes it worse.

Walkthrough

No walkthrough for this topic yet — the guided exercises below run the same way.

When to reach for it

Rather than the obvious alternative

Autoscaling

Adds capacity for legitimate load, and happily scales up to serve an attack at your expense. Use both: limits decide who deserves capacity, scaling provides it.

A queue

Delays work rather than rejecting it, which is right when the work must eventually happen. Rate limiting is for when the correct answer is "no".

A circuit breaker

Protects *you* from a failing dependency; rate limiting protects a dependency from you. They solve opposite directions of the same relationship.

Key terms

Rate limiting
Capping request rate per client.
Authentication vs authorization
Who you are versus what you may do.
Least privilege
Granting only the access required.
Defence in depth
Layering controls so one failure is not fatal.

How to spot it

Where it goes wrong

Per-instance counters

Ten instances each allowing 100 requests per minute is a limit of 1000, not 100. Share the state or divide the budget explicitly.

Limiting by IP alone

One corporate NAT is thousands of users on one address. Key by user or API key where you can, and treat IP as a coarse backstop.

Returning the wrong status

A 500 tells the client to retry immediately. A 429 with Retry-After tells it when to come back.

Rate limiting after the expensive work

The check has to happen before the database query it is meant to protect, or it protects nothing.

Build it

You have read how it works. This is where you write one — across several files, one rule at a time, run against tests in your browser.