Rate limiting looks like a simple counter until you actually implement it, and then the algorithm choice matters a lot — a naive fixed-window counter lets clients burst up to 2x the intended limit right at the window boundary, a bug that's invisible until someone actually exploits it.
Rate limiting restricts how many requests a client can make within a time period, protecting your API from abuse, ensuring fair resource allocation across clients, and preventing a single misbehaving client from degrading service for everyone else. The algorithm you choose — fixed window, sliding window, token bucket, or leaky bucket — determines the actual behavior at the edges, which matters more than it initially seems.
Why Rate Limiting Matters (and When to Skip It)
Without rate limiting, a single client (malicious or just buggy — a retry loop gone wrong is a common culprit) can consume disproportionate resources, degrading service for every other client or overwhelming downstream dependencies (databases, third-party APIs) that weren't designed for that load.
Skip elaborate rate limiting for truly internal services with a small, trusted, known set of callers where abuse isn't a realistic concern — the overhead of maintaining a full rate limiting layer isn't justified if there's no actual risk it's protecting against.
Getting Started with Rate Limiting
Token bucket, a common and well-behaved algorithm — tokens refill at a steady rate, and each request consumes one:
class TokenBucket {
private tokens: number;
private lastRefill: number;
constructor(private capacity: number, private refillRatePerSec: number) {
this.tokens = capacity;
this.lastRefill = Date.now();
}
tryConsume(): boolean {
this.refill();
if (this.tokens >= 1) {
this.tokens -= 1;
return true;
}
return false;
}
private refill() {
const now = Date.now();
const elapsed = (now - this.lastRefill) / 1000;
this.tokens = Math.min(this.capacity, this.tokens + elapsed * this.refillRatePerSec);
this.lastRefill = now;
}
}
Core Rate Limiting Concepts Every Developer Should Know
Fixed window counters are simple but allow boundary bursting. A limit of 100 requests/minute with a fixed window means a client can send 100 requests at 0:59 and another 100 at 1:00 — 200 requests in two seconds, technically compliant with the stated limit but clearly not the intended behavior.
Sliding window algorithms fix the boundary problem by considering a rolling time window rather than discrete fixed buckets, at the cost of slightly more computational and memory overhead to track request timestamps (or a weighted approximation) across the rolling window rather than a single counter.
Token bucket allows controlled bursting while enforcing an average rate. Unlike a strict fixed rate, token bucket lets a client burst up to the bucket capacity if it's been under-using its rate recently, then throttles once tokens are exhausted — this models real traffic patterns (bursty, not perfectly uniform) more gracefully than a hard per-window cap.
Distributed rate limiting needs shared state across instances. A rate limiter implemented with in-memory state on a single server doesn't work correctly behind a load balancer with multiple instances — you need a shared store (Redis is the common choice) so the limit applies correctly across the whole fleet, not per-instance.
// Redis-backed sliding window using a sorted set
async function isAllowed(key: string, limit: number, windowMs: number) {
const now = Date.now();
await redis.zremrangebyscore(key, 0, now - windowMs);
const count = await redis.zcard(key);
if (count >= limit) return false;
await redis.zadd(key, now, `${now}-${Math.random()}`);
return true;
}
Common Rate Limiting Mistakes and How to Fix Them
Mistake 1: using a naive fixed-window counter without accounting for boundary bursting, allowing effectively double the intended rate at window edges. Fix: use a sliding window or token bucket algorithm for any limit where boundary bursting is a real concern.
Mistake 2: implementing rate limiting with per-instance in-memory state behind a load balancer, making the effective limit N times higher than intended (N = instance count) and inconsistent depending on which instance handles a given request. Fix: use a shared store (Redis) for rate limit state across all instances.
Mistake 3: not returning proper rate limit headers (Retry-After, X-RateLimit-Remaining), leaving clients without the information needed to back off appropriately. Fix: return standard rate limit headers so well-behaved clients can self-throttle instead of hammering a 429 response.
When Should You Use Token Bucket Instead of Sliding Window?
Use token bucket when you want to allow controlled bursting while still enforcing a long-term average rate — a good fit for typical API traffic that's naturally bursty rather than uniform. Use sliding window when you need strict, precise enforcement of a rate limit without any burst allowance, and boundary-effect precision matters more than accommodating bursty client behavior.
Rate Limiting in Production
Use a shared, distributed store for rate limit state in any multi-instance deployment — this is a common and easy-to-miss mistake that silently multiplies your actual limit by instance count. Also return proper rate limit response headers so clients can implement correct backoff behavior instead of retrying blindly into repeated 429s.
If your API currently has no rate limiting or a naive fixed-window implementation, moving to a shared, sliding-window or token-bucket approach is a concrete, well-scoped improvement worth prioritizing before it becomes an incident.