Rate limiting

Problem

One bad actor can consume all your capacity. And even good actors need bounds so you can capacity-plan.

Context

Public APIs, multi-tenant systems, expensive endpoints.

Solution

Enforce a per-caller (per-IP, per-key, per-tenant) request budget. Common algorithms: token bucket (allows bursts), leaky bucket (smooths bursts), sliding window (accurate). Reject over-budget requests with 429.

Trade-offs

• Adds latency + complexity
• Global rate limits require distributed state (Redis)
• Rate limits are coarse — a huge request and a tiny one look the same

Failure modes

• Legitimate spike gets throttled (launch day) — need feature flag to raise
• Rate limit storage down → local fallback allows everything
• Retry-after honored badly by clients → retry storm

When to use

• Public APIs
• Multi-tenant fairness
• Anti-abuse (login endpoints, spam)

When NOT to use

• Fully internal, trusted callers
• Ultra-low-latency paths