Rate limiting is a control that caps how many requests a client, identified by IP address or user account, can make within a set time window, then blocks or delays requests over the limit. Servers signal it with HTTP 429 Too Many Requests, which may include a Retry-After header saying how long to wait before trying again.

Levels Rate Limits Get Enforced At

  • Per IP
  • Per subnet
  • Per API key
  • Per account

When a client exceeds them, the server can temporarily stop fulfilling its requests and respond with HTTP 429 "Too Many Requests". Distributing requests across a large, rotating proxy pool spreads them across many per-IP counters, helping keep each IP under per-IP rate limits at scale.

Common Rate-Limiting Algorithms

  • Fixed window: Counts requests and resets the counter at the start of each period, so traffic spikes can slip through around each reset.
  • Sliding window: Weights the previous period's count by how much of it overlaps the current window, which smooths out those spikes.
  • Token bucket: Tokens refill at a fixed rate up to a maximum. Each request spends one, so bursts are allowed up to the bucket's size.
  • Leaky bucket: Requests drain at a constant rate. When the bucket is full, new requests are rejected.

Why Sites Use Rate Limiting

Scraping is only one target. Rate limits also help:

  • Slow brute-force login attempts, which Cloudflare recommends tracking by both IP address and username.
  • Blunt DoS and DDoS traffic floods.
  • Keep API usage within per-user quotas, such as a set number of calls per hour or day, even when the overuse isn't malicious.