CDN and Anycast

In 1998, MIT professors Tom Leighton and Danny Lewin founded a company to solve a specific problem: the internet couldn't handle flash crowds. When a story went viral — a Yahoo homepage feature, an Olympic result — the origin server would melt under the sudden load. Every user, worldwide, hit the same rack of servers. Leighton and Lewin's idea was to replicate the content to edge servers physically close to users. The company they founded was Akamai.

Today, every major website relies on CDNs. Cloudflare has 300+ data centers. Akamai has 4000+. Fastly, Amazon CloudFront, Google Cloud CDN, Alibaba, Bunny CDN, each with their own edge networks. Together they absorb the majority of internet traffic — most requests to popular sites never reach the origin. When you load Netflix, ~99% of the bytes come from an Open Connect Appliance embedded inside your ISP's network. Origin is barely touched.

CDNs work on a beautiful trick: Anycast at Layer 3. The same IP address is announced from hundreds of physical locations via BGP. When your packet leaves your ISP, the internet's routing tables direct it to the closest instance. No DNS trickery required — the IP is the same everywhere. This concept extends beyond CDNs: Google DNS 8.8.8.8, Cloudflare 1.1.1.1, and the root DNS servers all run on Anycast.

Anycast — the same IP, everywhere

Every host on the internet has one IP. Or so we're taught. Anycast breaks that: the same IP is announced from N different physical locations. When a packet is routed to that IP, the internet's BGP routing tables direct it to the topologically nearest instance — usually also the geographically nearest.

This works because BGP routes are computed per-router. Each router sees multiple paths to the same destination IP and picks the one with the fewest AS hops. As long as each instance announces the route from a similarly-positioned network, traffic naturally lands nearby. No DNS decision required. Failover happens at the routing layer — if an instance goes offline, BGP withdraws its announcement and traffic reroutes to the next-closest one within seconds.

What a CDN actually does

A CDN is much more than a cache. Modern CDNs like Cloudflare and Fastly do:

Cache hit vs miss — what happens on the edge

On a cache hit, the CDN edge responds in <10ms — the request never touches your origin. On a cache miss, the edge fetches from origin, caches it, and returns it. A well-tuned CDN can have 95%+ cache hit rates for static assets — meaning your origin server carries 5% of the load a naïve architecture would need. That's the operational unlock.

Cache-Control — the language of caching

CDNs respect the Cache-Control HTTP header from your origin. Getting this right is what separates a 99% cache hit rate from 0%:

• max-age=3600 — browsers may cache for 1 hour.
• s-maxage=86400 — CDN edges specifically may cache for 24 hours. Overrides max-age for shared caches.
• public — any cache may store. private — only the end-user browser (not CDN).
• no-store — do not cache at all. no-cache — cache, but always revalidate with origin (via ETag).
• stale-while-revalidate=60 — serve stale content while refreshing in the background. The killer feature that keeps origin load steady even during traffic spikes.

GeoDNS vs Anycast — the two edge-routing strategies

Historically, DNS-based edge selection was more common (Akamai used it heavily). Modern CDNs (Cloudflare, Fastly, Bunny) overwhelmingly use pure Anycast. AWS CloudFront uses a hybrid: Anycast IPs, but with regional pool selection via internal routing.

DDoS absorption — the accidental superpower

Anycast turns out to be extraordinary at absorbing DDoS attacks. A single 1 Tbps attack against your origin would take you down. The same attack against Cloudflare's Anycast IP spreads naturally across 300+ data centers — each center sees ~3 Gbps, well within capacity. Cloudflare publishes reports of absorbing 26 Tbps+ attacks without customers noticing. This is the primary reason startups put Cloudflare in front of everything.

Applied in real systems — the CDN landscape

Key takeaways

• A CDN caches your content at edge servers physically close to users. Reduces origin load 20-100x. Reduces user latency 5-10x.
• Anycast (Layer 3) is the routing trick that makes it work. Same IP announced from many places via BGP; each user gets routed to the closest.
• Cache-Control headers from your origin drive CDN behavior. Get them right or you'll have 0% cache hit rate. s-maxage and stale-while-revalidate are the operator's superpowers.
• Anycast is a DDoS shield. A 1 Tbps attack against a 300-PoP Anycast is 3 Gbps per PoP — easily absorbed. Origin protection is often the #1 reason to adopt a CDN.
• Netflix Open Connect + Google Global Cache extend the CDN concept: physical hardware inside ISP networks. Nearly all bytes are served from inside the last mile.
• Edge compute (Cloudflare Workers, Fastly Compute) blurs the CDN + application server line. Personalized responses without round-tripping to origin.
• Every serious system-design answer at global scale starts with "CDN in front of everything."

References

• Karger et al. (1997) — Consistent Hashing. The Akamai foundation.
• Nygren, Sitaraman, Sun (2010) — The Akamai Network. ACM SIGOPS.
• Adhikari et al. (2012) — "Unreeling Netflix." Analysis of Netflix's CDN.
• Cloudflare Radar — public traffic analytics at radar.cloudflare.com.
• RFC 4271 — BGP-4.