Instagram

Same startup. Same engineer #4. Different Monday from the URL Shortener and Slack you've already shipped.

Your CTO drops by with coffee — and she's grinning this time. *"Users love the platform. They want a photo-sharing feature. Photos and short videos. Comment. Like. Follow. A feed that surfaces what they'd actually want to see. Ship an MVP in 12 weeks. Same design discipline as last time. But this one has a twist you haven't hit yet — the ranking is the product."*

She walks away.

Here's the new pattern to internalize before we design a single component:

URL Shortener was read-heavy. 100 redirects per URL created. Your job was to make ~40µs cache hits and avoid DB stampedes.

Slack was real-time write-heavy. Every message fans out to N members in <100ms. Your job was to route WebSockets by team affinity so writes and pushes don't cross-talk.

Instagram is a ranking system that also happens to store photos. The storage is trivial: S3 + a CDN. But at 2B MAU (Meta Q4 2024 earnings), the feed — deciding which of ~10,000 candidate posts to show you when you open the app — is a deep-learning ranking system running on custom silicon (Meta DLRM paper).

In 2022 Instagram made a fundamental shift: more than 50% of feed impressions are now from accounts you don't follow, ranked by an ML model (Adam Mosseri, 2022). That means the "who to follow" table you'd design at L4 is the wrong data model at L7. Understanding when the model breaks is the whole game.

Here's the shape you're going to design across the next 12 chapters:

The whole journey at a glance

Every 10× in user scale forces a fundamentally different feed generation strategy — and the crossover from "pull-based JOIN" to "pushed-fanout" to "ML-ranked candidate set" is the story:

text
    ═══════════ INSTAGRAM ARCHITECTURE ACROSS 4 SCALES ═══════════

    L4 (10K users)     L5 (100M users)    L6 (1B users)         L7 (2.4B users)
    boring stack       fanout + shard      global regions        ML ranks EVERYTHING
    12 weeks · $500/mo 6 months · $50K/mo  18 months · $500K/mo  ongoing · $5B/yr infra

    ┌────────┐         ┌────────┐         ┌──── Global CDN ────┐  ┌── Custom silicon ──┐
    │  Web   │         │  Web   │         │  photos + videos   │  │  Meta MTIA + GPU   │
    │ Mobile │         │ Mobile │         │  edge caches       │  │  DLRM inference    │
    └───┬────┘         └───┬────┘         └─┬──┬──┬─┬──────────┘  └──┬──┬──┬──┬──────┘
        │                  │                │  │  │ │                │  │  │  │
     ┌──▼───┐           ┌──▼──┐          ┌──▼──▼──▼──▼─────┐    ┌──▼──▼──▼──▼──────┐
     │ ALB  │           │ ALB │          │  Multi-region   │    │  20+ regions +   │
     │      │           │     │          │  ALB · anycast  │    │  edge presence   │
     └──┬───┘           └──┬──┘          └───┬──────┬──────┘    └────┬──────┬──────┘
        │                  │                 │      │                │      │
     ┌──▼──────┐       ┌───▼─────────┐   ┌───▼──────▼───┐    ┌───────▼──────▼──────┐
     │ 1 API   │       │  API pods   │   │  Regional    │    │  50+ ML services    │
     │ server  │       │  sharded    │   │  API + fanout│    │  DLRM ranker        │
     │         │       │  by user_id │   │  service     │    │  candidate gen      │
     └──┬──────┘       │             │   │              │    │  seed exposure      │
        │              └───┬─────────┘   └───┬──────────┘    │  causal recommend   │
        │                  │                 │               └──┬──┬──┬──┬─────────┘
        │              ┌───▼──┐          ┌───▼──────────┐       │  │  │  │
        │              │Redis │          │Redis Cluster │    ┌──▼──▼──▼──▼────┐
        │              │feed  │          │per-region    │    │TAO graph store │
        │              │cache │          │feed cache    │    │(Meta's cache)  │
        │              └───┬──┘          └───┬──────────┘    │+MyRocks tier   │
        │                  │                 │               └──┬─────────────┘
     ┌──▼───┐          ┌───▼─┐            ┌──▼──────────┐       │
     │MySQL │          │MySQL│            │Sharded MySQL│    ┌──▼──────────────┐
     │+ S3  │          │+ S3 │            │+ Vitess/TAO │    │Cassandra +      │
     │metadata          │+ CDN│            │+ Cassandra  │    │HDFS + Manifold  │
     │photo binaries    │    │            │per-region   │    │object store     │
     └──┬───┘          └──┬──┘            └───┬─────────┘    │(Meta's S3)      │
        │                 │                   │              └─────────────────┘
     ┌──▼───┐          ┌──▼──┐            ┌───▼──────────┐
     │  S3  │          │ S3  │            │  Kafka +     │
     │binary│          │+CDN │            │  Flink for   │
     │store │          │worker│           │  transcoding │
     └──────┘          │pool │            │  + ML sig    │
                       └─────┘            └──────────────┘

    ↑                  ↑                  ↑                      ↑
   Boring         Chronological       Push-based fanout       ML predicts what you want.
   pull-based     feed + follows.     for followers +         50%+ from accounts you
   JOIN works.    Fanout starts       ML-ranked "explore".    don't follow. DLRM on
                  hurting at 100M.    Regional isolation.     custom silicon (MTIA).

   Chapter 5      Chapters 6+6.5      Chapter 7+7.5           Chapter 8
   walks          walks through       walks through           walks through
   through        the fanout          the DLRM ranker,        Meta's custom hardware,
   L4 pull        vs. pull-merge      TAO graph store,        the "explore" shift, and
   feed in full   trade-off           and multi-region        cost as strategy
                                      Cassandra tier

  Key insight: at Instagram scale, storage is trivial. The PRODUCT is the model
  that ranks 10,000 candidate posts down to the 20 you'll see. That's the whole
  game — and that's why 2022's "50% from accounts you don't follow" shift was
  a fundamental data-model change, not just a UX tweak.

The same 4 tiers as clean architecture diagrams

The ASCII compressed all four tiers side-by-side. Here are the same four scales as clean Mermaid flowcharts — the shapes an interviewer expects on a whiteboard:

L4 · 10K users · pull-based JOIN · $500/mo · 12 weeks:

L5 · 100M users · fanout + hybrid · $50K/mo · 6 months:

L6 · 1B users · global regions + DLRM ranking · $500K/mo · 18 months:

L7 · 2.4B users · custom silicon + explore shift · $5B/yr infra:

Why every 10× breaks the architecture

Look at the four tiers side by side. Notice how the storage tier barely changes — MySQL → Sharded MySQL → Cassandra. That's boring engineering, and it's fine.

What DOES change dramatically:

1. Feed generation. L4 = JOIN across photos + follows. L5 = pushed fanout to follower Redis lists. L6 = candidate generation + ranking. L7 = ML ranking is the entire product.

1. The role of the "who I follow" graph. L4-L5 = the graph IS the feed source. L6-L7 = the graph is one signal among 50 the ranker uses.

1. The compute:cpu ratio flips. L4 = 90% storage/network, 10% CPU. L7 = 60% ML inference on custom silicon, 40% everything else.

The 3 senior insights before we start Chapter 1

1. Storage tiering is table stakes; ranking is the product. At L4 you'll want to overinvest in the DB schema. Resist. Photos → S3 + CDN. Metadata → MySQL. Move on. The interesting problem is the feed ranker.

1. Fanout has a cliff. Push fanout (write photo → write to N follower feeds) works up to ~10K followers per user. Above that (celebrities, verified accounts), you MUST pull-merge at read time. Instagram uses hybrid — push for regular users, pull for celebs. That decision comes in Chapter 6.

1. The 2022 "explore" shift changed the interview answer. If you say "the feed shows posts from accounts I follow, ranked by time", you're describing Instagram 2015. Since 2022, Instagram's own product blog announced that more than half of impressions are from accounts you don't follow — chosen by a deep-learning ranker over a pool of ~10,000 candidates. If you don't mention this shift, the interviewer will probe.

Chapter map for the journey ahead

• Chapter 1 — Requirements (functional + non-functional, per level)
• Chapter 2 — Capacity estimation (2B MAU, 100M photos/day upload, 1T impressions/day feed reads)
• Chapter 3 — API design (upload, feed, engagement, follow)
• Chapter 4 — Data model (photos, follows, likes — and why NOT Postgres for social graph)
• Chapter 4.5 — The follows graph: adjacency list vs graph store vs materialized fanout
• Chapter 5 — L4 MVP: pull-based feed with JOIN. Works up to ~10K users
• Chapter 6 — L5: fanout with Redis feed caches. The cliff at 10K followers
• Chapter 6.5 — Hybrid push/pull for celebrities. TAO-inspired graph store
• Chapter 7 — L6: DLRM ranker. Candidate generation. Regional Cassandra
• Chapter 7.5 — The ranking pipeline: signals, training, serving on custom silicon
• Chapter 8 — L7: the "explore" shift. Multi-region ML. Cost as strategy
• Chapter 9 — Failure modes: ranker outage, follow-graph inconsistency, viral post melting the cache
• Chapter 10 — Trade-off matrix
• Chapter 11 — Interview masterclass: 45-min mock, questions to ask
• Chapter 12 — Defense: the 20 hardest interview questions on Instagram

Ready? Chapter 1 next: understanding what the CTO actually asked for.