Skip to main content
intermediate

Slack

Team messaging with presence, threads, search. Sharded WebSocket, team-affinity routing.

Ch 0The scenario
Journey map
Slack 5 chapters · ~47 min total
Levels:L4 · BeginnerL5 · IntermediateL6 · AdvancedL7 · Senior
2
Design
Data model + the short-code generation problem.
~11 min
The full journey
5 chapters · beginner → super-senior
BeginnerIntermediateAdvancedSenior
Ch 0 · StartClick any chapter to jump →Ch 12 · Defense
Chapter 0
For beginner
5 min read

The scenario

Team chat — the harder cousin of a URL shortener

Your mentor

Same startup as the URL Shortener journey. Same engineer #4. Different Monday.

Your CTO drops by your desk with coffee. "Marketing loves the URL shortener you shipped. Now they want team chat inside the platform. Persistent channels, DMs, threads, presence indicators, file uploads. Like Slack. Ship an MVP in 12 weeks. Enterprise customers want compliance. Oh — and it needs to feel snappy, sub-100ms message delivery."

She walks away.

Now — same discipline, different problem shape. URL Shortener was read-heavy (100 redirects per URL created). Slack is write-heavy with real-time delivery (every keystroke fires a presence event; every message fans out to all channel members instantly). The architecture that worked for URL Shortener will not work here. Understanding why is the whole game.

Here's the shape you're going to design across the next 12 chapters:

The whole journey at a glance

Every 10× in team scale forces a fundamentally different architecture:

text
═══════════ SLACK ARCHITECTURE ACROSS 4 SCALES ═══════════ L4 (10K users) L5 (100K users) L6 (1M users) L7 (1B users) 100 teams 10K teams 100K teams enterprise-federated 12 weeks · $500/mo 6 months · $8K/mo 18 months · $200K/mo ongoing · $10M/yr ┌────────┐ ┌────────┐ ┌───────── CDN ─────┐ ┌─── Own CDN ────┐ │ Web │ │ Web │ │ (files + assets) │ │ files + edge │ │ Mobile │ │ Mobile │ └───┬──┬───────┬────┘ │ message relay │ └───┬────┘ └───┬────┘ │ │ │ └──┬──┬──┬──────┘ │ │ │ │ │ │ │ │ ┌──▼───┐ ┌──▼──┐ ┌──▼──▼──┬────▼──┐ ┌───▼┐ ▼ ▼ │ ALB │ │ ALB │ │us-east │eu-w-1 │ │us │eu ap │ WSS │ │ WSS │ │ ALB WSS│ ALB │ │reg │reg │reg └──┬───┘ └──┬──┘ └───┬────┴───┬───┘ └─┬──┘┬─┘┬┘ │ │ │ │ │ │ │ ┌──▼──────┐ ┌───▼─────────┐ ┌────▼────┐ ┌─▼─────┐ ┌──▼─┐ ▼ ▼ │ 1 WS │ │ 4 WS nodes │ │ 32 WS │ │ 32 WS │ │50 WS + edge│ │ server │ │ sticky-team │ │ nodes │ │ │ │ presence │ │ 100 tm │ │ affinity │ │ team- │ │ │ │ enterprise │ └──┬──────┘ │ routing │ │ affinity│ │ │ │ isolation │ │ └───┬─────────┘ │ shard by│ │ │ └──┬─────────┘ │ │ │ team_id │ │ │ │ │ ┌───▼──┐ └───┬─────┘ └───────┘ ┌──▼─────────┐ │ │Redis │ │ │ Regional │ │ │ pres │ ┌───▼─────────┐ │ Kafka + │ │ │ +typ │ │ Regional │ │ Flink for │ │ └───┬──┘ │ Kafka │ │ fanout + │ │ │ │ fanout │ │ history │ ┌──▼───┐ ┌───▼─┐ │ by team │ └──┬─────────┘ │MySQL │Post │ └───┬─────────┘ │ │(msgs + │gres │ │ ┌──▼──────────┐ │ teams) │+ 2 │ ┌───▼──────┐ │Sharded PG │ │Multi- │repl │ │ Sharded │ │(CockroachDB │ │AZ │icas │ │ MySQL │ │ or per- │ └──────┘ └─────┘ │ 8 shards │ │ enterprise) │ │ by team │ └─────────────┘ └──────────┘ ↑ ↑ ↑ ↑ Boring Add cache + shard Add sharded WS + Federated per works. WS by team. team-affinity + enterprise. MySQL holds all Kafka fanout. Cost = business messages. strategy. Chapter 5 Chapters 6+6.5 Chapter 7+7.5 Chapter 8 walks walks through walks through walks through through team-affinity sharding by team_id, enterprise- L4 in routing + Redis Kafka fanout for federated full presence channel messages architecture Key insight: EVERY message must be delivered to N channel members in real time. That's the difference from URL Shortener - reads and writes are BOTH high volume, and both require sub-100ms latency.

The same 4 tiers as clean architecture diagrams

The ASCII shows the whole story compressed side-by-side. Here are the same four tiers as clean Mermaid flowcharts — the shapes an interviewer expects you to sketch on a whiteboard:

L4 · 10K users · 100 teams · $500/mo · 12 weeks:

flowchart TD W([Web / Mobile clients]) -->|WSS| LB[ALB<br/>WebSocket · sticky<br/>$25/mo] LB --> WS[1 WebSocket server<br/>c5.large · 10K conns<br/>~5K team members] WS --> PG[(MySQL Multi-AZ<br/>messages + teams<br/>1 primary + 1 standby<br/>$150/mo)] classDef n fill:#dbeafe,stroke:#2563eb,color:#1e3a8a class LB,WS,PG n

L5 · 100K users · 10K teams · $8K/mo · 6 months:

flowchart TD W([Web / Mobile]) -->|WSS| LB[ALB<br/>WebSocket sticky] LB --> R{Team-affinity<br/>router<br/>hash by team_id} R --> W1[WS 1] R --> W2[WS 2] R --> W3[WS 3] R --> W4[WS 4] W1 --> RD[(Redis<br/>presence · typing<br/>fast state)] W2 --> RD W3 --> RD W4 --> RD W1 --> PG[(MySQL primary<br/>+ 2 read replicas<br/>messages · teams)] W2 --> PG W3 --> PG W4 --> PG classDef n fill:#dcfce7,stroke:#16a34a,color:#14532d class LB,R,W1,W2,W3,W4,RD,PG n

L6 · 1M users · 100K teams · $200K/mo · 18 months:

flowchart TD W([Web / Mobile]) --> CDN[CDN<br/>files + assets<br/>Cloudflare / CloudFront] CDN --> USE[us-east ALB<br/>WSS] CDN --> EU[eu-west ALB<br/>WSS] USE --> WSU[32 WS nodes<br/>team-affinity<br/>shard by team_id] EU --> WSE[32 WS nodes<br/>team-affinity<br/>shard by team_id] WSU --> KAF[[Regional Kafka<br/>fanout by team_id<br/>topic per shard]] WSE --> KAF KAF --> WSU KAF --> WSE WSU --> SPG[(Sharded MySQL<br/>8 shards by team_id<br/>PgBouncer + Vitess-style)] WSE --> SPG classDef n fill:#f3e8ff,stroke:#7c3aed,color:#4c1d95 class CDN,USE,EU,WSU,WSE,KAF,SPG n

L7 · 1B users · enterprise-federated · $10M/yr:

flowchart TD W([Web / Mobile / Desktop]) --> OCN[Own CDN + Edge relay<br/>files + presence + msg relay<br/>Slack Open Connect analog] OCN --> USR[us regional stack] OCN --> EUR[eu regional stack<br/>GDPR data residency] OCN --> APR[apac regional stack<br/>PIPL compliance] USR --> WSA[50 WS + edge presence<br/>enterprise isolation<br/>per-enterprise cell] EUR --> WSA APR --> WSA WSA --> RK[[Regional Kafka<br/>+ Flink for fanout + history<br/>exactly-once semantics]] RK --> DDB[(Sharded PG or<br/>CockroachDB / Spanner<br/>per-enterprise partition<br/>global consistency where needed)] classDef n fill:#fce7f3,stroke:#db2777,color:#831843 class OCN,USR,EUR,APR,WSA,RK,DDB n

Read the four in order: each tier adds exactly the components its bottleneck forced. L4 → L5 adds sticky team-affinity routing + Redis presence + read replicas because ONE WS node maxes out around 10-20K persistent connections. L5 → L6 adds CDN + multi-region + Kafka fanout + MySQL sharding because 100K → 1M users means a single Kafka topic (or single PG primary) becomes the bottleneck. L6 → L7 adds an owned CDN + per-region compliance stacks + per-enterprise data isolation because at 1B users the choice becomes strategy (build vs buy CDN, own network vs commercial) and regulatory (data residency per region).

Read this chart before diving in. Notice the differences from URL Shortener:

  • WebSockets everywhere. Slack is a persistent-connection product, not a request-response product. A single user holds an open connection for hours. That changes load balancer choices, changes memory profile per server, changes failover semantics.
  • Team-affinity routing. All members of Team A should ideally land on the same WebSocket server so message fanout is a single-server broadcast, not a cross-server RPC. That's a hard routing problem.
  • Sharding by team, not by user. Team is the unit of everything: messages, channels, presence, permissions. A user in 3 teams belongs to 3 shards.
  • Message fanout dominates the cost. Every message posted to a 50-person channel produces 50 delivery events. At scale, fanout math dominates architecture.

What's different from URL Shortener

DimensionURL ShortenerSlack
Read/write ratio100:1 (very read-heavy)~1:1 (balanced; write triggers N reads via fanout)
Latency budget<100ms redirect OK<100ms message delivery — much harder
Connection modelStateless HTTPStateful WebSocket (persistent connections)
Sharding keyshort_code hashteam_id (all data-per-team lives together)
FanoutNot reallyEvery message fans out to N channel members
ComplianceBasic HTTPS + GDPREnterprise: SOC 2 + HIPAA + data residency per team
First-party revenueAds / freemium tiersPer-user monthly subscriptions ($8-16/user/mo)

The takeaways apply to any real-time messaging system: Slack, Discord, WhatsApp, Teams, iMessage. If you understand these 4 scale tiers for Slack, you understand ~80% of the design surface for any real-time messaging system.

Where the rest of this journey goes

Chapters 1-4 — same discipline as URL Shortener:
- Ch 1 Requirements — the 6 clarifying questions specific to messaging
- Ch 2 Estimation — the fanout math: how much traffic does a 50-person channel actually create?
- Ch 3 API — REST endpoints for CRUD + WebSocket protocol for real-time
- Ch 4 Data model — messages, channels, teams, memberships (way more entities than URL Shortener)

Chapters 5-8 — the scale evolution shown above.

Chapters 9-12 — failures, trade-offs, masterclass, defense (mirror of URL Shortener structure).

Deliberate pace note. This journey follows the URL Shortener template. If you completed URL Shortener, you know the shape: each chapter builds on the previous. Don't skip ahead. The trade-offs at L6 only make sense once you've felt the pain of L5.

If you haven't done URL Shortener yet, do it first — it's the simpler shape and gives you the vocabulary for this one.

Chapters queued (session note)

Chapters 1-12 of this journey are queued for follow-on authoring sessions. This Chapter 0 exists so you can see the whole arc and decide if this journey is right for your current preparation. Come back as the remaining chapters ship.

References for what's coming:

  • Slack engineering blog: slack.engineering — real production architecture and scale content directly from the team
  • Nystrom & Warnock (2019) — "How Slack Built Shared Channels": slack.engineering/how-slack-built-shared-channels — reference for cross-workspace routing at L7
  • Slack's Nebula overlay network: github.com/slackhq/nebula — how Slack routes traffic between data centers
  • Discord engineering blog: discord.com/blog/engineering — parallel product, published architecture at 100M+ concurrent connections
  • WhatsApp: 900M users on ~50 servers (2015 talk by Rick Reed / Erlang factory conference) — how radical simplicity + Erlang scaled a messaging platform
  • Chapter 5.5 of URL Shortener journey — database decision framework applies here too (spoiler: MySQL by default, Cassandra at L7 for write-heavy fanout)
Key takeaway

Slack is a persistent-connection real-time messaging platform. Compared to URL Shortener: reads AND writes are both high-volume, WebSockets replace HTTP, team-affinity routing replaces simple load balancing, and message fanout math dominates architecture. Every 10x in teams forces new architecture at L4/L5/L6/L7 — same evolution discipline as URL Shortener.

You should now be able to answer
  • What makes real-time messaging fundamentally different from URL shortening architecturally?
  • How does the architecture evolve across L4/L5/L6/L7 for Slack?
  • Why is team-affinity routing critical at L6+?
  • What's the difference in read/write ratio between URL Shortener and Slack?
  • Why do WebSockets change every architectural decision downstream?
Coming next

Chapter 1 (upcoming): the 6 clarifying questions for a messaging platform. Persistence duration? Search? Compliance? Cross-team messaging? Each question forces different architecture. Chapter 1 hasn't been authored yet in this session — come back as follow-on chapters ship.

Chapter 1
For beginner
10 min read

Ask before you design

6 clarifying questions for a real-time messaging platform

Your mentor

Same discipline as URL Shortener Ch 1 — ask the questions FIRST, design SECOND. But because Slack is fundamentally different (persistent connections, real-time, presence, files, compliance), the 6 questions are different too.

I'm going to walk you through the 6 clarifying questions I would ask if I were interviewing for a Staff role at any FAANG right now. Each comes with the "why" — how the answer changes the architecture. Learn the "why" and you'll spot the same pattern in any real-time system.

Q1. What's the message-delivery guarantee — at-most-once, at-least-once, or exactly-once?

This is THE question. Get it wrong and every downstream decision is off.

At-most-once — send a message; if the recipient's WebSocket is disconnected, it's LOST. Simplest architecture: fire-and-forget WebSocket write. Used by: presence indicators ("Alice is typing"), volatile UI notifications.

At-least-once — guarantee delivery; recipient may see the same message twice on reconnect. Requires: message-ID uniqueness on client side, idempotency on backend. Used by: Slack, WhatsApp, Discord — the industry standard.

Exactly-once — guarantee delivery AND no duplicates. Requires: distributed consensus (Paxos/Raft), 2-phase commit, or client-side ACK protocol. Used by: financial transactions, NOT chat. Adding this to a chat app 5×'s the infra cost for no user-visible benefit.

The interview soundbite: "I'd design at-least-once with client-side dedup by message_id — the industry standard. Exactly-once is a red herring for chat."

Q2. What's the persistence duration — session-only, days, or forever?

Determines your storage tier + your compliance headache.

DurationArchitecture implicationCost
Session-onlyIn-memory only. Server restart = messages gone. Trivial.~$0/mo
30 daysHot storage (MySQL). Fast search.~$3K/mo at 10M msgs/day
1 yearHot + warm tier (MySQL + S3). Search across tiers.~$8K/mo
Forever (Slack default)Hot + warm + cold + compliance archive. Full-text search across all.~$50K/mo
Custom retention (enterprise)Per-workspace policy. Complex.+$20K/mo eng time

Slack's answer: forever, with enterprise-configurable retention. That's why they run petabyte-scale Solr + Elasticsearch for search.

Q3. What's the maximum team size?

Team size drives the fanout math. Every message fires N recipient events where N = channel member count.

  • 50 members (small startup team) — fanout is trivial. Single WebSocket server can handle 1000+ teams.
  • 500 members (mid-size company channel) — 10K events per message on a busy channel. Needs regional routing.
  • 10K members ("all-hands" or #general in a giant company) — 100K events per message. Needs sharding + fan-out via Kafka.
  • 100K members (Slack Connect, Discord public server) — 1M events per message. Needs a message relay tier + probabilistic delivery for typing indicators.

Slack's answer: 500K max per channel (Enterprise Grid), but 90% of channels are <50 members. Design for the common case, escape hatch for the rare huge channel.

Q4. Does search need to be real-time or eventual?

Search complexity multiplies with real-time requirements.

  • Eventual (5-min lag) — batch indexer runs every 5 min against MySQL. Simple. Cheap. Users don't notice until they DO.
  • Near-real-time (5-sec lag) — Kafka → Elasticsearch pipeline. 10× more complex, 5× cost.
  • True real-time — write to MySQL AND Elasticsearch in same request. Doubles write latency. Elasticsearch outages block writes.

Slack picked eventual (~5-30 sec). It's the sane default. If your interviewer pushes for real-time search, ask "what's the user-visible SLO?" — 5 seconds is almost always fine.

Q5. Files — how big and how many per user?

File uploads dominate cost at Slack scale.

  • Text-only — no upload path. Skip S3, skip antivirus, skip CDN for files. Cheap.
  • Images + docs (max 10 MB) — S3 + CloudFront. Simple. $0.02/GB storage.
  • Video calls / large files (max 1 GB) — needs pre-signed URLs, multipart upload, virus scan, transcoding pipeline. $0.30/GB egress.
  • Slack's answer: 5 GB per file, unlimited uploads, kept forever. Files are ~40% of Slack's total infra cost.

Q6. What compliance regime — none, SOC 2, HIPAA, or FedRAMP?

Compliance changes your ENTIRE architecture, not just a checkbox.

  • None (consumer app) — pick any cloud, any region, any encryption. Move fast.
  • SOC 2 — audit trail on every access, encryption at rest, encryption in transit, SSO required. +30% eng time. Table-stakes for enterprise sales.
  • HIPAA (US healthcare) — BAA with cloud provider, per-workspace encryption keys, dedicated infrastructure available. Slack HIPAA-eligible plan starts at $30K/yr per customer.
  • FedRAMP (US government) — AWS GovCloud only, US citizen operators only, security clearances, air-gapped networks. Slack FedRAMP took 3 YEARS to certify. Six-figure floor per customer.

The interview soundbite: "I'd default to SOC 2 as table-stakes; HIPAA and FedRAMP are separate SKUs with dedicated cells — same code, different deployment."


The requirements summary

For our design, here's what we're going to assume (matches Slack Free/Standard tier — most common):

RequirementAnswer
Delivery guaranteeAt-least-once with client-side dedup
PersistenceHot 30 days, warm indefinitely, cold in enterprise archive
Max team10K members typical, 500K max Enterprise
Search latencyNear-real-time (5-30 sec)
Files5 GB per, kept forever, virus-scanned
ComplianceSOC 2 default, HIPAA + FedRAMP as separate SKUs

These 6 answers unlock the next 11 chapters. Chapter 2 (estimation) turns these requirements into RPS + storage math. Chapter 3 (API) codifies the delivery guarantee + search semantics into REST + WebSocket contracts. Chapters 5-9 progressively evolve the architecture across L4-L7.

Newbie insight: the 6 questions aren't universal — they're specific to real-time messaging. When you interview for a different problem (feed generation, video streaming, ride matching), you have to derive the 6 questions from scratch based on the problem's SHAPE. The skill is knowing to ASK, not knowing which questions.

Clarifying questions

    Functional

      Non-functional

        Key takeaway

        6 clarifying questions unlock every architectural decision downstream: (1) delivery guarantee, (2) persistence duration, (3) team size, (4) search latency, (5) file size/count, (6) compliance regime. Slack's answers: at-least-once, forever, 500K max, ~5s search, 5GB files, SOC 2 default. Every downstream chapter derives from these.

        You should now be able to answer
        • Why is at-least-once the industry standard for chat?
        • Why does exactly-once delivery add 5× cost with no user-visible benefit?
        • How does team size affect fanout architecture?
        • Why is real-time search rarely worth its complexity?
        • What are the four compliance regimes and how do they change architecture?
        Coming next

        Chapter 2 (upcoming): estimation. Now that we have the requirements, we do the math. WebSocket connections × messages/sec × storage per message = the numbers that drive every architecture decision. Also: how do you estimate concurrent presence events? Chapter 2 hasn't been authored yet — come back as it ships.

        Chapter 2
        For beginner
        9 min read

        Do the math

        Concurrent WebSockets, messages/sec, storage, presence events

        Your mentor

        Same funnel discipline as URL Shortener Ch 2. Different quantities.

        Slack estimation has FIVE funnels, not one:
        1. Concurrent WebSocket connections — the big number that scares junior engineers
        2. Messages sent per second — what most people think Slack is about
        3. Presence events per second — the hidden 100× multiplier
        4. File bytes per day — the cost driver
        5. Storage across tiers — the compliance driver

        Get each funnel right and the architecture derives itself.

        Assumptions we'll use

        Let me anchor to Slack's published-ish numbers so this feels real. From their 2019 engineering blog + IPO S-1:
        - ~12 million daily active users (DAU)
        - Average DAU is connected ~10 hours per day
        - Average DAU sends ~50 messages/day, receives ~500
        - 30% of DAU are on mobile (~lower connection quality)

        We'll design for 20M DAU to give ourselves 65% growth headroom.

        Funnel 1 — Concurrent WebSocket connections

        This is the number that dictates your entire connection tier.

        ```
        20M DAU × (10 hrs connected / 24 hrs) = 8.3M concurrent connections (average)
        × 1.5 peak factor
        = 12.5M peak concurrent
        ```

        Why this matters: each WebSocket connection holds a TCP socket + a small amount of server memory (~30KB with Erlang, ~50KB with Node.js, ~500KB with naive Java).

        Boxes needed at 100K connections per WS server (Node.js production benchmark): 12.5M / 100K = 125 WebSocket servers minimum, plus HA headroom → ~180 boxes. Erlang at 2M connections/box gets you to ~6 boxes, which is why WhatsApp famously ran 900M users on ~50 servers.

        Funnel 2 — Messages sent per second

        ```
        20M DAU × 50 messages/day × 1/86400 = 11,574 messages/sec average
        × 3× peak factor (12pm-2pm)
        = ~35K messages/sec peak
        ```

        That's the write rate. Now the FANOUT:

        ```
        35K msgs/sec × ~10 recipients per message (avg channel size) = 350K delivery events/sec
        × 3× typing/read/reaction ratio
        = ~1M realtime events/sec
        ```

        Every message is amplified 30× by the time it hits every recipient's screen with all its reactions and read receipts. This is why fanout dominates the architecture.

        Funnel 3 — Presence events per second (the hidden multiplier)

        Presence is the silent killer of naive designs.

        • Users typing → send "X is typing" event every 500ms while active
        • Users going online/offline → 4-6 events per DAU per day
        • Users reading channels → mark-as-read event per channel per user

        ```
        20M DAU × 100 typing events/day = 2B events/day
        20M DAU × 5 status changes/day = 100M events/day
        20M DAU × 20 channel-reads/day = 400M events/day
        Total: ~2.5B presence events/day = ~29K presence events/sec (average)
        × 3× peak = ~90K/sec
        ```

        That's ~3× your message rate. And presence is USELESS if it's not real-time. This is where you need Redis + probabilistic dropping.

        Funnel 4 — File bytes per day

        The cost bomb. Slack's files are 40% of infra cost.

        ```
        20M DAU × 20% upload files daily × avg 2 MB per file = 8 TB/day uploaded
        × 365 days = 2.9 PB/year of NEW files
        + kept forever (compliance)
        = 30 PB storage after 10 years
        ```

        At S3 Standard ($0.023/GB/mo): $690K/mo just for file storage at 10-year mark. That's the number that drives you to tiered storage (S3 IA, Glacier).

        Funnel 5 — Storage across tiers

        Text messages (small), files (big), search indexes (bigger).

        DataBytesVolume/day1-year total
        Text message (avg)200 B35K/sec × 86400 = 3B msgs/day → 600 GB/day220 TB/year
        File uploads2 MB avg8 TB/day2.9 PB/year
        Search index~500 B / msg1.5 TB/day550 TB/year
        Presence events (not stored)000
        Total hot storage per year~10 TB/day3.7 PB/year

        Interview soundbite: "At 20M DAU we're looking at 12.5M peak WebSockets, 35K msgs/sec with 30× fanout amplification, 90K presence events/sec, 8 TB/day file uploads. The presence tier alone justifies Redis Cluster; file storage tiering is where the CFO gets involved."

        From estimation to architecture (preview)

        The 5 funnels above force each tier of the architecture:

        FunnelImplies
        12.5M concurrent WS~180 WS servers (Node.js) or ~10 (Erlang). Team-affinity routing.
        35K msgs/secMySQL can barely handle. Need Cassandra or Kafka-backed fanout at L6+.
        90K presence/secRedis Cluster. Cannot go through MySQL.
        8 TB/day filesTiered S3 (Standard → IA → Glacier) with lifecycle policies.
        3.7 PB/year hotElasticsearch cluster of ~50 nodes.

        Cost bottom line (L6 at 20M DAU)

        • Connection tier (180 WS servers × m5.large): ~$12K/mo
        • Message tier (Kafka + Cassandra): ~$40K/mo
        • Presence (Redis Cluster ×8): ~$3K/mo
        • File storage (year 1): ~$50K/mo growing to ~$700K/mo in 10 years
        • Search (Elasticsearch × 50 c5.2xlarge): ~$25K/mo
        • Bandwidth egress: ~$60K/mo
        • Total year 1: ~$190K/mo (~$2.3M/year infra)

        At 12M paying DAU × $10/mo (average) = $120M ARR. Infra is 2% of revenue. Sustainable.

        Newbie insight: the 5-funnel model is universal for messaging systems. WhatsApp, Discord, Teams, Signal all do this exact calculation. What differs is which funnel dominates: WhatsApp = messages, Discord = presence, Slack = files. Which funnel dominates decides your engineering priorities.

        Key takeaway

        Slack has 5 estimation funnels: concurrent WebSockets (12.5M peak), messages/sec (35K with 30× fanout), presence events/sec (90K), file bytes/day (8 TB), tiered storage (3.7 PB/year). Each funnel forces a specific tier of the architecture. Files dominate cost; presence dominates event volume.

        You should now be able to answer
        • How do you derive concurrent WebSocket connections from DAU?
        • Why is the 30× fanout amplification the dominant write-side cost?
        • Why do presence events exceed message events by 3×?
        • What's the tiered-storage cost for keeping files forever?
        • Why do files dominate Slack's infra cost?
        Concept deep-dives referenced in this chapter

        Every concept below has its own interactive, animated page in the Learning Tracks section. Read them any time you want to go deeper than the mentor prose above — they're the reusable foundation this chapter is built on.

        Coming next

        Chapter 3 (upcoming): the API + WebSocket protocol design. REST for auth + history, WebSocket for real-time. The protocol contract you pick locks in your delivery semantics forever. Chapter 3 hasn't shipped yet — coming soon.

        Chapter 3
        For beginner
        12 min read

        Design the contract

        REST + WebSocket — the two-protocol architecture

        Your mentor

        Slack has two protocols working together. Get this split right and everything else falls into place. Get it wrong and you'll be paying WebSocket costs for things HTTP would do better.

        The two-protocol rule

        REST/HTTP for:
        - Authentication + session establishment
        - Fetching history (messages before you connected)
        - Uploading files (multipart, resumable)
        - Search queries
        - Any request that has an answer and returns

        WebSocket for:
        - Receiving real-time messages
        - Sending new messages
        - Presence + typing indicators
        - Read receipts
        - Reactions

        The interview soundbite: "HTTP is for state-transfer; WebSocket is for state-changes. If it can be answered with a single response, use REST. If it's an event stream, use WebSocket."

        The REST API surface (6 endpoints, that's it)

        Slack's public API has hundreds of endpoints. The MVP needs 6:

        1. POST /v1/auth/login Body: { email, password | oauth_token } Returns: { user_id, session_token, ws_url, initial_state }

        The ws_url is the client's next stop. initial_state contains the user's team list + unread counts so the client can render before opening the WebSocket. This is the "TL;DR everything I need to know" endpoint.

        2. GET /v1/channels/{channel_id}/history?before={message_id}&limit=50 Fetches historical messages. Cursor-based pagination via before message_id.

        Why cursor and not offset? Because messages get inserted concurrently — an offset-based query at 50-99 will miss messages inserted during your scroll. Cursor-based (before/after specific message_id) is idempotent even under concurrent writes.

        3. POST /v1/channels/{channel_id}/messages Body: { client_message_id, text, thread_parent_id? } Returns: { server_message_id, timestamp }

        The critical detail: the client generates client_message_id (UUID) at compose time. Server accepts idempotently — if two requests arrive with same client_message_id, server returns the same server_message_id both times. This is how the client survives retry-storms on flaky mobile networks WITHOUT sending duplicate messages.

        4. POST /v1/files/upload Multipart upload. Returns pre-signed S3 URL for large files (>10 MB) so the browser uploads directly to S3, not through our API tier.

        5. GET /v1/search?q={query}&channel={channel_id}&limit=20 Full-text search backed by Elasticsearch. Returns highlighted snippets.

        6. GET /v1/users/{user_id}/presence Explicit presence check. Rarely used — presence usually comes via WebSocket. This endpoint exists for "is X online?" polls in unusual UI states.

        That's it. 6 endpoints for the MVP. Every other endpoint in Slack's real API is a specialization of one of these 6.

        The WebSocket protocol

        The WebSocket carries an event stream — a bidirectional flow of typed messages. Slack's actual production protocol (documented at api.slack.com/rtm) is a good model.

        Frame format

        Every WebSocket frame is a JSON blob with a type field:

        json
        { "type": "message", "channel_id": "C123", "user_id": "U456", "text": "hi", "ts": "1234567890.001", "client_message_id": "uuid" } { "type": "presence", "user_id": "U456", "status": "online" } { "type": "typing", "channel_id": "C123", "user_id": "U456" } { "type": "reaction_added", "message_ts": "1234567890.001", "reaction": "thumbsup", "user_id": "U456" } { "type": "ack", "client_message_id": "uuid", "server_message_id": "M789" } { "type": "ping", "id": 42 } { "type": "pong", "id": 42 }

        The 4-step message send

        Every outgoing message flows through 4 states on the client:

        1. compose — user hits Enter. Client generates client_message_id, adds message to UI as "sending" (grey).
        2. send — client emits WebSocket { type: "message", client_message_id, text, channel_id } frame.
        3. ack — server responds { type: "ack", client_message_id, server_message_id, timestamp }. Client updates the local message record with server_message_id + turns UI from grey to normal.
        4. broadcast — server independently pushes { type: "message", server_message_id, ... } to ALL other connected users in that channel. Their clients render new message.

        Why the ack/broadcast separation? The sending client already has the message in its UI from step 1. It just needs the server_message_id and timestamp. Other clients need the whole message. Separating these two events halves the bytes on the wire for busy channels.

        Reconnect + catch-up

        WebSockets are lossy — flaky WiFi, mobile network handoff, laptop lid closing. The client MUST expect disconnect and reconnect. The protocol handles this:

        1. Client detects disconnect (missing ping/pong).
        2. Client stores last_ack_timestamp (server timestamp of last message it received).
        3. Client reconnects via new WebSocket.
        4. First frame client sends: { type: "reconnect", last_ack_timestamp: "1234567890.500" }
        5. Server replays all messages between last_ack_timestamp and now — as normal broadcast frames.
        6. Client dedups by server_message_id.

        This is why at-least-once + client-side dedup by server_message_id is the industry standard. Exactly-once would require distributed consensus over reconnects — 10× more complexity for no user-visible benefit.

        Heartbeat + backpressure

        Every 30 seconds the client sends { type: "ping", id: N }. Server responds with { type: "pong", id: N }. Missing 2 pongs → client tears down + reconnects.

        When the server's send buffer for a client fills (slow client on bad network), server MUST drop the client. Slack's approach: drop the connection cleanly with a "reconnect_url" hint. Client reconnects to a less-loaded WebSocket server.

        API decisions summary (with rationale)

        DecisionChoiceWhy
        Sync or async writesSync ack, async broadcastSender needs immediate confirmation; recipients can tolerate a few ms
        PaginationCursor (by message_id)Idempotent under concurrent writes
        Idempotencyclient_message_id UUIDClient-generated; server dedups; survives retries
        Real-time transportWebSocket, not SSEBidirectional (client sends messages back)
        FallbackHTTP long-pollFor corporate proxies that block WS on port 443
        Rate limitPer WS + per REST endpointWS limits are per-user per-minute; REST per-IP
        Auth on WSQuery param or first-frame authCookies don't work reliably in WS; use ?token=jwt in URL
        VersioningSec-WebSocket-Protocol: slack.v2Standard WS subprotocol negotiation

        The interview soundbite (60-sec version)

        "REST for state-transfer, WebSocket for state-changes. 6 REST endpoints: auth, history, send-message, upload, search, presence. WebSocket protocol with typed frames — message, presence, typing, ack, ping, pong. Ack + broadcast separation reduces bandwidth on busy channels. Client generates message_id for idempotency; server dedups. Reconnect with last_ack_timestamp for catch-up. At-least-once delivery with client-side dedup is the industry standard — exactly-once is a red herring."

        That's the whole contract in 90 seconds. Now Chapter 4 designs the data model that backs it.

        MethodPathPurpose
        Key takeaway

        Slack's contract is REST + WebSocket. REST = state-transfer (auth, history, send, upload, search, presence). WebSocket = state-changes (message, presence, typing, ack). Client-generated message_id makes retries idempotent. Reconnect with last_ack_timestamp replays missed messages. At-least-once with client-side dedup = industry standard.

        You should now be able to answer
        • Why split REST + WebSocket instead of doing everything over one protocol?
        • Why client-generated message_id for idempotency?
        • How does the ack/broadcast separation halve WebSocket bandwidth?
        • Why cursor-based pagination instead of offset?
        • How does the client survive a WebSocket reconnect without losing messages?
        Concept deep-dives referenced in this chapter

        Every concept below has its own interactive, animated page in the Learning Tracks section. Read them any time you want to go deeper than the mentor prose above — they're the reusable foundation this chapter is built on.

        Coming next

        Chapter 4 (upcoming): the data model. Given the contract, what tables (or key-value spaces) do we need? channels, messages, users, threads, reactions — each with a schema decision that ties to a scale trade-off. Chapter 4 hasn't shipped yet — coming soon.

        Chapter 4
        For beginner
        11 min read

        Model the data

        5 tables that run Slack — with the schema trade-offs that matter

        Your mentor

        The data model is where amateurs make Slack architecturally impossible. There are 5 core entities. Get the relationships right and Chapters 5-9 write themselves. Get them wrong and you spend 18 months migrating.

        The 5 core entities

        ```
        Team ─┬─ Users (members)
        ├─ Channels ──┬─ Messages ──┬─ Reactions
        │ │ ├─ Threads (as messages with thread_parent_id)
        │ │ └─ Files (message attachments)
        │ └─ Membership (users in channel)
        └─ Presence (user online/offline)
        ```

        Table 1: teams

        The Slack "workspace" — the top-level scoping entity.

        sql
        CREATE TABLE teams ( team_id UUID PRIMARY KEY, name TEXT NOT NULL, domain TEXT UNIQUE NOT NULL, -- acme.slack.com plan TEXT NOT NULL, -- free / standard / plus / enterprise created_at TIMESTAMPTZ DEFAULT NOW(), region TEXT NOT NULL, -- us-east-1, eu-west-1, ap-south-1 data_residency TEXT DEFAULT 'flexible', -- flexible / us-only / eu-only (GDPR) retention_days INT DEFAULT NULL, -- NULL = forever encryption_key_id UUID -- customer-managed key for HIPAA ); CREATE INDEX idx_teams_domain ON teams (domain);

        The one schema decision that matters most: region is a physical column, not a computed value. This is the sharding key at L6+. Every downstream table has a team_id FK and is sharded by the team's region. Cross-region queries are RARE and require going through a coordinator.

        Table 2: users

        sql
        CREATE TABLE users ( user_id UUID PRIMARY KEY, team_id UUID NOT NULL REFERENCES teams, email TEXT NOT NULL, display_name TEXT NOT NULL, status TEXT DEFAULT 'active', -- active / deactivated / deleted role TEXT DEFAULT 'member', -- owner / admin / member / guest created_at TIMESTAMPTZ DEFAULT NOW(), last_seen_at TIMESTAMPTZ, UNIQUE (team_id, email) ); CREATE INDEX idx_users_team ON users (team_id);

        Note: user_id is globally unique, not per-team. Why? Because Slack Connect (cross-team DMs) needs a single global user identity. If user_id were team-scoped, cross-team messaging would need a translation layer forever.

        Slack's actual gotcha: a user can be in multiple workspaces with the same email. That means the "primary key" for identity is (email × workspace_domain), NOT email alone. Slack Connect resolves this by upgrading email → global user_id at first sign-in and mapping the two workspaces to that global identity.

        Table 3: channels

        sql
        CREATE TABLE channels ( channel_id UUID PRIMARY KEY, team_id UUID NOT NULL REFERENCES teams, name TEXT NOT NULL, channel_type TEXT NOT NULL, -- public / private / dm / mpdm topic TEXT, purpose TEXT, created_at TIMESTAMPTZ DEFAULT NOW(), created_by UUID REFERENCES users, archived BOOLEAN DEFAULT FALSE, is_shared BOOLEAN DEFAULT FALSE, -- Slack Connect flag UNIQUE (team_id, name) ); CREATE INDEX idx_channels_team ON channels (team_id) WHERE NOT archived;

        Design decision — one table or four?

        Public channels, private channels, DMs, group DMs are all "channels" architecturally — they hold messages, have members, get searched. Slack uses ONE table with a discriminator column. Trade-off: some code paths need to filter by channel_type; benefit: message-write path is identical for all four, simplifying the WebSocket protocol.

        Table 4: memberships (the many-to-many)

        sql
        CREATE TABLE memberships ( channel_id UUID NOT NULL REFERENCES channels, user_id UUID NOT NULL REFERENCES users, joined_at TIMESTAMPTZ DEFAULT NOW(), last_read_ts TIMESTAMPTZ, -- for unread badges notification_pref TEXT DEFAULT 'all', -- all / mentions / muted PRIMARY KEY (channel_id, user_id) ); CREATE INDEX idx_membership_user ON memberships (user_id); -- "channels I'm in" CREATE INDEX idx_membership_channel ON memberships (channel_id); -- "who's in this channel"

        Two indexes on the same table — is that OK? For a read-heavy table like this, YES. Membership is read on every message send (to fan out) AND on every user login (to load channels). Missing either index is a 100× slowdown.

        last_read_ts is a hot column — it updates on EVERY channel view. At scale you split it out into a channel_reads table or into Redis. But at L4/L5 keeping it inline saves a join.

        Table 5: messages — where scale lives

        sql
        CREATE TABLE messages ( message_id UUID PRIMARY KEY, channel_id UUID NOT NULL REFERENCES channels, team_id UUID NOT NULL, -- denormalized for sharding user_id UUID NOT NULL REFERENCES users, client_message_id UUID NOT NULL, -- for idempotency dedup (Ch 3) text TEXT NOT NULL, ts TIMESTAMPTZ NOT NULL DEFAULT NOW(), edited_at TIMESTAMPTZ, thread_parent_id UUID REFERENCES messages, -- NULL = top-level, non-NULL = reply attachments_json JSONB, -- files, links, formatted blocks UNIQUE (channel_id, client_message_id) ); -- The critical index: messages by channel, ordered by time CREATE INDEX idx_messages_channel_ts ON messages (channel_id, ts DESC); -- Thread-lookup index CREATE INDEX idx_messages_thread ON messages (thread_parent_id) WHERE thread_parent_id IS NOT NULL;

        Every decision here matters:

        1. `team_id` denormalized — the channel already has team_id. Why duplicate it in messages? Because at L6+ we shard by team_id, and joining channels every query is a shard-hop away. Denormalization = fewer shard hops.
        1. `client_message_id` UNIQUE per channel — enforces idempotency at the DB level. Even if the app tier's dedup breaks, the DB refuses duplicate inserts.
        1. `thread_parent_id` self-reference — threads are messages. Slack's schema treats a threaded reply as a message with a parent pointer. UI renders them differently, but the storage layer treats them identically.
        1. `attachments_json` denormalized — files are looked up as-attached-to-messages 100× more often than as-standalone-files. JSON blob avoids the join.
        1. The (channel_id, ts DESC) index is the WORKHORSE — it powers channel history load, unread scanning, search-in-channel. Get this wrong and every user complains that Slack is slow.

        The reactions table (bonus)

        sql
        CREATE TABLE reactions ( message_id UUID NOT NULL REFERENCES messages, user_id UUID NOT NULL REFERENCES users, reaction TEXT NOT NULL, -- ":thumbsup:", ":heart:", custom ts TIMESTAMPTZ DEFAULT NOW(), PRIMARY KEY (message_id, user_id, reaction) ); CREATE INDEX idx_reactions_message ON reactions (message_id);

        Denormalization opportunity: at L6+ you cache reaction counts on the message itself (reactions_summary_json) so the fan-out doesn't need a JOIN for every WebSocket broadcast.

        Scale trade-offs across the 5 tables

        ScaleStorage strategy
        L4 (100 teams)All 5 tables in single MySQL. Cross-table JOINs work.
        L5 (10K teams)Same MySQL. Add read replicas for message queries.
        L6 (100K teams)SHARD by team_id. Each shard has all 5 tables scoped to a set of teams. Cross-team = coordinator layer.
        L7 (1M+ teams)Regional shards + tiered storage. Old messages → Cassandra or S3-Parquet. Hot messages in MySQL. Search index in Elasticsearch.

        The interview soundbite

        "5 core entities: teams, users, channels, memberships, messages. Users are globally unique for Slack Connect. Channels are one table with a type discriminator. Memberships is the M:N junction with two indexes because both directions get queried on every message. Messages denormalize team_id for shard routing, threads are self-referential, reactions have a lightweight (message_id, user_id, reaction) primary key. At L6+ shard by team_id; hot vs. cold storage tier at L7."

        Newbie insight: the temptation is to normalize everything (3NF or higher) at design time and denormalize later "when you need to." Wrong. You'll never migrate the schema at scale — DDL locks tables for hours. Design for denormalization from Day 1 in the columns you know will be hot. team_id in messages is the canonical example: never joinable in production because you'd hop shards.

        Key takeaway

        5 tables run Slack: teams, users, channels, memberships, messages. Global user_id (for Slack Connect), unified channel table with type discriminator, memberships as bi-directional-indexed M:N, messages denormalize team_id for sharding + client_message_id for idempotency + thread_parent_id for threads. Sharding key at L6+ is team_id. Design for denormalization on Day 1.

        You should now be able to answer
        • What are the 5 core entities of a Slack-like system?
        • Why is user_id globally unique instead of team-scoped?
        • Why unify public/private/DM channels into one table?
        • Why does messages.team_id exist even though channels already has team_id?
        • How does the schema evolve as you shard at L6+?
        Coming next

        Chapter 5 (upcoming): the MVP architecture. Now that we have the contract (Ch 3) and the schema (Ch 4), we build the smallest working version — 1 WebSocket server, 1 MySQL, 100 teams. Everything Chapters 6-9 do is an incremental evolution of this MVP. Chapter 5 hasn't shipped yet — coming soon.

        Components used in this design

        Study each component's deep dive to understand it in isolation.

        Patterns applied in this design

        Study each pattern's deep dive for the recurring solution logic.

        Explore next — related systems

        If you enjoyed this problem, these share similar patterns or challenges.