Redis
An in-memory key-value store used for caching, pub/sub, rate limiting, distributed locks, and simple queues.
Why it exists
RAM is 100,000× faster than disk. When your workload is hot-set-bound (a small fraction of your data is 99% of the traffic), moving that hot set into memory drops your read latency from milliseconds to microseconds and takes load off your primary database. Redis is the industry's default in-memory cache — but it's also a Swiss army knife for anything that needs sub-millisecond coordination.
How it works
Redis is a single-threaded, in-memory key-value store with pluggable data types (strings, lists, sets, sorted sets, hashes, streams, HyperLogLog, bitmaps, geospatial). Because it's single-threaded, individual commands are atomic — huge win for correctness. Persistence is optional (RDB snapshots + AOF logs). Replication is leader-follower with async replication by default. Redis Cluster shards keys across nodes using hash slots (16384 slots hashed via CRC16).
Scaling characteristics
A single Redis node on modern hardware handles ~100K QPS at sub-millisecond latency (~200 KB payloads). Redis Cluster scales linearly to millions of QPS across nodes. Memory is the ceiling: at ~100 bytes per key, 100M keys need ~10 GB. Cost: memory is the expensive resource; the CPU is barely used.
When to use it
- Cache-aside for hot reads from a slower store
- Session storage (fast, TTL-based expiry, no need for durability)
- Rate limiting via INCR + EXPIRE
- Distributed locks (Redlock — with caveats)
- Pub/sub for lightweight event fan-out
- Leaderboards via sorted sets
- Real-time counters and analytics preaggregates
When NOT to use it
- Primary storage for anything you cannot afford to lose (Redis is memory-first; even AOF has a lag window)
- Very large values (Redis is tuned for small hot keys; >100KB values hurt latency)
- Complex queries — Redis doesn't do JOINs or ad-hoc queries
Failure modes
- Node down → cache miss stampede on the database. Mitigate with circuit breakers + load shedding.
- Hot key on a single node → CPU pegged even though the cluster is fine. Mitigate with local in-process caching or key splitting.
- Big-key writes block the single-threaded server. Mitigate with SCAN + chunked writes.
- Persistence disabled + power loss = full cache loss on restart. Mitigate with warm-up policy or AOF.
- Redis Cluster resharding is disruptive on massive datasets — plan capacity in advance.
Alternatives
- Memcached — simpler, multi-threaded, no persistence, no advanced data types. Great for pure hot-set caching.
- In-process LRU (Guava, Caffeine, node-lru-cache) — zero-hop but shared across replicas; adequate for smaller scale
- DynamoDB DAX / ElastiCache — managed alternatives with less operational overhead
- Local NVMe SSD with mmap — for cache sets larger than RAM but too small for a cluster
Interview questions
- You have a hot key that saturates one Redis shard — how do you handle it?
- How does Redis Cluster distribute keys, and what happens when you add a node?
- What are the trade-offs of Redis persistence (RDB vs AOF vs both vs none)?
- How would you build a rate limiter in Redis for 1M users?
- How does Redlock work, and what are its known limitations?
- What happens when Redis fails over — writes, reads, replicas?
Systems that use this component
See how the real designs on this platform put redis to work — concrete usage context per system.
URL Shortener
Cache-aside from L5 onwards; 92% hit rate on hot URLs cuts DB load 10x
Open systemTwitter/X timeline
Timeline fan-out cache; user session state; rate limiting
Open systemFeed cache; presence; hot post promotion
Open systemPresence + online status; message routing table
Open systemSlack
Channel membership cache; presence; typing indicators
Open systemRide Sharing
Driver location updates (GEOADD); trip state; surge pricing
Open systemTicket Booking
Seat reservation locks with TTL; queue positions during flash sales
Open systemPayment System
Idempotency-key deduplication; rate limiting; fraud counters
Open systemIn 2009, Salvatore Sanfilippo (antirez) — an Italian developer — was working on a startup that needed a very fast in-memory database. Existing options (Memcached) were too simple; SQL databases were too slow. Antirez wrote something in a few thousand lines of C that stored keys mapping to data structures, not just strings. Lists. Sets. Sorted sets. Hashes. All in-memory, all with sub-millisecond latency. He called it Redis (REmote DIctionary Server).
Today Redis is on nearly every cloud provider (AWS ElastiCache, GCP Memorystore, Azure Cache), backs the caching tier of most major websites, powers rate limiters, session stores, real-time leaderboards, job queues, and even acts as a lightweight event bus. Its data-structure-first design remains its differentiator — no other in-memory store gives you sorted sets with O(log N) rank queries, or bitmap operations, or geo-spatial queries in a single command.
Historical framing
- 2009 — Salvatore Sanfilippo (antirez) creates Redis. Written in C.
- 2011 — VMware sponsors development; Redis moves under Redis Labs (now Redis Ltd.).
- 2015 — Redis Cluster GA (16384 hash slots, native sharding).
- 2018 — Redis Streams data type — Kafka-lite for lightweight event processing.
- 2019 — RedisJSON, RedisSearch, RedisTimeSeries, RedisAI modules mature.
- 2020 — antirez steps back from lead maintainer role; Redis Ltd. takes over.
- 2024 — Redis license changes to SSPL/RSAL; AWS forks to Valkey; other forks: KeyDB (multi-threaded), Dragonfly (C++ rewrite).
Data types — the killer feature
Memcached only gives you strings. Redis gives you 10+ purpose-built data structures. Click any to see the commands and typical use case.
- SET foo "bar"
- GET foo
- INCR counter
- APPEND log " event"
Persistence — the RDB vs AOF debate
"What if the server crashes?" is the first question of every Redis interview. Redis offers two persistence modes — RDB (snapshot) and AOF (append-only log) — usable together or independently.
save 900 1 (snapshot every 15 min if ≥1 key changed).appendfsync everysec (default, ~1s max data loss).Redis Cluster — sharding across 16,384 hash slots
Single-node Redis has a memory ceiling (typically ~256 GB per box). Redis Cluster shards keys across N shards using 16,384 hash slots. Each shard owns a range of slots. The client hashes the key with CRC16(key) % 16384 and connects directly to the shard owning that slot.
Why 16,384 slots and not more? The cluster gossip protocol uses a bitmap of slots per node — 16,384 bits = 2 KB per node message. Keeps gossip cheap even in a 1,000-node cluster. Multi-key commands (MSET, MGET) require the keys to hash to the same slot — use hash tags like {user:123}:name to force co-location.
Sentinel — HA for non-cluster setups
If you don't need sharding but do need high availability, Redis Sentinel is the answer. Sentinel is a separate process fleet that monitors the master + replicas and promotes a replica if the master dies.
Eviction policies — what to drop when memory is full
Redis is in-memory. When memory is full, it evicts. The policy you pick determines what gets dropped — and if you get it wrong, your hit rate craters.
Pipelining — the throughput multiplier
A naive Redis client sends one command, waits for the reply, sends the next. At 1 ms RTT per command, you cap at ~1,000 ops/sec. Pipelining sends many commands in one batch and reads the replies at the end. 100× throughput is normal.
Product comparison
| Product | Hosting | Threading | Modules | License | Best for |
|---|---|---|---|---|---|
| Redis OSS | Self-hosted | Single (mostly) | Load your own | SSPL/RSAL (2024+) | Full control, self-hosted |
| Redis Enterprise | Redis Ltd. managed | Multi-shard | Included | Commercial | Enterprise SLAs + support |
| AWS ElastiCache | AWS managed | Single | Some Redis modules | Managed | AWS-native cache tier |
| GCP Memorystore | GCP managed | Single | Basic | Managed | GCP-native cache tier |
| Azure Cache for Redis | Azure managed | Single | Enterprise tier | Managed | Azure-native cache |
| Valkey | Self-hosted (OSS fork) | Single | Compatible | BSD-3 (permissive) | OSS replacement, AWS-backed |
| KeyDB | Self-hosted | Multi-threaded | Compatible | BSD-3 → SSPL | Higher throughput per box |
| Dragonfly | Self-hosted (C++) | Multi-threaded | Partial compat | BSL | Lower memory, higher throughput |
| Memcached | Self-hosted / managed | Multi | None | BSD | Simple KV cache, no persistence |
Applied in real systems
Twitter — timeline cache (pre-Manhattan)
Twitter famously used Redis for their user timeline cache. Each user's home timeline: a fan-in sorted set of ~800 recent tweets, updated on every follower's write. Sorted sets with ZRANGE gave them O(log N) inserts + range queries.
GitHub — session storage
GitHub uses Redis for session tokens (SET user_session:xyz ...) with TTL. Millions of active sessions. TTL-based expiration means no cleanup job needed.
Instagram — feed cache + counters
Redis for follower/following counts (INCR/DECR on integer keys), the ranked feed cache, and rate limiting on API endpoints. Their engineering blog details the Postgres-plus-Redis architecture.
Snapchat — leaderboards + best-friend rank
Sorted sets everywhere. Every user's 'best friends' is a sorted set keyed on friend_id with score = interaction count. ZINCRBY on every message. ZRANGE for display.
Uber — ride cache + geo queries
Redis Geo commands (GEOADD, GEORADIUS) for driver-location lookups. Redis handles millions of driver position updates per second across their global fleet.
Discord — presence tracking with bitmaps
Discord tracks whether users are online with Redis bitmaps (SETBIT/GETBIT). Storing 1 bit per user, 100M users = 12.5 MB. Fast intersection (BITCOUNT) for 'who of my friends is online'.
Stripe — idempotency keys + rate limits
Every Stripe API request accepts an Idempotency-Key. Redis stores 'request-id → response' with TTL. Retry gets the cached response. Also Redis for per-API-key rate limits.
Robinhood — real-time quotes (streams)
Redis Streams for market data ingestion. Each ticker is a stream; consumers (mobile clients, analytics) read at their own pace with XREAD BLOCK. Millions of ticks per second.
Netflix EVCache — memcached-based, similar pattern
Netflix uses EVCache (built on Memcached) as their main cache tier. Similar in spirit to Redis-as-cache. Handles 30M ops/sec + petabyte cache across regions.
Cloudflare — Redis Cell for rate limiting
Cloudflare uses Redis Cell (their CRDT-based rate limiter module) at edge PoPs. Millions of rate-limit decisions per second globally without a single-point Redis.
Airbnb — search results cache
Search results (expensive Elasticsearch queries) cached in Redis with TTL. Cache-aside pattern: check Redis first, miss → hit Elasticsearch → write to Redis. Massively reduces search load.
Shopify — cart state
Shopping cart contents stored in Redis for fast reads during checkout. Move to Postgres only on order submit. Cart abandon rate + lifetime managed via TTL.
Key takeaways
- Data structures, not just strings. Sorted sets, bitmaps, HyperLogLog, streams, geo — Redis solves problems no key-value store can. This is Redis's superpower.
- RDB + AOF: RDB for compact snapshots, AOF for durable log. Most production setups use both. AOF with fsync-every-second is the standard.
- Redis Cluster: 16,384 hash slots, CRC16-based routing. Multi-key ops require hash tags for co-location. Rebalancing moves slots, not individual keys.
- Sentinel: quorum-based failover for non-clustered setups. 3+ Sentinels required (odd number for quorum).
- Eviction policy is critical: allkeys-lru is the standard cache policy; volatile-ttl for mixed workloads; noeviction if Redis is your primary store.
- Pipelining is a 100× multiplier. Batch commands whenever possible. MULTI/EXEC is the transactional flavor.
- License landscape shifted in 2024. Valkey (AWS-backed fork), KeyDB, and Dragonfly are viable alternatives. Watch this space.
- Every serious cache-heavy architecture uses Redis (or a Redis-compatible fork). Learning Redis's data types is a P0 backend engineering skill.
References
- Sanfilippo (2009-2020) — antirez blog + Redis source code. The definitive reference.
- Redis official docs — redis.io/documentation (data types, persistence, cluster, sentinel).
- Carlson (2013) — Redis in Action. Manning. Classic reference.
- Redis Cluster spec — the 16,384-slot design rationale in the official spec doc.
- Kleppmann on Redlock — Kleppmann's 2016 blog critiquing Redis-based distributed locking.