Replication — leader, multi-leader, leaderless

Replication is the foundation of every reliable database. You store the same data on multiple machines so that when one dies — and machines do die — you can keep serving reads and writes from another. But how to keep replicas in sync is one of the oldest and hardest problems in distributed systems. Three families have emerged: single-leader (the standard), multi-leader (for multi-region writes), and leaderless (Dynamo-style). Each makes a different consistency-vs-availability trade-off.

Replication answers three business needs at once: availability (if one node dies, keep serving), scale reads (multiple replicas serve read traffic), and geographic proximity (put a replica near the user). But every replication topology forces a trade-off — the CAP triangle in action. You cannot have strong consistency, instant availability, and multi-region writes simultaneously.

The three topologies visualized

Single-leader replication — the default

Also called master-slave (deprecated terminology) or primary-replica. One node is the leader; all writes go there. Followers replicate from the leader — either synchronously (leader waits for follower ACK before returning to client — strong durability, higher latency) or asynchronously (leader ACKs client immediately, follower catches up eventually — lower latency, risk of data loss on leader failure).

The trade-offs of async vs sync replication: Async is the default (Postgres streaming replication, MySQL binlog) because it doesn't hurt write latency. Cost: if the leader dies before replicating, that write is lost. Semi-synchronous (MySQL, Postgres) requires at least one replica to ACK — a compromise. Fully synchronous is rare because one slow replica stalls all writes.

Failover — the hard part

When the leader dies, you need to promote a follower. This sounds simple until you handle:

• Split brain: if the network partitions and both sides elect a new leader, you get two leaders accepting conflicting writes. Requires consensus (Raft, Paxos) to prevent.
• Lost writes: with async replication, some writes on the old leader never made it to the new leader. Data loss is the norm.
• Timeout tuning: too aggressive (false failovers on GC pauses) or too slow (users wait for minutes).
• Redirect clients: apps need to know where the new leader is. DNS, service discovery, or client-side libraries handle this.

Multi-leader replication — for multi-region writes

Single-leader gives you strong consistency but only one region can accept writes. If your users are global, this means every write from Tokyo travels to (say) the US East region — 200ms of round-trip. Multi-leader lets every region accept its own writes, then replicates asynchronously to peers.

The catch: conflicts. Alice in Tokyo updates her profile to "pronouns: they/them." At the same instant, Alice in San Francisco (via her phone on a plane over the Pacific) updates "pronouns: she/her." Both regions accept the write. Which wins? Conflict resolution options:

• Last-write-wins (LWW) — pick the highest timestamp. Simple, arbitrary, sometimes loses data.
• Application-level — surface the conflict to the app (like Git merge conflicts). Riak siblings pattern.
• CRDTs — data structures that merge without conflict (G-Counter, LWW-Set, etc.). Perfect for counters, sets, some documents.

Leaderless replication — the Dynamo model

Amazon Dynamo (2007) took a radical position: no leader at all. Any replica accepts writes. Any replica serves reads. Consistency is achieved via quorums: write to W of N replicas, read from R of N, and require R + W > N.

Cassandra, DynamoDB, Riak all use this. The nice property: every node is equal. No leader election, no failover — if a node is down, the others just pick up the slack. Cost: reads may see stale data unless you hit the read quorum, and clock skew across replicas can produce weird outcomes.

Comparison — which one to pick

Applied in real systems

Key takeaways

• Three topologies: single-leader (default, strong consistency), multi-leader (multi-region writes, conflict resolution needed), leaderless (Dynamo-style, quorums).
• Async vs sync replication is the fundamental latency-vs-durability trade-off. Async is faster but risks data loss on failover.
• Failover is hard: split-brain, lost writes, timeout tuning. Consensus (Raft) is the only clean solution for automatic failover.
• Multi-leader = conflict resolution. LWW, app-level, CRDTs. Every solution loses some data or surfaces the mess to the app.
• Leaderless (Dynamo): quorum-based. Any node takes writes, any node serves reads. R+W>N gives strong reads. Cassandra + DynamoDB + Riak.
• Every serious database has one of these three. Learn to identify which and its exact consistency guarantees.

References

• DeCandia et al. (2007) — Dynamo.
• Kleppmann (2017) — DDIA, Chapter 5.
• Postgres, MySQL, MongoDB, Cassandra docs on replication.