Database Types — SQL, NoSQL, NewSQL
The taxonomy of database engines: relational, key-value, document, wide-column, graph, time-series, search, vector, and NewSQL — with when to pick each.
In 1970, Edgar F. "Ted" Codd at IBM published a paper titled "A Relational Model of Data for Large Shared Data Banks." It proposed something radical: instead of navigating pointers through hierarchical or network databases, express data as relations (tables) and query them with a declarative language (what SQL would become). For 35 years, this was the only real game in town. IBM DB2, Oracle, Sybase, Informix, SQL Server, MySQL, and PostgreSQL all inherited Codd's design.
Then, between 2006 and 2010, something broke. Google published the BigTable paper (2006). Amazon published Dynamo (2007). Facebook released Cassandra (2008). 10gen released MongoDB (2009). The engineers at these companies had hit a wall: their workloads needed global scale, and traditional SQL databases couldn't provide it without giving up availability. A wave of NoSQL ("Not Only SQL") databases was born, each specialized for a specific data shape: key-value, document, wide-column, graph, time-series, search, and later vector.
In 2012, Google published Spanner — showing that with careful engineering (atomic clocks, TrueTime, Paxos), you could actually have distributed ACID transactions at planetary scale. This kicked off the NewSQL era: CockroachDB, TiDB, YugabyteDB — databases that look like SQL to your application but scale horizontally like NoSQL underneath. Today the landscape has ~9 major database families with hundreds of products, and the "which database?" decision is genuinely the most consequential choice in most system designs.
The seminal papers
- Codd (1970) — "A Relational Model of Data for Large Shared Data Banks." The paper that defined SQL databases. CACM.
- Chang et al. (2006) — Bigtable: A Distributed Storage System for Structured Data. OSDI. Wide-column, LSM-based.
- DeCandia et al. (2007) — Dynamo: Amazon's Highly Available Key-value Store. SOSP.
- Corbett et al. (2012) — Google Spanner. OSDI. Launched NewSQL.
- Kleppmann (2017) — Designing Data-Intensive Applications. Kleppmann's book is the modern reference for the whole taxonomy.
The three big families
- Ad-hoc queries via JOIN
- Strict schemas
- ACID transactions
- 50 years of maturity
- Vertical scaling ceiling
- Schema changes need migrations
- Struggle with unstructured data
- Horizontal scale-out
- Flexible schemas
- Specialized for data shape
- Massive throughput
- Weaker consistency (usually)
- Category-specific limits
- Hard to do ad-hoc analytics
- Distributed ACID
- SQL queries
- No manual sharding
- Higher latency than either
- Operational complexity
- Newer, less proven
Interactive: click any type to explore
Every serious database landing in one of 9 categories below. Click any to see: data model, query interface, real products, when to use it, and when to avoid it.
CREATE TABLE users (id INT, name TEXT, email TEXT); SELECT users.name, orders.total FROM users JOIN orders ON users.id = orders.user_id;
- Transactional workloads (banking, orders, users)
- Complex ad-hoc queries
- Applications with relationships (JOIN)
- Strong consistency needs
- Multi-region write scale
- Very high write throughput (>50k/s per node)
- Schemaless or nested-object data
SQL vs NoSQL — the fundamental split
The most-asked interview question of the last decade. The honest answer: NoSQL is not one thing — it's 7 different specialized categories. But the split from SQL matters because they make opposite trade-offs:
| Dimension | SQL family | NoSQL family |
|---|---|---|
| Data model | Tables (rows + columns) with strict schema | Depends on subtype: keys, docs, wide-columns, graphs, time-series, text, vectors |
| Query language | SQL — standardized, powerful, JOIN-friendly | Each product has its own — CQL, MongoQL, Cypher, PromQL, ES Query DSL |
| Schema | Enforced at write time. ALTER TABLE for changes. | Usually schemaless or 'schema on read' — each record can have different fields |
| ACID transactions | Full ACID within a database. Cross-shard is complex (2PC). | Usually eventual consistency. Some products offer tunable (Cassandra CL) or bounded (DynamoDB TransactWriteItems). |
| JOIN queries | First-class — the whole point of the relational model | Usually not supported — denormalize instead, or use app-side joins |
| Horizontal scaling | Hard — vertical scaling first, then manual sharding (Vitess, Citus) | Native — most were designed to scale-out from day 1 |
| Consistency model | Strong (linearizable within a node) | Usually eventual, sometimes causal, rarely strong |
| Best fit workload | OLTP (transactional). Small-to-medium ad-hoc analytics. | High-write throughput, flexible schemas, specific data shapes |
| Maturity | 50+ years. Tooling galore. | 10-15 years. Tooling improving fast. |
| Team knowledge | Every engineer knows SQL | Each product is a learning curve |
The SQL family — relational + columnar + NewSQL
SQL databases split into three subfamilies based on what they optimize for.
The NoSQL family — seven data shapes
NoSQL is where the taxonomy gets fun. Each NoSQL category is specialized for a specific data shape and access pattern. Pick the wrong one for your workload and you'll fight the database forever.
The product landscape — real names, real categories
Interactive: which database should I use?
Answer 4 questions about your workload. Get a recommendation. (This is a simplification — real decisions need more nuance — but it's a great mental starting point.)
Applied in real systems — who uses what for what
Every serious tech company runs multiple databases — one per data shape. Here are some real production choices.
Netflix — a database for every job
Cassandra for viewing history and personalization (writes are king). DynamoDB for playback session state. EVCache (memcached) for the recommendation cache. Elasticsearch for the internal search across metadata. MySQL / Aurora for billing.
Uber — MySQL + Cassandra + Postgres
MySQL for trip records (ACID, sharded via Vitess). Cassandra for driver locations (write-heavy). Postgres (Schemaless) for the internal document store. Presto for analytics. Redis for the ETA cache.
Stripe — Postgres + MongoDB
MongoDB for the primary transactional store (chosen in 2010, before NewSQL matured). Postgres for analytical workloads and newer services. Redis for rate limiting. Stripe famously wrote "Online migrations at scale" about moving data between engines.
Instagram — Postgres + Cassandra
Postgres for user accounts, relationships (sharded across thousands of instances). Cassandra for the feed and activity data. Memcached for read-heavy caching. Went from 1 Postgres instance to 5000+ over 5 years.
Discord — Cassandra → ScyllaDB
Trillions of chat messages. Started on Cassandra 2016. In 2022 migrated to ScyllaDB (Cassandra rewrite in C++). Store partitioned by (channel_id, day_bucket). MongoDB for user metadata. Elasticsearch for search.
Airbnb — MySQL + Presto + Druid
MySQL primary for listings, bookings (heavily sharded via Vitess). ElasticSearch for search (listings by location + filters). Druid for real-time analytics. Presto for offline analytics on the data warehouse.
LinkedIn — Espresso + Voldemort + Kafka
Built their own: Espresso (document DB on MySQL), Voldemort (Dynamo-clone key-value, open-sourced 2009), Ambry (blob storage), and famously Kafka (which grew far beyond LinkedIn). Now migrating some to PostgreSQL.
Google — BigTable → Spanner → F1
Bigtable for wide-column workloads (Search index, Analytics). Spanner for globally-consistent SQL (AdWords, Play). Firestore for mobile apps. BigQuery for analytics. Colossus for object storage. Every one was built at Google.
Amazon.com — DynamoDB + Aurora + Redshift
DynamoDB for shopping cart, session data (Amazon's own creation). Aurora (MySQL-compatible) for transactional workloads. Redshift for analytical warehouse. OpenSearch for product search. Neptune for personalization graphs.
Pinterest — HBase + MySQL + Redis
HBase for the pin metadata (wide-column, write-heavy). MySQL (sharded, called "MySQL infra") for user data. Redis for the home feed cache. Elasticsearch for search.
Figma — Postgres + custom LiveGraph
Postgres primary (heavily sharded). Built custom real-time collab layer called LiveGraph on top. Uses DynamoDB for OT operations queue. Elasticsearch for search across files.
LLM apps — Vector DBs + Postgres
Every RAG (retrieval-augmented generation) system in 2024+ uses a vector database: Pinecone, Weaviate, Milvus, Qdrant, or simply pgvector for those on Postgres. Store embeddings, query by cosine similarity.
Prometheus — the time-series standard
Every Kubernetes cluster ships with Prometheus for metrics. Custom TSDB (not general-purpose). PromQL query language. Pull-based scraping. Long-term storage typically offloaded to Thanos or VictoriaMetrics (S3-backed).
Neo4j — fraud detection at banks
Neo4j is the market leader in graph databases. Used by UBS, eBay (fraud rings), NASA (mission planning), PayPal (money-laundering detection). Cypher query language is the standard for graph traversal.
Key takeaways
- 3 families: SQL (Codd 1970), NoSQL (2007-10 wave), NewSQL (Spanner 2012 onward).
- SQL has 3 subfamilies: relational OLTP (Postgres, MySQL, Oracle), columnar OLAP (BigQuery, Redshift, Snowflake, ClickHouse), and NewSQL (CockroachDB, TiDB, YugabyteDB, Spanner).
- NoSQL is 7 categories, not 1: key-value, document, wide-column, graph, time-series, search, vector. Each specialized for a specific data shape.
- Trade-offs: SQL gives you flexible queries (JOIN, ad-hoc analytics), strict schemas, and ACID — within one node. NoSQL gives you horizontal scaling, flexible schemas, and specialized data models — but each category makes different consistency compromises.
- NewSQL closes the gap: distributed ACID with SQL semantics. The 2020s default for globally-consistent OLTP workloads.
- You will use multiple databases. Every serious tech company runs 3-8 databases in production, one per workload shape. Netflix, Uber, LinkedIn, Airbnb — all polyglot persistence.
- The 4 decision questions: consistency requirement (ACID or eventual?), query shape (JOIN or point lookup?), scale (single node OK or must distribute?), data shape (rows, docs, keys, graphs, time-series, blobs, vectors?). Answer these → category becomes obvious.
References
- Codd (1970) — A Relational Model of Data. CACM. The foundational SQL paper.
- Chang et al. (2006) — Bigtable. OSDI.
- DeCandia et al. (2007) — Dynamo. SOSP.
- Corbett et al. (2012) — Spanner. OSDI.
- Stonebraker & Cattell (2011) — "10 rules for scalable performance in simple operation datastores." A NoSQL-critical assessment.
- Kleppmann (2017) — Designing Data-Intensive Applications. Chapters 2-3 are the taxonomy reference.
- db-engines.com — real-time ranking of databases by popularity. Great for seeing the landscape.
Practice what you just read
Every foundation concept has a companion quiz to close the loop.