Connection pooling
Why creating a TCP connection per request kills you at scale.
Every TCP connection has setup cost — ~2ms LAN, ~50ms cross-region. For a database talking to a client, that's prohibitive at scale. Connection pools solve this by keeping a fixed number of connections open + reusing them across requests.
Interactive pool sizing
Healthy — target is 60-70% steady state
The 4 pool sizing rules
1. Postgres primary: pool size = (cores × 2) + effective_spindles. On a 4-core RDS, that's 8 + 1 = 9. Adding more kills throughput via context switching.
2. Redis: single connection per app instance is often enough (Redis is single-threaded, more connections = more contention).
3. External APIs (Stripe, Twilio): pool per host, respect their rate limit. Typical: 20-50 connections per host.
4. HTTP client: reuse via keepalive. Node's http.Agent, Go's http.Transport, Python's requests.Session.
The 3am debugging story
Your Postgres CPU pegs at 100%. Pool exhaustion? Actually — it's the opposite. You set pool size = 200 to “handle spikes.” PG can only usefully run 8 queries at once on 4 cores. 200 clients competing for 4 cores = context switch storm. Reduce pool to 10. CPU drops to 30%. Latency drops. Throughput increases. This is real, and it's from the PgBouncer FAQ.
Applied in these systems
Practice what you just read
Every foundation concept has a companion quiz to close the loop.