Sync vs async communication
When to make the caller wait, and when to fire-and-forget.
Every service-to-service interaction is either synchronous (caller waits for response) or asynchronous (caller sends a message and moves on). The choice affects everything: coupling, failure modes, latency budgets, complexity, monitoring, retries.
Most engineers default to synchronous — it's simpler to reason about. But asynchronous is the tool for eliminating cascading failures, absorbing spikes, and decoupling teams. Every serious system has both. Learning which to use where is a Staff-level skill.
Sync vs async at a glance
| Dimension | Sync | Async |
|---|---|---|
| Caller behavior | Waits for response | Sends message and returns immediately |
| Latency of caller path | Slowest step in chain | Time to enqueue only (~1ms) |
| Coupling | Tight — caller depends on callee availability | Loose — buffered by queue |
| Failure propagation | Cascades up the call chain | Absorbed by queue; retryable |
| Load absorption | Downstream must scale with load | Queue buffers spikes for slow downstream |
| Debugging | Easier — linear call stack | Harder — logs scatter across time |
| Retries | Caller retries with backoff | Broker retries; DLQ for failures |
| When to use | User-facing responses | Background work, notifications, spikes |
The failure modes are different
When to pick which — the checklist
Applied in real systems
gRPC / REST — the sync default
Every RPC is a synchronous call. Client sends, waits for response, times out if none. Standard for user-facing APIs.
Kafka — the async backbone
Producers send messages to topics. Consumers process at their own pace. Producers don't wait. Kafka is the reference async message bus.
AWS SQS — managed async queues
Fully-managed distributed queue. Producers send, consumers poll. Dead-letter queues for failures. Standard for AWS async workflows.
RabbitMQ — AMQP async broker
Rich routing (topics, fanout, direct). Widely used in enterprise. Slightly harder to scale than Kafka but more flexible for complex fan-out patterns.
Temporal — async workflows with state
Async workflow orchestration with durable state. Great for long-running processes (booking flows, sagas, human-in-the-loop).
Webhooks — async callbacks
Stripe, Twilio, GitHub — send POST to your URL when events happen. Async by nature. Signature verification for security.
Key takeaways
- Sync: caller waits. Simple. Cascading failures on downstream slowness.
- Async: fire-and-forget via queue. Decoupled. Failure isolation. Adds latency + eventual-consistency complexity.
- Use sync for user-facing responses — user is waiting. Use async for background work, batch processing, notifications.
- Async buffers spikes. Producers keep sending even when consumers are slow.
- Every serious system has both. gRPC/REST for user requests, Kafka/SQS for internal fan-out.
References
- Kleppmann (2017) — DDIA, Chapter 11.
- Kafka documentation and books by Neha Narkhede et al.
Practice what you just read
Every foundation concept has a companion quiz to close the loop.