Message Queue

Why it exists

Some work doesn't need to happen in the request/response cycle — sending a welcome email, resizing an image, computing an analytics rollup. If you do it inline, the user waits. If you drop it, you lose reliability. A message queue is the middle path: the producing service enqueues the work and returns immediately; a pool of workers pulls from the queue and processes at their own pace. This decoupling absorbs bursts and lets each side scale independently.

How it works

A queue is FIFO by default (with per-message-ACK semantics — the message stays visible until a worker explicitly acknowledges completion). Producers publish messages to a queue; workers poll the queue, process one message at a time, and ACK on success. On failure or timeout, the message becomes visible again for another worker (visibility timeout). Messages that fail N times land in a dead-letter queue for manual inspection. This differs from Kafka: queues do per-message ACK, not offset-based; consumption removes the message, not just advances a pointer.

Scaling characteristics

A single queue can handle 10K-100K messages/sec depending on message size and durability guarantees. Horizontal scale by adding partitions (like Kafka) or by splitting hot queues into topic hierarchies (per-user queues). Worker pools scale independently: 10 workers vs 1000 depending on the depth of the queue and processing time per message. Queue depth is your primary alarm — a growing queue means consumers are behind.

When to use it

• Background jobs (email, thumbnail generation, PDF rendering)
• Rate-smoothing bursty traffic — request comes in, task is enqueued, worker processes at bounded rate
• Retryable work with delayed retry semantics (backoff + DLQ)
• Fan-out from one producer to one consumer (unlike pub/sub which is 1-to-N)
• Isolating slow-but-non-critical work from the request path

When NOT to use it

• Sub-second delivery latency — queues typically buffer, adding ~50-500ms of latency
• Ordered event streams with replay (use Kafka)
• You need per-message durability guarantees stronger than what your queue offers — Kafka's disk-first design wins here
• Cross-region low-latency — most queue services are single-region or replicate with delay

Failure modes

• Worker crashes mid-message: visibility timeout returns the message to the queue; but if the worker committed a side effect (email sent) before crashing, you get a duplicate. Mitigate with idempotent workers.
• Poison message stalls the queue: one bad message keeps failing and blocks workers. Mitigate with DLQ + retry limit.
• Slow consumers create a backlog: queue depth grows, memory pressure at queue broker. Mitigate with autoscaling consumers + circuit breaker on producer.
• Message loss on broker failure if durability isn't configured — always enable replication + fsync for critical work.
• Fan-out amplification: one message triggers 10 downstream messages, each triggers 10 more — quickly overwhelms the queue. Mitigate with backpressure and per-tier rate limits.

Alternatives

• Kafka — for event streaming with replay + higher throughput; different semantics (offset-based, not per-message ACK)
• Cron + database polling — dead simple for low volume; doesn't scale but zero moving parts
• In-process job queue (Celery, Sidekiq) — piggybacks on Redis; great for smaller scale, harder to isolate failure
• Serverless (AWS SQS + Lambda) — managed queue + managed workers; cost-effective for spiky loads

Interview questions

• Your queue depth is climbing. What are the possible causes and how do you diagnose?
• A message fails 5 times. Where does it go, and what should you do about it?
• How would you guarantee exactly-once processing for a queue where each message triggers a payment?
• You're at 100K messages/sec on a single queue. When do you start sharding?
• How do you handle a message that requires a service that's currently down?
• Design a delayed-execution queue (message becomes visible after 24 hours).