HTTP idempotency & retries

In 2011, Stripe had a problem. A merchant would call their POST /charges endpoint, the request would take 1.2 seconds because a bank check was slow, and their client library would time out at 1 second and retry. Sometimes the merchant would end up charging the customer twice. Sometimes they'd get a 500 Internal Server Error even though the charge went through — and retry, again causing a double charge. This is not a weird edge case. It is every API call that involves money and networks.

Stripe's answer, published in 2015, was a pattern that had lived in academic databases since the 1980s but had never crossed cleanly into HTTP APIs: idempotency keys. The client attaches a unique token to every mutating request. If the server sees the same token twice, it returns the original response instead of performing the operation again. Retries are safe. Money doesn't double. Networks can fail. Correctness is preserved.

The pattern is now industry standard. Shopify requires idempotency on all cart-mutating endpoints. PayPal, Adyen, and Square follow the same pattern. OpenAI added it to their API in 2023 for expensive LLM calls. AWS SDK auto-generates idempotency tokens for many services. The core idea: network operations are inherently unreliable — the API contract should make retries safe. If you build any API that mutates state, you will end up implementing this. Better to understand it deeply than to reinvent it badly.

The formal definition

An operation is idempotent if applying it N times has the same effect as applying it once. In HTTP, this means: sending the same request 1 time or 10 times leaves the server in the same state and returns the same response.

Idempotency is NOT the same as safety. A safe method has no side effects at all (GET, HEAD). An idempotent method may have side effects, but repeating them doesn't stack (PUT, DELETE). A non-idempotent method stacks side effects on repeat (POST creates a new row per call).

The classic bug — what happens WITHOUT idempotency keys

Before we build the solution, feel the disaster. Alice hits "pay $100" on a merchant site. The request travels through her flaky home Wi-Fi. Watch what happens:

Alice was charged $200 for a $100 purchase. The merchant server did nothing wrong. The client library did nothing wrong (retrying on network error is the correct behavior). The bank did nothing wrong. But the composition failed — because the API had no way to tell "is this a first attempt or a retry?"

The fix — idempotency keys

The client generates a unique token (UUID, random 24-byte string, or timestamp+hash) and sends it in the Idempotency-Key header. The server:

1. Extracts the key from headers.
2. Looks it up in a dedupe store (typically Redis, sometimes DynamoDB or Postgres).
3. If found + status is COMPLETE — returns the cached response body. Does NOT re-execute the operation.
4. If found + status is IN_PROGRESS — returns 409 Conflict (concurrent retry from another connection).
5. If not found — marks IN_PROGRESS with the request fingerprint, executes the operation, stores the response, marks COMPLETE.
6. TTL on the store entry: 24 hours is Stripe's convention. Long enough for retries + human investigation, short enough that Redis doesn't balloon.

The Redis dedupe state machine

The dedupe store entry moves through a small state machine. Understanding this is the difference between an idempotency layer that works and one that has subtle races.

   ┌────────────┐
   │  ABSENT    │  (key never seen)
   └──────┬─────┘
          │  1. First request: SET NX + TTL 24h
          ▼
   ┌────────────┐
   │ IN_PROGRESS│  (executing operation)
   └──────┬─────┘
          │  Success?
          │   ┌── yes → SET COMPLETE + response body
          │   └── no  → SET FAILED + error body
          ▼
   ┌────────────┐         ┌────────────┐
   │  COMPLETE  │         │   FAILED   │
   └──────┬─────┘         └──────┬─────┘
          │                      │
          │  Retry with same key: return cached body
          │  (any state below returns cached, not re-execute)
          │                      │
          └──────────┬───────────┘
                     ▼
              ┌────────────┐
              │  EXPIRED   │  (after 24h TTL)
              │            │  Next retry treated as
              │            │  new request. Rare in
              │            │  practice.
              └────────────┘

   Concurrent retry while IN_PROGRESS?
   → return 409 Conflict (client should wait + retry)

The request fingerprint — a subtle but critical detail

Same key + same body → return cached response.
Same key + DIFFERENT body → reject with 422 Unprocessable Entity.

Why? Because if a client sends POST /charges {amount: 100} with key abc123, then follows up with POST /charges {amount: 200} also with key abc123, one of two things is happening:

• Client bug — they reused the key. Silently returning the $100 charge response for a $200 request would cause a data-integrity nightmare.
• Attack — someone stole a client's key from logs and is trying to piggyback different requests on it. Rejecting protects the client.

Either way, reject with a clear error so the bug surfaces at development time rather than silently corrupting production. Stripe does this. Shopify does this. Any serious implementation does this.

Common implementation pitfalls

Idempotency in distributed transactions (sagas)

In a saga pattern, a multi-step business transaction is decomposed into local transactions per service. Each step MUST be idempotent because the orchestrator retries on failure. Same for compensating actions: cancel-order called twice = still cancelled.

This is why Temporal, AWS Step Functions, and Cadence workflow engines require every activity to be idempotent — they replay activities on worker restarts and cannot afford double-execution. The pattern from your HTTP API layer is the same pattern that makes distributed workflows correct.

Real-world implementations — same pattern, different scales

Key takeaways

1. Idempotency is a property of the API contract, not the implementation. A GET is idempotent because HTTP says so. A POST is idempotent only if the server treats idempotency-keys properly.
2. Networks are unreliable; retries are inevitable. Idempotency turns retry-on-failure from a client-side hack into a first-class API property.
3. The pattern is uniform: client generates key, server dedupes via Redis with 24h TTL and fingerprint check. Same key + same body → cached response. Same key + different body → reject.
4. Enforce at multiple layers: API layer (Redis check), DB layer (uniqueness constraint on key). Belt-and-suspenders for concurrent-retry races.
5. Distributed workflows depend on it: Temporal, Step Functions, Cadence, Kafka producers all require activity idempotence for their correctness guarantees.