Idempotency

A property where making the same API request multiple times produces the same result.

Idempotency means that making the same request multiple times produces the same result as making it once. This is critical for API reliability - if a network timeout occurs, clients can safely retry without causing duplicate actions.

Why It Matters

Network failures happen: Request sent, server processed it, but response was lost. Client doesn't know if it worked. Should they retry?

Without idempotency: Retry creates duplicate order, double charge, or duplicate record.

With idempotency: Retry returns same result as original request. Safe to retry any failed request.

HTTP Methods and Idempotency

MethodIdempotentSafeDescription
GETYesYesReading data never changes it
HEADYesYesSame as GET, no body
OPTIONSYesYesReturns allowed methods
PUTYesNoReplace entire resource
DELETEYesNoDelete returns same state
POSTNoNoCreates new resource each time
PATCHDependsNoDepends on implementation

Safe = doesn't modify server state Idempotent = multiple identical requests = same result

Implementing Idempotency

Idempotency key: Client generates unique key (UUID) and sends with request. Server stores key with result. On retry, server returns cached result instead of re-processing.

How it works:

  1. Client generates idempotency key
  2. Client sends request with key in header
  3. Server checks if key exists
  4. If exists: return stored result
  5. If new: process request, store result with key
  6. Return result

Idempotency Key Best Practices

Key format: Use UUID v4 or combine user ID + timestamp + random. Must be unique per distinct operation.

Key lifetime: Store keys for 24-48 hours. Too short = retries fail. Too long = storage grows.

Key scope: Keys should be scoped to user/account. Different users can use same key.

Response storage: Store full response (status code, body) to return on retry.

Making Operations Idempotent

Create operations (POST): Use idempotency keys. Or use PUT with client-generated ID: PUT /orders/uuid-123

Update operations: Use conditional updates. UPDATE WHERE id = 1 AND version = 5 fails if version changed.

Financial operations: Store transaction IDs. Check if transaction already processed before executing.

Increment operations: Instead of balance += 10, use SET balance = 110 WHERE balance = 100 (check current value).

Common Patterns

Optimistic locking: Include version number. Update only if version matches. Prevents concurrent modification.

Unique constraints: Database-level uniqueness prevents duplicates even under race conditions.

Request deduplication: Hash request body + timestamp window. Reject duplicates within window.

Two-phase operations:

  1. Reserve/prepare (idempotent)
  2. Commit/execute (idempotent with reference to step 1)

Real-World Examples

Payment processing: Stripe requires idempotency key for charges. Retry same key = same result, no double charge.

Order creation: Store order with unique reference. Retry creates no duplicate if reference exists.

Email sending: Store sent message IDs. Skip if already sent.

Common Mistakes

1. Generating key server-side: Client must generate key. Server-generated key changes on retry.

2. Short key expiration: Network issues can last hours. Keep keys at least 24 hours.

3. Not storing full response: Store status code and body. Returning just "already processed" doesn't help client.

4. Ignoring partial failures: If request partially succeeded, retry logic must handle incomplete state.

5. Using POST for idempotent operations: If operation is naturally idempotent, use PUT or PATCH instead.

When Idempotency Matters Most

  • Payment and financial transactions
  • Order processing
  • Any operation that can't be easily reversed
  • Operations called from unreliable networks
  • Webhook handlers (may be called multiple times)

Code Examples

Implementing Idempotency Keys

// Middleware for idempotency key handling
const idempotencyMiddleware = async (req, res, next) => {
  const key = req.headers['idempotency-key'];

  if (!key) {
    return next(); // No key, process normally
  }

  // Check for existing result
  const cached = await redis.get(`idempotency:${req.user.id}:${key}`);

  if (cached) {
    const { statusCode, body } = JSON.parse(cached);
    return res.status(statusCode).json(body);
  }

  // Store original res.json to intercept response
  const originalJson = res.json.bind(res);
  res.json = async (body) => {
    // Cache the response for 24 hours
    await redis.setex(
      `idempotency:${req.user.id}:${key}`,
      86400,
      JSON.stringify({ statusCode: res.statusCode, body })
    );
    return originalJson(body);
  };

  next();
};

// Client-side usage
async function createOrder(orderData) {
  const idempotencyKey = crypto.randomUUID();

  const response = await fetch('/api/orders', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Idempotency-Key': idempotencyKey
    },
    body: JSON.stringify(orderData)
  });

  return response.json();
}