Idempotent Webhook Handlers with Delivery IDs and Replay Protection

This guide, part of Webhook-Triggered Rebuilds, makes a webhook handler safe against the three ways CMS deliveries misbehave in production: the same event delivered twice, an old event replayed by an attacker, and events arriving in a different order from the changes they describe.

CMS platforms deliver webhooks at least once, not exactly once. If your endpoint times out, returns an error or the connection drops after processing, the CMS retries, and your handler sees the same event again. Signatures prove an event came from the CMS, but not that it is new: a captured, validly signed request can be replayed later. And deliveries are concurrent, so the event for revision 7 of an entry can arrive after the event for revision 8. A handler that is correct under all three is one you never have to think about again.

The Problem

An e-commerce site’s webhook handler purged CDN tags and triggered regeneration for each product publish. During a CDN slowdown, the handler took longer than the CMS’s 10-second timeout, so the CMS retried each webhook up to three times. Every retry ran the full purge and regeneration again, which slowed the handler further and caused more retries, until the CDN’s purge API rate-limited the site and nothing was purged for 20 minutes. Separately, a security review showed that a signed webhook captured from a log could be replayed indefinitely to force expensive purges, because the signature carried no timestamp check.

How Idempotency Works for Webhooks

Three mechanisms, applied in order after the signature check:

Replay window. When the CMS includes a timestamp in the signed content, as Sanity’s signature header and several others do, reject events whose timestamp is more than a few minutes old. That makes captured requests worthless shortly after capture. Where the CMS signs only the body, add your own timestamp header in the webhook configuration if the platform supports templated headers, or rely on the next two checks.

Delivery id. Many platforms send a unique id per delivery or per event, in a header or in the body. Store processed ids for a day; if an id repeats, return success without doing anything. Returning 2xx matters: the CMS will keep retrying anything else.

Revision ordering. Events describe entry states, and a later revision supersedes an earlier one. Store the highest revision processed per entry; ignore events for older revisions. Use the CMS’s revision number where it exists, sys.revision in Contentful, _rev changes in Sanity, updatedAt elsewhere, as a monotonic marker.

Implementation

The handler below uses Redis for the processed-id set and the per-entry revision map. The checks are cheap, so they run synchronously before the handler acknowledges the event; the actual work runs in a queue.

// app/api/cms-webhook/route.ts
import { createHmac, timingSafeEqual } from "node:crypto";
import Redis from "ioredis";
import { enqueue } from "@/lib/queue";

const redis = new Redis(process.env.REDIS_URL ?? "");
const REPLAY_WINDOW_S = 300;

interface NormalizedEvent {
  deliveryId: string | null;
  entryId: string;
  revision: number;
  event: "publish" | "unpublish" | "delete";
}

function verify(raw: string, header: string | null): { ok: boolean; timestamp?: number } {
  // Header format "t=<unix>,v1=<hex hmac of `${t}.${raw}`>" (common convention; adapt per CMS)
  const parts = Object.fromEntries((header ?? "").split(",").map((p) => p.split("=") as [string, string]));
  const t = Number(parts.t);
  const expected = createHmac("sha256", process.env.CMS_WEBHOOK_SECRET ?? "").update(`${parts.t}.${raw}`).digest("hex");
  const a = Buffer.from(parts.v1 ?? "", "hex");
  const b = Buffer.from(expected, "hex");
  return { ok: a.length === b.length && timingSafeEqual(a, b), timestamp: t };
}

export async function POST(req: Request): Promise<Response> {
  const raw = await req.text();
  const { ok, timestamp } = verify(raw, req.headers.get("x-cms-signature"));
  if (!ok) return new Response("invalid signature", { status: 401 });
  if (!timestamp || Math.abs(Date.now() / 1000 - timestamp) > REPLAY_WINDOW_S) {
    return new Response("stale or replayed event", { status: 401 });
  }

  const evt = JSON.parse(raw) as NormalizedEvent;

  // 1. Delivery id: first writer wins, duplicates are acknowledged and skipped.
  if (evt.deliveryId) {
    const fresh = await redis.set(`wh:delivery:${evt.deliveryId}`, "1", "EX", 86_400, "NX");
    if (!fresh) return Response.json({ duplicate: true }, { status: 200 });
  }

  // 2. Revision ordering: only newer revisions change anything (atomic compare-and-set in Lua).
  const newer = (await redis.eval(
    `local cur = tonumber(redis.call('GET', KEYS[1]) or '-1')
     if tonumber(ARGV[1]) > cur then redis.call('SET', KEYS[1], ARGV[1], 'EX', 2592000) return 1 end
     return 0`,
    1,
    `wh:rev:${evt.entryId}`,
    String(evt.revision),
  )) as number;
  if (!newer && evt.event === "publish") return Response.json({ stale: true }, { status: 200 });

  // 3. Acknowledge fast, work later.
  await enqueue({ entryId: evt.entryId, event: evt.event, revision: evt.revision });
  return Response.json({ queued: true }, { status: 202 });
}

Unpublish and delete events bypass the revision check here because they may carry no new revision; handle them idempotently in the worker instead, since unpublishing an already unpublished entry changes nothing. The worker itself should also be idempotent: revalidating a tag twice or purging a key twice is harmless, which is one reason tag-based invalidation is a good fit for at-least-once delivery.

Normalizing payloads before the checks

The handler above works on a NormalizedEvent, not on the raw CMS payload. A small adapter per CMS maps the platform’s fields into that shape: where the delivery id lives, which field holds the revision, and how the event type is named. Keeping the checks platform-independent pays off the first time a second CMS is added, or when a CMS changes its payload format between API versions: only the adapter changes, and the adapter can be tested with stored fixtures from each platform. Verify the signature before running the adapter, so malformed or hostile payloads never reach parsing code that might throw or behave unexpectedly.

Configuration Reference

Gotchas & Edge Cases

• Returning errors for duplicates. A 409 or 400 for a duplicate delivery makes the CMS retry it again. Acknowledge duplicates with 2xx.
• Checking the timestamp outside the signature. A timestamp in an unsigned header can be changed by an attacker. Only trust timestamps that are part of the signed material.
• Clock skew. A replay window smaller than the skew between the CMS and your servers rejects valid events. Keep servers on NTP and the window at several minutes.
• Revision numbers that reset. Copying content between environments or restoring backups can reset revision counters. Include the environment in the revision key, and reset the stored revisions after a restore.
• Doing the work before acknowledging. Slow processing is the root cause of most retry storms. Verify, deduplicate, enqueue, respond, in that order.

Worked Example

The e-commerce team moved purging and regeneration into a queue worker and added the three checks. Webhook responses dropped from several seconds to about 40 milliseconds, so the CMS stopped retrying. During the next CDN slowdown, the worker’s queue grew for a few minutes and then drained, with each product purged exactly once. The security review’s replay test now fails with a 401 after five minutes, and the dashboard shows the duplicate rate, which hovered around two percent of deliveries, as a normal background figure rather than a source of incidents.

Rollout Checklist

• Find out which idempotency signals your CMS sends, per the table above.
• Verify the signature over the raw body, including the timestamp where it is signed.
• Deduplicate by delivery id and order by revision before doing any work.
• Acknowledge within two seconds and process in a queue.
• Make the worker’s actions idempotent, preferring tag invalidation over counters or appends.

Frequently Asked Questions

Related

• Webhook-Triggered Rebuilds
• Monitoring CMS Webhook Delivery, Failures and Retries
• Preview & Draft Workflow Patterns