Handling Contentful Rate Limits and the Sync API

This guide, part of the Contentful Integration Guide, deals with the limits that shape large Contentful integrations: per-second request limits on the delivery APIs, query complexity limits on GraphQL, and the cost of reading an entire space repeatedly. It shows how to pool and back off requests correctly, trim queries, and use the Sync API for jobs that need all content, such as search indexing and exports.

Most sites never notice Contentful’s rate limits during normal traffic, because responses are cached at the CDN. They appear in three situations: full static builds that fetch thousands of pages in parallel, background jobs that re-read the whole space, and traffic spikes on uncached server-rendered pages. Each needs a different treatment, but all start with understanding which requests count and how to recover when a 429 arrives.

The Problem

A retailer’s search indexing job re-read all 60,000 entries every hour through the Delivery API, page by page, to find changes. Together with the static site build, which fetched products in parallel without limits, it regularly exceeded the rate limit. Builds failed intermittently with 429 errors, and when a build retried, it made the problem worse. The GraphQL product listing query, which requested nested variants and related products, also hit the complexity limit for categories with many products.

How the Limits Work

Delivery and preview limits. The Content Delivery API and Preview API allow a certain number of uncached requests per second per space; requests served from Contentful’s CDN cache do not count against the limit. When exceeded, the API responds with 429 and headers such as X-Contentful-RateLimit-Reset, giving the seconds to wait.

GraphQL complexity. The GraphQL Content API computes a complexity score from the requested collections, limits and nesting, and rejects queries above the limit. Collections with high limit values nested inside other collections multiply quickly.

Sync API. The Sync API returns all published entries and assets of an environment in pages, then a nextSyncToken. Later calls with that token return only what changed, including deletions, since the last sync. It is designed for exactly the jobs that otherwise re-read everything.

Implementation

A small pool limits concurrency for build-time fetches and retries 429 responses after the time Contentful indicates, with jitter so parallel workers do not retry in lockstep.

// lib/contentful/pool.ts
export function createPool(concurrency: number) {
  let active = 0;
  const queue: (() => void)[] = [];
  const next = () => { active--; queue.shift()?.(); };
  return async function run<T>(task: () => Promise<T>): Promise<T> {
    if (active >= concurrency) await new Promise<void>((r) => queue.push(r));
    active++;
    try { return await task(); } finally { next(); }
  };
}

export async function fetchWithRateLimit(url: string, init: RequestInit, attempts = 5): Promise<Response> {
  for (let i = 0; i < attempts; i++) {
    const res = await fetch(url, init);
    if (res.status !== 429 && res.status < 500) return res;
    const reset = Number(res.headers.get("x-contentful-ratelimit-reset"));
    const wait = (Number.isFinite(reset) && reset > 0 ? reset * 1000 : 2 ** i * 500) + Math.random() * 250;
    await new Promise((r) => setTimeout(r, wait));
  }
  throw new Error(`Contentful request failed after ${attempts} attempts: ${url}`);
}

For jobs that need all content, the Sync API replaces repeated full reads. The job stores the sync token between runs.

// jobs/sync-index.ts
import { fetchWithRateLimit } from "@/lib/contentful/pool";
import { tokenStore, searchIndex } from "@/lib/job-deps";

const BASE = `https://cdn.contentful.com/spaces/${process.env.CONTENTFUL_SPACE_ID}/environments/master/sync`;
const headers = { Authorization: `Bearer ${process.env.CONTENTFUL_ACCESS_TOKEN}` };

export async function runSync() {
  let token = await tokenStore.get("contentful-sync");
  let url = token ? `${BASE}?sync_token=${token}` : `${BASE}?initial=true&type=Entry`;
  while (url) {
    const res = await fetchWithRateLimit(url, { headers });
    const page = (await res.json()) as { items: { sys: { id: string; type: string } }[]; nextPageUrl?: string; nextSyncUrl?: string };
    for (const item of page.items) {
      if (item.sys.type === "DeletedEntry") await searchIndex.remove(item.sys.id);
      else if (item.sys.type === "Entry") await searchIndex.upsert(item);
    }
    if (page.nextPageUrl) url = page.nextPageUrl;
    else {
      token = new URL(page.nextSyncUrl!).searchParams.get("sync_token");
      await tokenStore.set("contentful-sync", token!);
      url = "";
    }
  }
}

Sync returns entries with all locales in each field, so the index job can build per-locale documents from one response. It does not resolve links; look up referenced entries from the synced data itself, which the job already has.

Trimming GraphQL queries

When a query hits the complexity limit, reduce the multiplication: lower limit values on nested collections, fetch nested data in a second query for the items that need it, and select only fields the component renders. Paginate large collections with skip and limit in pages that stay well under the limit, rather than requesting everything at once.

Making builds need fewer requests

The best way to stay within limits is to make fewer requests. Static builds that fetch each page’s entry separately make as many requests as there are pages; fetching list pages of entries with the fields each page needs, a hundred at a time, reduces that by two orders of magnitude, at the cost of a slightly larger response per request. Shared data such as navigation, footers and site settings should be fetched once per build and reused, not once per page. And incremental regeneration avoids full builds on publish altogether, so the large request volume happens only on the first build or after a deploy that invalidates everything. Combined, these changes usually matter more than any tuning of concurrency or backoff, which only decide how gracefully a build meets the limit, not whether it approaches it.

Configuration Reference

Gotchas & Edge Cases

• Preview traffic. The Preview API has its own limit and is never CDN-cached. Heavy live-preview usage by many editors can approach it; debounce preview refetches.
• Retrying everything. Retrying 4xx errors other than 429 wastes attempts; a 400 or 404 will not succeed on retry.
• Sync token loss. If the token is lost, the next run performs a full initial sync. Store tokens durably.
• Sync and environments. Sync tokens are per environment. After an alias switch, start a new initial sync against the new environment.

Worked Example

The retailer replaced its hourly full read with the Sync API and a stored token, which reduced the indexing job from about 600 requests per run to between one and five. The static build moved to a pool of eight concurrent requests with reset-aware retries, and the product listing query was split into a listing query and a separate variants query for visible products. Build failures from rate limits stopped entirely, and the search index updated within minutes of changes instead of up to an hour.

Monitoring Usage

Contentful’s usage dashboards show API calls per month by API, which is useful for plan decisions but too coarse for debugging. Add your own metrics: requests per workload, build, pages, preview, jobs, the number of 429 responses and total wait time spent in backoff, and GraphQL complexity errors by query name. A build whose backoff time grows over weeks is approaching the limit as content grows and needs attention before it starts failing. Alert on any 429s during page rendering, which indicates that caching is not working as intended, since cached page traffic should never reach the limit.

Rollout Checklist

• Serve page renders from cached queries and make sure caching works.
• Pool build-time requests and honour reset headers with jitter.
• Replace full reads in background jobs with the Sync API.
• Keep GraphQL queries under complexity limits by splitting and trimming.
• Store sync tokens durably and restart syncs after alias switches.
• Monitor 429s, backoff time and complexity errors per workload.

Frequently Asked Questions

Related

• Contentful Integration Guide
• Rendering Contentful Rich Text with Embedded Entries
• Load Testing GraphQL Endpoints for Enterprise Scale