Quick answer: A reliable JSON API client detects every documented rate-limit response, honors
Retry-After, applies bounded exponential backoff with jitter when guidance is absent, controls shared concurrency, avoids restarting pagination, caps total retry work, and records quota decisions. Blind immediate retries consume more capacity and can keep an integration rate-limited longer.
A 429 is feedback that the client must change its schedule, not permission to run the same loop faster. These seven mistakes cover the details that matter when workers, pages, and retries share one provider quota.
For the wider status and error-body contract, read JSON API error response formats. Rate-limit handling needs the HTTP headers as well as the JSON body.
Inspect a JSON error body in the JSON Viewer. Rate-limit timing still comes from the status, response headers, and provider policy.
1. Do you detect only HTTP 429 and ignore provider-specific signals?
Handle the statuses and fields documented by the provider, not only a generic 429 branch. Some services use 403 for secondary limits, distinguish quota exhaustion from abuse protection, or identify scope through headers. Parse the JSON error safely, but let the HTTP status and authoritative headers determine retry timing.
RFC 6585 defines 429 Too Many Requests and allows Retry-After. It intentionally does not define how a service identifies callers or counts requests; that scope belongs to the API contract.
GitHub's current REST rate-limit documentation tells clients to handle both 403 and 429. It exposes x-ratelimit-remaining, x-ratelimit-reset, and x-ratelimit-resource, while secondary limits may not provide a remaining count.
Do not treat every 403 as throttling. Authentication, authorization, and policy failures also use it. Check documented headers and machine-readable fields, then return a distinct internal error with the calculated next-attempt time.
Avoid scraping human messages. Preserve a request ID for support, but never log access tokens or sensitive payloads. If the response is not valid JSON, status and headers still contain the information needed to slow down.
2. Do you parse Retry-After as milliseconds or one fixed format?
Retry-After can be a non-negative number of seconds or an HTTP date. Parse both forms, use the response Date when practical, handle clock skew conservatively, and reject invalid values. Do not interpret 120 as milliseconds, and do not retry early because a valid delay exceeds an in-process timer limit.
RFC 9110 permits both forms:
Retry-After: 120
Retry-After: Tue, 13 Oct 2026 03:00:00 GMT
For a number, wait at least that many seconds. For a date, calculate against the server-provided Date when available, then fall back cautiously to the local clock. Clamp a past date to zero but consider a safety margin for latency and clock disagreement.
When the field is missing or malformed, follow the provider's documented fallback. GitHub supplies separate guidance involving its reset header or an initial delay for secondary limits.
If a valid wait is longer than the job deadline, do not occupy a scarce worker. Save the next-attempt timestamp and defer the job. Cancellation must interrupt waiting so shutdown or user action does not leave hidden retries alive.
3. Do all workers retry after the same fixed delay?
Fixed delays synchronize clients: a fleet receives 429 together, sleeps for the same interval, and returns as another spike. Use exponential backoff with jitter when no authoritative server delay exists. Cap the delay, attempt count, and total elapsed time so recovery does not become an endless background traffic source.
Google's current quota error guidance describes truncated exponential backoff with jitter:
delay = min(maxDelay, baseDelay × 2^attempt + randomJitter)
Follow the provider's algorithm when one exists. Before adding a loop, verify whether the installed SDK already retries eligible responses. Nested retries multiply attempts and hide the real waiting time.
Count failed calls against the same budget because unsuccessful requests may consume quota too. Record attempt number and cumulative elapsed time. When the budget expires, return the error, move durable work to a delayed queue, or require operator action.
Mutating retries also need safety. Pair backoff with the JSON API idempotency checklist so a lost response cannot turn into duplicate orders.
4. Is concurrency controlled per worker instead of per quota scope?
Limiting each process independently does not protect a quota shared by many pods, users, or jobs. Coordinate concurrency and pacing at the scope the provider enforces, such as token, account, endpoint, or resource family. Use a shared limiter or scheduler and reserve capacity for urgent interactive requests when appropriate.
Ten workers allowing ten concurrent requests still produce 100 calls. Autoscaling can increase that number precisely when latency is rising. A local semaphore helps, but it is not a global quota strategy.
| Limit dimension | Useful control |
|---|---|
| Requests per token | Token-scoped queue or bucket |
| Concurrent requests | Shared semaphore |
| Endpoint cost | Weighted request units |
| Write limit | Separate mutation lane |
| Daily quota | Admission control and forecasting |
GitHub recommends avoiding concurrent requests when managing secondary limits and documents different costs for some request types. Limits can change, so make pacing configuration adjustable without deploying code.
Fairness matters. One bulk export should not starve checkout or login traffic sharing credentials. Give jobs priorities and per-tenant ceilings, then monitor queue age so reduced throughput becomes visible before users report timeouts.
5. Does a rate-limited page restart the whole JSON export?
Resume paginated work from the last durable cursor or next link instead of restarting after a 429. Persist progress only after that page's records are safely committed, deduplicate by stable identity when needed, and retry the same page under provider guidance. Restarting wastes quota and may duplicate processing.
If an export reaches page 800 and restarts, it rereads 799 pages and consumes the capacity needed to progress. Save the server-provided next link or opaque cursor with job state.
Checkpoint records and the cursor in one transaction where possible. If data commits but the cursor does not, a retry may repeat the page, so writes need safe upserts or deduplication. If the cursor commits first, a crash can skip data.
Do not build the next URL by incrementing fields you do not own. Follow the navigation contract. Our JSON API pagination mistakes guide covers ordering, opaque cursors, stop signals, and concurrent writes.
Apply one global retry budget rather than resetting attempts inside every helper. A large job with three retries per page can otherwise create millions of extra calls during provider degradation.
6. Can retries outlive the user's deadline or business value?
Give each operation a total deadline and retry budget, not merely a timeout per HTTP attempt. Stop when the result is no longer useful, the caller cancels, or the next permitted attempt falls beyond the deadline. Defer durable background work instead of holding connections and workers through long rate-limit windows.
A five-second timeout repeated ten times lasts far longer after sleeps are included. Pass one deadline through every layer and calculate remaining time before starting a request or delay.
- Interactive reads may fail quickly with a clear retry time.
- Background sync can move to a durable delayed queue.
- Mutations require idempotency and outcome reconciliation.
- Low-value enrichment may be dropped when its budget expires.
- Compliance exports may pause while preserving checkpoints.
Use circuit breaking carefully. Its state should match quota scope; one tenant's exhausted token should not necessarily stop another tenant using separate credentials.
Return useful JSON errors stating whether work was deferred, when it may resume, and the job identifier. Do not expose provider credentials, internal thresholds, or stack traces.
7. Can you explain which quota caused the slowdown?
Record safe structured fields for every rate-limit decision: provider, endpoint family, credential alias, status, attempt, selected delay, header source, remaining retry budget, and next-attempt time. Track throttled calls, retry outcomes, queue age, and abandoned work. Without evidence, teams often add concurrency and worsen the incident.
{
"event": "api_rate_limited",
"provider": "example",
"resource": "search",
"status": 429,
"attempt": 2,
"delayMs": 4380,
"delaySource": "exponential_jitter"
}
Log the decision, not the sensitive response. Use an internal alias for credential scope and remove secret query values.
For background, rate limiting describes controlling traffic rate. It does not define a provider quota, so technical behavior here relies on HTTP specifications and current vendor documentation.
The proposed IETF RateLimit fields remain an Internet-Draft in October 2026. Treat them as evolving work and implement what your provider documents today. Do not assume headers will remain present or consistent across endpoints.
Run failure tests before production. Simulate a numeric Retry-After, an HTTP-date value, a missing header, a malformed header, repeated 429 responses, cancellation during sleep, and two workers sharing one credential. Confirm that no attempt starts early, deadlines stop work, queued jobs retain their checkpoint, and logs explain the chosen delay. Also test a successful response after recovery so the limiter releases capacity gradually instead of unleashing every waiting request at once. A controlled ramp protects both the provider and your own database from a sudden backlog spike. Keep these fixtures independent of real provider quotas; tests that deliberately exhaust a live account are slow, expensive, and can disrupt unrelated integrations.
These seven checks form one review: detect the real signal, parse timing correctly, add jitter, coordinate concurrency, preserve pagination progress, bound retries, and observe decisions. That turns throttling into scheduled work instead of a self-sustaining retry storm.
For Microsoft Graph clients, add these JSON batch tests for partial failures and retry reconstruction. A successful batch envelope can still contain throttled operations that need individual scheduling.