Designing around third-party rate limits before they design around you

Every integration we've built against a marketplace or supplier API has a rate limit, and each one is different - requests per second, per minute, per day, sometimes a concurrency cap on top. The teams that struggle are the ones that treat the limit as an error to catch. The ones that don't treat it as a fixed input to the design, like disk space or memory.

Catching 429s is the fallback, not the plan

Reacting to `429 Too Many Requests` means you've already sent too much, the provider is now refusing you, and depending on how strict they are you may be blocked for longer than the one request that tripped it. Backoff-on-429 has to exist, but if it's your only mechanism, throughput collapses under load - exactly when you need it.

Meter on your side, under the real limit

Put a limiter in front of the client - a token bucket or leaky bucket sized to something under the published rate, so you're not riding the ceiling. Every outbound call takes a token; no token, the call waits. This makes your request rate a property you control and can reason about, instead of an emergent result of how many workers happen to be running.

One limiter per provider, shared across workers

• The limit is per API key or account, so the limiter has to be shared state - Redis, or a single gateway process - not a per-worker counter that ten workers each think they own.
• Track the different windows the provider enforces (per-second and per-day, say) in the same limiter.
• If you hold multiple keys, key the limiter by credential and let work spread across them.

Prioritise, because not all calls are equal

When you're limit-bound, what you spend the budget on matters. A customer's order pull or a stock zero-out is worth more than the hourly catalogue refresh. Run separate queues with weights, or a priority field, so time-sensitive work goes first and bulk work fills whatever capacity is left.

Spread scheduled work out

Cron jobs that all fire at the top of the hour create a spike that trips the limit and then sits idle for 59 minutes. Jitter start times, chunk large jobs, and stretch them across their available window. Smooth and slow finishes ahead of spiky and blocked.

Back off properly when you do get limited

• Honour the `Retry-After` header if the provider sends one - it's them telling you exactly how long to wait.
• Otherwise use exponential backoff with jitter, so a fleet of workers doesn't retry in lockstep and re-trip the limit together.
• Feed the event back to the limiter so it slows every call to that provider, not just the one that failed.

The Timestamp Converter helps when you're lining up `Retry-After` values and rate-limit reset headers against your own logs; the JSON Formatter makes a provider's error body readable while you work out which window you hit.

A rate limit you designed for is a scheduling parameter. A rate limit you react to is an outage waiting for a busy day.

What's the best way to handle API rate limits?

Meter your own requests below the published limit with a shared token-bucket limiter, rather than waiting for 429 responses. Give one limiter to each provider account, prioritise time-sensitive calls over bulk work, and spread scheduled jobs across their window instead of firing them all at once. Keep backoff-with-jitter and Retry-After handling as the fallback.