API rate limiting and handling 429 responses

What does API rate limiting mean?

According to the IETF's HTTP standard, 429 Too Many Requests indicates that the user has sent too many requests within a given period. The response should include details explaining the condition. It may also include a Retry-After header indicating how long to wait before sending another request.

The standard does not specify how the server identifies the user or counts requests. The number of requests can be limited for a resource, across an entire server, or across a group of servers. Authentication credentials or a stateful cookie can be used to identify the user.

A 429 response must not be stored in a cache. However, the IETF does not require servers to use 429 in every case. During an attack or when too many requests come from a single party, sending this response to every request also places a load on the server. The server can take other steps, such as dropping connections.

Request rate and concurrent requests

Stripe applies its general API rate limit to the total API usage for an account. Some endpoints also have their own limits. In Stripe, each named API operation counts as a separate endpoint. Requests sent to the same endpoint with different object IDs count toward that endpoint's shared limit.

Stripe treats the concurrency limit separately from the request rate limit. A rate limit generally concerns the number of requests in a time interval. A concurrency limit counts requests in progress at a given moment. Requests that take a long time or require more processing can cause the concurrency limit to be reached.

Stripe gives list requests and requests that include expansions as examples. These requests generally require more processing and can take longer. Concurrency limits apply to requests across the Stripe API, not just to endpoints that also have a rate limit. Limits can differ for specific endpoints and accounts.

When a global-concurrency or endpoint-concurrency response is received, Stripe specifies reducing concurrent requests and retrying with exponential backoff and jitter. It also recommends measuring API request duration and monitoring timeouts to identify long-running requests.

Distinguishing the cause of a Stripe response

For requests that hit a rate limit, Stripe returns 429 and a Stripe-Rate-Limited-Reason header explaining the cause. The header value distinguishes whether the limit relates to overall API usage, a specific endpoint, or concurrent requests.

global-rate indicates that the account's overall rate limit has been exceeded. endpoint-rate points to the rate limit of the endpoint to which the request was sent. In these cases, Stripe specifies reducing the overall request rate or the rate of requests sent to the relevant endpoint, respectively.

global-concurrency means that the overall concurrency limit has been exceeded. endpoint-concurrency corresponds to the concurrency limit of a specific endpoint. Stripe's specified response is to reduce the total number of concurrent requests in the first case and the number of concurrent requests to the relevant endpoint in the second.

resource-specific indicates that a rate limit related to endpoints within a resource has been exceeded. Stripe gives subscription update and creation endpoints as examples. For this value, Stripe specifies reducing the total rate of requests sent to endpoints within the relevant resource.

According to Stripe, if a 429 response does not have this header, the response was not caused by a rate limit. One possible cause is a lock_timeout error. For some operations, Stripe locks an object to prevent concurrent operations on the same object from producing inconsistent results. If a lock held elsewhere cannot be acquired in time, the request can time out.

Stripe does not process these failed requests or assign them a request ID. If lock_timeout occurs frequently with multiple concurrent requests to the same object, Stripe specifies sending the requests sequentially or at a lower rate. Queuing changes to the same object can also reduce lock contention.

Determining the retry time

To handle rate limiting, Stripe specifies monitoring 429 responses and implementing a retry mechanism. In its recommended exponential backoff pattern, the wait time between retries increases. Adding randomness to the wait is intended to prevent many requests from being resent at the same time.

Stripe says traffic control can also be performed on the client side. Traffic can be limited at a general level, and the request rate can be reduced when heavy rate limiting is detected. One method Stripe gives for controlling API usage is a client-side token bucket algorithm.

The GitHub REST API applies a primary rate limit and a secondary rate limit. When the primary limit is exceeded, it returns a 403 or 429 response and the x-ratelimit-remaining value is zero. GitHub specifies that the request should not be retried before the time in x-ratelimit-reset.

A 403 or 429 can also be returned when the secondary limit is exceeded. In this case, the error message indicates that the secondary limit has been reached. If the response has retry-after, GitHub says to wait the number of seconds in that field before retrying. If x-ratelimit-remaining is zero, the wait lasts until the time specified in x-ratelimit-reset.

If neither of these conditions applies, GitHub specifies waiting at least one minute before retrying. If the request continues to fail because of the secondary limit, retry intervals are increased exponentially and an error is returned after a certain number of attempts. GitHub also states that continuing to send requests while rate limited can lead to the integration being blocked.

Reading GitHub response headers

GitHub reports the status of the primary limit in headers sent with responses. x-ratelimit-limit shows the maximum number of requests that can be made in the relevant time interval. x-ratelimit-remaining shows the requests remaining in the same interval, while x-ratelimit-used shows the requests already made.

x-ratelimit-reset gives the time when the current limit window will reset in UTC epoch seconds. x-ratelimit-resource shows which rate limit resource the request counted against. GitHub specifies using these headers to adjust the request rate and backoff time, but avoiding logic that depends on the remaining request count having an exact value.

GitHub processes API requests in multiple regions. Values can vary from response to response depending on which region processed the request. In the same limit window, a later response can have a higher x-ratelimit-remaining value.

GET /rate_limit provides an overview of resource groups for the authenticated user. A call to this endpoint does not count against the primary limit; it can count against the secondary limit. According to GitHub, response headers are the authoritative source for the current status and can differ from the values reported by the endpoint. When the two do not match, GitHub specifies relying on the response headers.

There is no way to query the status of the secondary limit. GitHub says this limit can be reached in cases such as too many concurrent requests, too many requests to a single endpoint, or too much content created in a short time. The secondary limit can also be applied for other undisclosed reasons.

Limits and scope

According to OWASP, handling API requests requires capacity such as network bandwidth, CPU, memory, and storage. Counting only the requests made within a given period does not directly limit the number of operations performed within a single request. In OWASP's GraphQL example, the endpoint has a rate limit and the uploaded image's size is checked; however, the number of times the uploadPic operation can be attempted in a single request is not limited. OWASP states that this can lead to server memory exhaustion.

OWASP says rate limiting should be configured according to business needs and that stricter rules can be applied to some endpoints. It also recommends limiting how many times or how often a single API client or user can perform a particular operation. One-time code validation and password recovery requests are examples of these operations.

OWASP also specifies server-side validation for parameters in the request body and query string. In particular, this includes the parameter that determines the number of records returned in the response. It recommends defining maximum values for incoming parameters and payloads, such as string length, the number of elements in an array, and uploaded file size.