Rate Limits & Quotas

PAIS enforces rate limits at the API gateway (Kong) to ensure fair access across research groups.

Current limits

Limit	Value	Applies to
Requests per minute	60	Per API key
Tokens per minute	50,000	Per API key
Concurrent requests	4	Per API key

Note

Limits are subject to change as the platform grows. Contact the PAIS team for elevated limits on specific workloads (e.g., a large embedding batch job).

Rate limit responses

When a limit is exceeded, the API returns:

HTTP 429 Too Many Requests
Retry-After: 15

{
  "error": {
    "message": "Rate limit exceeded. Retry after 15 seconds.",
    "type": "rate_limit_error",
    "code": "rate_limit_exceeded"
  }
}

Handling rate limits in code

import time
from openai import RateLimitError

def call_with_backoff(client, **kwargs):
    for attempt in range(5):
        try:
            return client.chat.completions.create(**kwargs)
        except RateLimitError as e:
            wait = 2 ** attempt
            print(f"Rate limited, waiting {wait}s...")
            time.sleep(wait)
    raise RuntimeError("Exceeded retry budget")

For large batch jobs, use time.sleep(1) between requests to stay comfortably under the per-minute limit.

Research group quotas

GPU resources are also governed by Kubernetes ResourceQuotas at the namespace level:

Resource group	GPU quota	CPU quota	Memory quota
rg-compsci	2 GPU slices	8 CPU	32 Gi
rg-abi	1 GPU slice	4 CPU	16 Gi
rg-general	shared pool	shared	shared

These quotas apply to active compute workloads (notebooks, training jobs) — not to API calls through the inference endpoint.