Requests per minute
You can hit this even with tiny prompts if too many users send requests at once.
Quota and reliability
Rate Limits Are a Product Requirement, Not an Error Message.
AI API free tiers often limit requests, tokens, daily usage, or audio seconds. A reliable app needs retry behavior, queues, user messaging, caching, fallbacks, and billing alerts before public traffic arrives.
Last updated May 21, 2026. Rate-limit values change by provider, model, account tier, and region; verify current limits in official dashboards.
Rate-limit terms
Know which quota your app is hitting.
Requests per day
Daily caps matter for classroom demos, public links, cron jobs, and agent loops.
Tokens per minute
Long prompts, large retrieved contexts, and verbose outputs can exhaust token limits quickly.
Too many requests
A 429 should trigger backoff, queueing, fallback, or a clear user message instead of a blank UI.
| Problem | Why it happens | Fix | User-facing fallback |
|---|---|---|---|
| Demo breaks during judging | Shared traffic exceeds RPM or RPD | Use sample inputs, caching, and a backup provider | Show saved response and retry button |
| RAG app hits TPM | Retrieved context is too large | Reduce chunks, summarize context, add reranking | Ask user to narrow question |
| Agent burns quota | Tool loop or retry loop is unconstrained | Set max steps, tool-call budget, and approval gates | Pause task and request confirmation |
| Batch job stalls | Too many concurrent requests | Queue work and respect retry-after headers | Show progress and estimated delay |
| Costs spike after free tier | No usage alert or monthly cap | Add billing alerts, per-user limits, and logs | Throttle non-essential requests |
Implementation playbook
Design for quota before launch day.
Free-tier demos are safer when the app knows what to do when the API says no. Build this before sharing the link, not after the first failure.
Rate-limit checklist
- Log provider, model, request count, token count, latency, and status code.
- Read retry-after or equivalent rate-limit headers when available.
- Add exponential backoff with a maximum retry count.
- Queue long jobs instead of blocking the UI.
- Cache repeated prompts or retrieved context where safe.
- Add a fallback provider or saved response for demos.
- Set per-user limits and billing alerts.
People also ask
AI API rate-limit questions.
What does a 429 error mean?
It usually means the app exceeded a rate limit such as requests per minute, tokens per minute, or daily quota. Back off, retry later, queue work, or switch to a fallback.
How do I avoid rate limits in a hackathon?
Use fewer model calls, cache repeated outputs, keep prompts short, add fallback responses, and do not let agent loops run without a max step count.
Do token limits matter more than request limits?
For RAG and long-context apps, token limits can matter more because each request may include many retrieved chunks and a long answer.