When an AI provider is overloaded, your request gets an error instead of an answer, usually 429 (too many requests) or 503 (service unavailable). Retry a few times with growing pauses, then switch to a backup model. Tell the user when that switch changes price or quality.

This guide is for people who build apps that call an AI model, and for anyone curious why their AI tool said "busy, try again". It reflects the error codes and provider docs as of October 2026.

See it in actionRetry with pauses, then switch to a backup
The main model returns 503, is retried after 3 and then 8 seconds, and still fails. The chat says it is switching to a backup from another provider. Illustrative example. Use the playback controls or the thin timeline at the top to explore.

In short:

  • 429 usually means a rate limit. It often clears within seconds.
  • 503 and "overloaded" mean the provider lacks capacity for everyone.
  • Back-to-back retries waste attempts. Pause a few seconds between them.
  • A backup from another provider survives an outage that hits the first one.

Busy errors come in three kinds

Not every error means "busy", and the code tells you which kind you got. Here is how the common codes compare:

Code What it usually means Retry?
429 Your rate limit or quota hit Yes, after a pause
503 / 500 Provider error or short on capacity Yes, a few times with backoff
529 Anthropic's "overloaded" code Yes, with backoff
400 / 403 Bad request or no access No, fix it

Google's Gemini troubleshooting guide lists RESOURCE_EXHAUSTED for 429 and UNAVAILABLE for 503. Anthropic's API errors page says 529 can happen "when the API experiences high traffic across all users".

One trap: a 429 is not always temporary. If the message says your prepaid credits ran out, or a daily quota is used up, waiting seconds will not help. Read the message before you retry.

Rate limits are per minute, so wait seconds

Rate limits are counted per minute or per day. A per-minute limit clears within seconds, so a short pause often fixes a 429. Gemini documents requests per minute, tokens per minute and requests per day. Going one request over the per-minute limit fails that request, not the next minute's.

Many 429 responses say how long to wait. The HTTP Retry-After header carries it, and Google adds a RetryInfo detail. Honour it when it is short. If it asks for minutes, switch to a backup instead of making the user stare at a spinner.

Back-to-back retries burn every attempt

Retrying immediately is a common mistake. Two retries fired within one second land inside the same capacity spike and both fail.

We made exactly this mistake. Our planner used to retry twice with no pause, so a spike of a few seconds burned both attempts and the user got "try again later". The fix was pauses of 3 and then 8 seconds before giving up on the main model.

The standard pattern is exponential backoff with jitter: double the wait each time and add a little randomness so thousands of clients do not retry in sync. The AWS Builders' Library article on backoff and jitter explains why the randomness matters.

Flutter apps can retry with the retry package

Flutter apps often call Gemini through Firebase AI Logic. There, the retry package adds backoff with jitter.

import 'dart:async';
              import 'package:firebase_ai/firebase_ai.dart';
              import 'package:retry/retry.dart';
              final model =
                  FirebaseAI.googleAI().generativeModel(model: 'gemini-3.8-flash');
              
              const _retry = RetryOptions(
                maxAttempts: 4,
                delayFactor: Duration(seconds: 1),
                maxDelay: Duration(seconds: 10),
              );
              
              Future<String?> ask(String prompt) => _retry.retry(
                    () => model
                        .generateContent([Content.text(prompt)])
                        .timeout(const Duration(seconds: 30))
                        .then((r) => r.text),
                    retryIf: (e) => e is ServerException || e is TimeoutException,
                  );

The retryIf line matters most. ServerException is thrown when the server failed to generate content, so it is worth retrying. A bad prompt or a missing permission will fail the same way every time, so it should not be retried.

A backup model should come from another provider

When retries run out, a second model can answer. Where it comes from matters. A cheaper model from the same provider is fine for a rate limit, because limits are usually counted per model. In a real outage, a model from the same provider can fail too.

Two more rules from our own logs:

  • Count every fallback: Fallbacks are invisible by design. Without a counter you cannot tell a bad model from a busy one.
  • Keep the cooldown short. A shared "primary is down" timer can push every user onto the fallback for minutes.

In our backend the cooldown is shared across users, so failing over on the first 429 would move everyone to the fallback at once. That is why the main model is retried in the same call first, and only repeated failures start a cooldown.

The circuit breaker pattern describes this "stop calling, then probe again" behaviour. Silent fallbacks shows how to count switches.

How GrowApps handles a busy provider

GrowApps uses two different responses, because the two problems are different.

  • Rate limit (429): The main model is retried up to twice in the same call, honouring the provider's wait if it is 8 seconds or less. If it still fails, calls go to a smaller Gemini model for a cooldown of one to five minutes, then return to the main model.
  • Provider busy (503 or overloaded) during planning: The main model is retried twice, with 3 and 8 second pauses. Then the chat says it will switch to a backup model from a different provider, and how much more that costs in percent. Tapping Stop within 15 seconds keeps waiting instead.

The backup covers that one message only. Your next message goes back to the main model. You can see how a build runs on the GrowApps home page.

Try a retry drill in ten minutes

Wrap your AI call in the retry above and add onRetry: (e) => debugPrint('retry: $e'). Set the timeout to 1 millisecond and run it on a test project, because the timed-out requests may still reach the server. The log should show three retries with growing gaps, then the call should throw. Catch that error and show a clear message instead of hanging. Next, read why one model per job beats one model for everything.

Frequently asked questions

Does "the model is busy" mean I ran out of credits?

Usually not. Busy messages are normally a provider capacity spike that clears in minutes. A credit problem says so in the message and does not clear by waiting.

Why not always use the backup model?

It costs more and behaves differently. Each model has its own habits with tools and long context, so we keep the model we test most for normal work.

Sources and further reading