A silent fallback swaps in a different AI model when the main one fails, without telling anyone. It keeps the app running, but quality, cost and behaviour change. Log which model served every call, count fallbacks, and alert when their share rises.

This guide is for developers running AI features in production, and for users who suspect their AI tool got worse overnight. It draws on incidents in our own builder between August and October 2026.

In short:

  • Transparent fallbacks leave little trace unless you add one.
  • Configuration, not fallback, caused our worst "fallback" scare.
  • An unpriced fallback model may bill nothing, silently.
  • Costlier backups deserve a notice. Cheaper brief ones may not.

Fallbacks are invisible by design

A good fallback is transparent: the agent gets its answer from another model, and the session carries on. Libraries such as LiteLLM offer this as a reliability feature.

That is also the problem. In our backend, the only trace of a fallback used to be one info line in a log. No counter, no chart, no alert. A session could run on the backup model end to end and nobody would know.

A missing counter hides real damage

In late August 2026 we found whole sessions served entirely by Gemini 3.5 Flash-Lite, the small model we use as a fallback. The quality drop was real. In one 10-turn session it took zero screenshots, although our instructions require them, and the app shipped with a layout overflow.

We fixed the gaps right away. Now the main model is retried in the same call before any switch. Our health endpoint also counts rate limits and fallbacks per model, plus which models are cooling down right now.

Check configuration before blaming fallback

Two days later we found that those sessions were not fallbacks at all. They belonged to projects configured to run every turn on the small model, a cost-saving lane for first builds.

The symptom looked identical: a session served only by the small model. The cause was a setting. We moved that lane to the main model, and corrected our own notes. Before blaming a fallback, check which model the configuration asked for.

Price every fallback model

A fallback model needs a price in your billing table like any other. If it has none, its calls may be charged at zero.

Our fallback code now carries a warning that the fallback must be a priced model.

Some fallbacks never serve at all

A fallback can also hide a model that never works. One cloud platform's OpenAI models rejected a stop-sequence field that our agent framework sends by default.

Every first call failed with a validation error, a fallback answered, and the run finished normally. The platform's metrics gave it away: invocations equal to client errors, with almost no output tokens. If you add a new model, confirm it actually served requests, not just that runs completed.

Log the served model on every call

The fix for invisible fallbacks is one log line per model call, with the model that answered next to the one you asked for:

{
                "requested_model": "gemini-3.8-flash",
                "served_model": "gemini-3.5-flash-lite",
                "fallback_reason": "429",
                "cached_input_tokens": 98210,
                "output_tokens": 412
              }

The served_model field matters most. Count it per hour, chart the share served by each model, and alert when the fallback's share jumps. Summing tokens per served model also tells you what each fallback costs.

Decide which switches users hear about

Not every fallback needs a message, but some do. Use the user's money and the expected quality as the test:

Fallback Costs user more? Tell them?
Brief switch to a smaller model No Optional, but count it
Switch to a pricier backup Yes Yes, before it happens
Model never served at all Varies Fix it, it is a bug

How GrowApps handles fallbacks

When our main model is rate limited, GrowApps retries it, then uses Gemini 3.5 Flash-Lite briefly during a cooldown, without a chat message. When the provider is overloaded during planning, the chat says it will switch to a backup from another provider. It also says how much more that costs, and that you can tap Stop to wait instead.

Every run records which models served it. See a build on GrowApps.

Find your fallbacks in ten minutes

Search your logs for the model name returned with each response, not the one you requested. Count responses per model for the last day. If any model you did not configure shows up, or your fallback serves more than a few percent, find out why before users notice. Then read what happens when a provider is overloaded.

Sources and further reading