Model Fallback
ELI5 - The Vibe Check
Model fallback is your AI app's plan B, C, and D for when the primary model has a bad day: it's rate-limited, it's down, or it just times out. Instead of showing the user an error, the system quietly retries with a different model or provider and the user never even notices. It's the same instinct as keeping a backup generator, except the power outage is OpenAI having an incident at 2am.
Real Talk
Model fallback is a resilience pattern where a request is automatically retried against an alternate model or provider when the primary one errors, times out, or is rate-limited, often configured as an ordered chain (for example, a specific model version, then the same model from a second provider, then a smaller local model). Libraries like LiteLLM and OpenRouter implement this natively, and fallback logic is frequently combined with a model router to also account for cost and capability, not just availability.
Show Me The Code
response = router.completion(
model="primary-model",
fallbacks=["backup-provider-model", "local-model"],
messages=messages,
timeout=8
)
When You'll Hear This
"The fallback chain kicked in during the outage, users never saw a single error." / "We added a local model as the last fallback so we're never fully down."
Related Terms
Circuit Breaker
Circuit Breaker is like the electrical circuit breaker in your house.
Exponential Backoff
Exponential backoff doubles the wait time between each retry. Wait 1s, then 2s, then 4s, then 8s.
LLM (Large Language Model)
An LLM is a humongous AI that read basically the entire internet and learned to predict what words come next, really really well.
Model Router
Not every question needs your smartest, most expensive model.
Model Serving
Model serving is the infrastructure that takes a trained AI model and makes it available as a fast, reliable API.