[{"data":1,"prerenderedAt":78},["ShallowReactive",2],{"term-m\u002Fmodel-router":3,"related-m\u002Fmodel-router":60},{"id":4,"title":5,"acronym":6,"body":7,"category":40,"description":41,"difficulty":42,"extension":43,"letter":44,"meta":45,"navigation":46,"path":47,"related":48,"seo":54,"sitemap":55,"stem":58,"subcategory":6,"__hash__":59},"terms\u002Fterms\u002Fm\u002Fmodel-router.md","Model Router",null,{"type":8,"value":9,"toc":33},"minimark",[10,15,19,23,26,30],[11,12,14],"h2",{"id":13},"eli5-the-vibe-check","ELI5 - The Vibe Check",[16,17,18],"p",{},"Not every question needs your smartest, most expensive model. A model router looks at each incoming request and decides which model should actually handle it: a cheap fast model for 'what's the weather,' a big expensive one for 'debug this race condition.' It's a dispatcher for AI models instead of restaurant orders, sending each ticket to the cook who can actually make it well without wasting a specialist on toast.",[11,20,22],{"id":21},"real-talk","Real Talk",[16,24,25],{},"A model router is a component that classifies incoming requests and dynamically dispatches each one to the most appropriate model from a pool, balancing cost, latency, and capability. Routing logic can be rule-based (task type, prompt length), a lightweight trained classifier, or LLM-based triage, and it typically pairs with fallback logic so a failed or overloaded model doesn't take the whole system down.",[11,27,29],{"id":28},"when-youll-hear-this","When You'll Hear This",[16,31,32],{},"\"The router sends simple classification tasks to Haiku and reasoning-heavy ones to Opus.\" \u002F \"Our model router cut average cost per request by two-thirds.\"",{"title":34,"searchDepth":35,"depth":35,"links":36},"",2,[37,38,39],{"id":13,"depth":35,"text":14},{"id":21,"depth":35,"text":22},{"id":28,"depth":35,"text":29},"ai","Not every question needs your smartest, most expensive model.","intermediate","md","m",{},true,"\u002Fterms\u002Fm\u002Fmodel-router",[49,50,51,52,53],"Model Fallback","Cost Per Token","Inference","LLM","Latency Budget",{"title":5,"description":41},{"changefreq":56,"priority":57},"weekly",0.7,"terms\u002Fm\u002Fmodel-router","tQaSTD7ROf_dKftiUsfr36pH5NDv4B0loH788eiK068",[61,65,68,71,75],{"title":50,"path":62,"acronym":6,"category":40,"difficulty":63,"description":64},"\u002Fterms\u002Fc\u002Fcost-per-token","beginner","Cost per token is how much each token (input or output) costs with a given AI provider. Flagship models cost more per token than cheap ones.",{"title":51,"path":66,"acronym":6,"category":40,"difficulty":42,"description":67},"\u002Fterms\u002Fi\u002Finference","Inference is when the AI actually runs and generates output — as opposed to training, which is when it's learning.",{"title":53,"path":69,"acronym":6,"category":40,"difficulty":42,"description":70},"\u002Fterms\u002Fl\u002Flatency-budget","A latency budget is deciding upfront how many milliseconds each part of your AI pipeline is allowed to spend before the whole thing feels slow to a user.",{"title":52,"path":72,"acronym":73,"category":40,"difficulty":63,"description":74},"\u002Fterms\u002Fl\u002Fllm","Large Language Model","An LLM is a humongous AI that read basically the entire internet and learned to predict what words come next, really really well.",{"title":49,"path":76,"acronym":6,"category":40,"difficulty":42,"description":77},"\u002Fterms\u002Fm\u002Fmodel-fallback","Model fallback is your AI app's plan B, C, and D for when the primary model has a bad day: it's rate-limited, it's down, or it just times out.",1792528100760]