Model routing
A step that picks, for each request, which model handles it, using a fixed rule or a model that judges the request.
Google for Developers (Machine Learning Glossary, 2026) says: “The algorithm that determines the ideal model for inference in model cascading. A model router is itself typically a machine learning model that gradually learns how to pick the best model for a given input. However, a model router could sometimes be a simpler, non-machine learning algorithm.” In practice, many routers are fixed rules. Cascading is a different method: it tries a cheap model first and moves up only if the answer isn't good enough.
Also called
- LLM routing
- Model cascading
Kind of
- LLM gateway An intermediary that your company's calls to models route through; when it holds the only API keys, it is both a control point and a log of every call.
Often confused with
- Router agent An agent which has responsibility to sort incoming requests and pass them to the appropriate agent.