# LLM gateway

An intermediary that your company's calls to models route through; when it holds the only API keys, it is both a control point and a log of every call.

Keep the two benefits separate: control (key custody, nothing routes around it) and observability (logs of what goes into and comes out of each call). Most gateways also give one API across model providers, with failover and cost tracking.

Routing is a capability of the gateway: deciding in real time which model a request goes to. The decision can be a deterministic rule (size, cost, who is asking) or a model that classifies the request. It usually trades off cost, latency and capability: send easy work to a cheap model, escalate hard work to a frontier model. The catch is that the router has to guess how hard a request is before the work is done. Router and gateway can be separate, but splitting them is extra work most companies shouldn't take on.

Also called: LLM gateway, AI gateway, model gateway, LLM router, model routing

Often confused with: Agent gateway

Related: Model-agnostic

Source: https://redfinchadvisory.com/lexicon/llm-gateway (version 1.0, changed 2026-10-21)