What Is an AI Coding Agent Router?

What an AI coding agent router does

An AI coding agent router sits between a coding agent and the AI models it calls. The agent (Claude Code, Codex CLI, Cursor, AirCode or your own) sends its model requests to the router instead of straight to one provider. The router then decides, request by request, which model should answer.

That makes it different from a plain LLM gateway. A gateway mainly connects applications to models: one API, many providers, keys and logs in one place. A router adds a decision on top: which model is the right fit for this step of this task, given what it needs and what it will cost. For a longer look at the two ideas, see LLM gateway vs LLM router.

Why coding agents are a different workload

A chat assistant usually turns one question into one model request. A coding agent turns one task into many. A request like "fix the failing login test" can become a dozen or more model calls: plan the change, search the repository, read files, edit code, run the tests, read the output, fix again and summarise.

• Many requests per task. Each step of the agent's loop is a separate model call, so the model choice is made many times per task, not once.
• Large, repeated context. Every call can carry the system instructions, tool definitions, file contents and the conversation so far. Much of it is the same from one call to the next.
• Tool results pile up. Search results, file reads and test logs are fed back into the context and keep growing as the session runs.
• Loops and retries. When a fix fails, the agent tries again. Productive retries are normal; unproductive repetition burns tokens without moving the task forward.

Sending every one of those calls to the most capable model is simple, but much of the work, such as finding a function or summarising a log, doesn't need it. Sending everything to a small model saves money until a hard step fails. Routing is the middle path. Our guide to how coding agents use tokens goes deeper into where the tokens go.

Task difficulty, not prompt size

A common mistake is to treat a long prompt as a hard task. In a coding agent, the prompt is long almost all the time, because the repository and history come along. Renaming a variable inside a large codebase can carry a very large context and still be routine work. Designing a new service from a short description can carry little context and need deep reasoning.

A useful router therefore looks at what the step is trying to do, what reasoning it needs and how much context it must hold, rather than simply counting tokens. Context size still matters, because the chosen model has to fit it, but it is one input among several.

How model selection works in practice

Most routers expose one or more routing models: names you put in your tool's model setting that stand for "let the router choose". Behind that name, the router matches each request to an underlying model. Some routing models choose across providers; others keep requests inside one provider family for teams that prefer it.

Two habits separate careful routers from naive ones. First, consistency: once a task has needed a stronger model, dropping it back to a weaker one halfway through risks breaking work that was going well. Second, escalation: when a step fails a check, moving it up to a more capable model is often cheaper than letting a small model retry repeatedly.

Caching economics

Caching changes the arithmetic of routing, and it is easy to get wrong.

Prompt caching at the provider

Many providers bill repeated input that they have already cached at a lower rate than fresh input. Coding agents benefit a lot from this, because so much of each request repeats. The catch is that a cache belongs to a model. If a router switches to a different model with a lower list price, the new model may have to process the whole context fresh, and that request can end up costing more than staying put. A router should weigh the cache it already has, not just the price list.

Reusing earlier responses

Some routers can also return an eligible earlier response when the same request arrives again, instead of calling a model at all. That helps with repeated, deterministic work. It needs care: answers that depend on fresh state should not be served from cache.

Note. Caching is also why your tool's cache markers need to reach the provider. Claude Code, for example, uses Anthropic's cache_control field, and its documentation warns that a gateway that strips it makes the conversation bill as uncached input on every turn.

Fallback, escalation and agent loops

Agents run for a long time, so reliability matters as much as price. Providers hit rate limits and outages. Models occasionally return an empty response or a malformed tool call. A router that can retry on another eligible model, or repair what can safely be repaired, keeps the agent moving instead of handing it an error.

Loops are the other risk. An agent that keeps running the same search or making the same edit can consume tokens quickly while making no progress. Routers can watch for repeated operations and intervene, and budgets add a spending ceiling. Treat both as safety nets: budgets and loop guards are best-effort and don't replace watching long-running agents.

How to evaluate a router for your coding agent

Before you change your tool's settings, check that the router fits the way your agent actually talks to models.

• API format. Claude Code speaks the Anthropic Messages format. Current Codex CLI releases need an OpenAI Responses-compatible endpoint. Many other tools use OpenAI Chat Completions. The router has to offer the format your tool uses.
• Streaming. Coding agents stream responses. A router that buffers whole responses can make the tool stall.
• Tool calling. Tool definitions and tool results must pass through intact, in the shape the tool expects.
• Cache handling. Cache markers should survive the trip, and model switches should account for cache state.
• Overhead. Each step adds the router's processing time, so the overhead per request should be small.
• Controls and visibility. Look for budgets, usage logs per key and a clear way to choose or restrict models.
• Vendor position. Read what your tool's vendor says about gateways. Anthropic, for example, says it doesn't endorse or audit third-party gateways and doesn't support routing Claude Code to non-Claude models through any gateway.

How Air Router approaches it

Air Router is AirLLM's intelligent routing layer for AI coding agents and other LLM workloads. It routes each step of an agent's work to a suitable model, starting simple steps on fast models and escalating when checks fail, without moving a task to a weaker model mid-way. It considers existing prompt caches before switching models, reduces repeated file content and tool history before a request reaches the model, and is designed to interrupt unproductive agent loops.

You connect a tool by changing its base URL, key and model name: https://api.airllm.ai/v1 for OpenAI-style tools and https://api.airllm.ai/ for Claude Code. You choose a routing model such as airllm-auto (across providers), airllm-anthropic or airllm-openai; the routing models page explains each one. Step-by-step setup for each tool is in the Developer Guide, and the tool-specific guides cover Claude Code routing and Codex CLI routing.

Frequently asked questions

Related

Sources

• Claude Code docs — LLM gateway configuration
• Claude Code docs — LLM gateway protocol requirements
• Codex docs — gateway compatibility