Databricks、Unity AI Gateway でスマートルーティング導入しコスト削減

The price and performance frontier for coding tasks features a huge diversity of models and harnesses: in 2026 alone, we’ve seen 33 new models released. In our prior post about benchmarking against the Databricks codebase, we found that models cluster into capability tiers and that much everyday work (e.g., flipping a flag, a single-file edit, a well-scoped bug fix) did not require the most expensive models.

So how do you reduce AI coding costs without sacrificing developer productivity? One of the biggest opportunities is matching each task to the right model instead of defaulting every task to the most capable (and most expensive) option. Just leveraging lower cost models can save you 50%+, but it’s incredibly daunting for users. With a proliferation of great models and capable harnesses, coding agent users are constantly faced with choice overload. Instead of wasting time trying to select the best model for every task, many are setting the most capable at the highest effort and moving on. Instead of asking users to choose or stunting productivity with hard caps, we knew we needed to innovate.

That’s why we’re launching the next major cost control in Unity AI Gateway: Smart Routing, now available in Beta. Unity AI Gateway provides a central place to get access to AI, manage spend, and enforce controls across your entire enterprise, and Smart Routing adds intelligent optimization by automatically matching tasks to the right model based on complexity. Smart Routing works directly in Claude Code and Codex, allowing you to optimize the tools developers already use.

And we’re going beyond model routing. With Omnigent, our meta-harness for coding agents, teams can leverage the full power of Smart Routing by optimizing across both models and coding harnesses, giving developers the right combination for the task without having to choose it themselves.

The results speak for themselves: On internal coding workloads, Smart Routing outperformed any single model at just 65% of the cost per task of a leading model like Opus 5. On public benchmarks, Smart Routing matched Opus 5 on performance at *less than half the cost*.

Here’s what we learned:

• Most of the win comes from using cheaper models for simpler tasks. We see a wide diversity of tasks internally, and most of them don’t need a premium model. We configured the router to select cheaper models for simpler tasks, but to “escalate” complex work that requires frontier-model performance.
• We saw good results using only the information available at the start of the task (description and metadata). We didn’t provide the answer, tests, or anything about the repository, and Smart Routing was still able to choose effectively.
• There’s still substantial room for improvement in effectively sizing work complexity and escalating when needed. A router with perfect foresight would beat every single model at a fraction of what ours spends, and in real-life sessions, it can also be helpful to reassess midway through whether the task has gotten more complicated. Closing this gap is both a research and harness design problem. We need to learn from real user feedback to improve.

Let’s walk through how we built this.

How does intelligent model routing work?

Intelligent model routing selects the model best suited for a task based on factors like complexity, capability and cost. For coding agents, an important decision is when that routing should happen.

There are generally two approaches to routing:

• Per-request routing: In a given session, some teams have explored routing each request purely based on the complexity of the prompt for that message. The challenge is that, at scale, costs are dominated by cache hit rate. Having a high cache hit rate requires routing consecutive turns to the same model (and, currently, to the same effort level for the popular models).
• Task-aware routing: At the beginning of the session, you start by assessing the complexity of the task and then suggest a model and harness for it that you stick to for the duration of the session. This preserves the cache-hit rate and provides an opportunity for future optimization, such as upgrading or downgrading as needed when the cache becomes stale (e.g. when there is a compaction event).

How our Smart Router works

We opted for task-aware routing to preserve cache efficiency while matching each coding task to the appropriate model and harness. The most interesting problem is judging how difficult a task is before starting it. We wanted to start simple, so our router currently uses a single policy and applies it to every task the same way.

First, we classify the task. For this, we use a cheaper, low-latency model that reads the task description and labels it with a handful of semantic fields: what part of the system changes, what code evidence the prompt carries (a snippet, a traceback, or nothing explicit), how it appears to be failing, how localized the fix looks, and what kind of project it belongs to. From these, the router derives a task-type family and a language family. Using a frontier model would tax every request (even the simple ones we want to save on), so the extractor is intentionally small and fast.

Then, we triangulate on which model class is best. The router defaults to a medium-sized model and uses the labels to move in either direction, escalating to a more expensive model when the task demands frontier-level capability and knowledge or delegating down to a cheaper one when it does not. This means the single policy can leverage a whole suite of models.

The early results are promising. Against our own internal benchmark, which no labs have had access to, we saw 35% savings. Against public coding benchmarks that demonstrate our results generalize, we achieved 56% cost savings. We expect to see this grow as we learn more about our own use cases and with our design partners.

How do you route coding tasks across models and harnesses?

Smart Routing handles the routing decision, but then you need to be able to act on it. It works natively inside of Claude Code and Codex, but for coding agents, we see better performance by choosing not only the right model, but also the right coding harness. Helping engineers take advantage of the chosen model and harness requires a layer sitting above the individual coding sessions to orchestrate across them. This is why we built Omnigent.

Smart Routing is implemented in Omnigent at two levels:

First, developers using Omnigent can select Smart Routing instead of manually choosing a specific coding harness. Omnigent then automatically selects both the harness and model for each task, with model routing powered by Smart Routing in Unity AI Gateway. This design gives developers and admins the freedom to provide customizations, such as org-level guidance or the option to use previous conversation history, without changing the client every time.

This also means all sub-agent launches go through the Smart Routing API, allowing sub-agents to leverage a different harness and model. The user's initial prompt is often underspecified and hard to judge in terms of complexity, so sub-agents allow you to adjust new work based on new information, with a fresh cache and clear instructions. A single task can experience nuanced routing decisions across planning and parallel sub-agent work (e.g., you can route large codebase summarization tasks to cheaper models while designing the architecture with more expensive ones), leading to even more substantial savings.

How do you evaluate whether model routing is working?

Effective model routing needs to optimize for both cost and developer productivity, and not only cost alone. It’s critical to have feedback signals since routers are still early technologies that will need substantial iteration. Our first step here was to log all coding session traces for later evaluation. We want to consider both cost and developer experience – we don’t want to optimize cost at the expense of productivity.

With Unity AI Gateway, traces for coding agents can be recorded into Unity Catalog. This is extremely sensitive data, and it needs to stay governed by sophisticated tagging and access policies in most mature enterprises.

We used both AI models and human review to analyze traces to evaluate changes to the router. When we ran this analysis on our own sessions before routing, we observed that a large share of sessions were spending frontier-model money on work that did not need it, simply because the default model was the most expensive. In practice, the way to validate that the router is useful is to continuously monitor the following metrics:

• Breakdown of sessions by models
• # sessions that are completed by the routed models end-to-end
• Amount of dollar savings from routing

Where we are taking intelligent model routing next

We believe this is an area with significant opportunity and plan to continue conducting substantial research here. It’s still early, so we have a lot to learn.

Our first challenge has been unreliable benchmarking data that doesn’t match real user behavior. Benchmark tasks are unusually well-behaved, with each arriving as a self-contained statement of work. While routers perform well on such tasks, real sessions often are not like that at all.

• Opening prompts are rarely precise, since what a developer types first is a symptom or a rough intention rather than a specification, and our router reads that first message and commits.
• Sessions get reused, so the decision that was right for the first request can be wrong by the fourth, and nothing asks the router again.

So we’re researching a few new directions to gather more information and try new techniques:

• Start where task scoping is free. PR reviews, sub-agent launches, batch migrations and scheduled jobs are fully specified at the outset because a machine wrote the task statement. Routing works on this class today without changing anyone's habits, which is why we are deploying here first.
• Route after a few turns, not on the first one. For interactive work, the opening prompt is the worst available moment to decide, and nothing forces us to decide there. Instead, it can be useful to let a cheap model handle the initial exchange and ask clarifying questions, then route the task once it has taken shape. It is better to explore with a fast and small model, so this should improve cost and experience together.
• Make sessions smaller. Sessions that stay on one task route better and cost less, and tooling can encourage that by making a fresh session the obvious move when the subject changes.
• Make switching cheap. All of the above need mid-session model changes to be affordable. In today’s world, with costs dominated by cache-hit-rate, switching mid-session is untenable at scale. Context compaction is the natural seam, since a cache miss is already happening there (this is how Cognition’s Devin Fusion does it). Over time, we want the routing layer to explicitly price cache misses rather than bake that into how we use the router.

Routing is usually pitched as a way to spend less. While most of our wins come from paying lower prices for easy work, the same machinery can help us decide when to spend more to get a better outcome. Valuemaxxing cuts both ways: take the cheap model when it suffices, and be confident in spending more when the value justifies it.

Coding tools often incentivize users to consume more and more tokens, but what we really want is to optimize productive *output* per dollar, not tokens. Picking a cheaper, faster model when it suffices doesn’t just save money. It also saves time, and it keeps scarce frontier capacity available for the tasks that genuinely need it.

Try Smart Routing today

Smart Routing is now available in Beta through Unity AI Gateway. It automatically routes coding tasks to the right model based on complexity, helping teams achieve frontier-level performance with 30%+ cost savings by selecting the best model for every task. For teams looking to reduce AI coding costs without limiting developer choice or productivity, Smart Routing provides an alternative to manually selecting models or relying on blunt spending caps. And with Omnigent, you can extend intelligent routing across models and coding harnesses.

To get started, visit our docs pages:

• For model routing: Enable Smart Routing through Unity AI Gateway
• For model + harness routing: Use Smart Routing with Omnigent (v0.9.0+)

Learn more about Unity AI Gateway by visiting our website.