If you are spending engineering hours wiring an LLM router into your application, you may be routing your budget into a wall. A new study from Indiana University at Bloomington evaluates four open-source model routers under a common protocol and finds that a content-free fixed-tier policy, which always sends every request to the same mid-tier model, matches or nearly matches the best router on every benchmark tested. The study, published on arXiv on September 11, 2026, tested 290 frozen tasks against 2,610 candidate model outcomes and found no statistically significant task-specific advantage from any router on any benchmark. The gains that did appear track the mix of model tiers a router selects, not whether it picks the right model for the right task.
LLM model routing promises to cut costs by sending easy prompts to cheap models and hard prompts to expensive ones. The idea is intuitive: why pay for GPT-class inference when a smaller model would do? Vendors like OpenRouter, now owned by Stripe after a $7 billion acquisition that made AI model routing pay, have built businesses on this premise. But the new evidence suggests the routing layer itself may not be earning its keep, at least not in the open-source implementations tested.
What exactly did the researchers test?
The team evaluated four open-source routers spanning three routing paradigms. RouteLLM uses trained machine-learning classifiers to predict whether a cheaper model can match a stronger one on a given prompt. LiteLLM Router uses static programmatic rules, load balancing, and cost-limit fallbacks without inspecting prompt content. vLLM Semantic Router and Aurelio Semantic Router both embed incoming prompts and match them against databases of historical queries or intent labels.
The researchers ran all four against four benchmarks: RouterBench (which replays logged outcomes), BFCL v4 (function calling), tau2-bench (tool-use agent tasks), and WebArena (web interaction tasks). Unlike RouterBench, which replays pre-recorded results, the other three benchmarks executed live model calls through a single shared evaluation harness, ensuring every router faced identical conditions.
The candidate model pool was locked at two tiers: a stronger tier and a cheaper tier. Every router could only choose between these two options for each task. This design matters because it isolates the routing decision from the model pool question. If a router cannot beat a fixed policy with only two models to choose from, adding more models is unlikely to rescue it.
The headline result: three of the four routers emitted constant or near-constant tier assignments, meaning they effectively picked the same model for almost every prompt. Only vLLM Semantic Router varied its selection materially based on prompt content, and it had the highest observed success rate on none of the four benchmarks.
How badly do the routers lose to a fixed-tier baseline?
The fixed-tier baseline called Always-Mid, which sends every request to the stronger tier, reproduces the aggregate-leading router Aurelio Semantic Router on every benchmark. It matches Aurelio exactly on RouterBench, BFCL v4, and WebArena, and trails by just 0.003 on tau2-bench. The chart below shows the gap, or rather the near-total absence of one.
The wider set of fixed-tier heuristics, including Always-Strongest and Always-Cheapest, matches or exceeds the highest observed router success on three of the four benchmarks. In other words, the best-performing strategies in this study required zero prompt inspection and zero machine learning.
The statistical analysis is equally blunt. For vLLM Semantic Router, the only router whose assignments varied with prompt content, task-level superiority tests detected no statistically significant task-specific success advantage over a content-blind allocation with the same tier shares. Equivalence to a share-matched random allocation was established only on WebArena, and only at the protocol-declared plus-or-minus five percentage-point margin. At a tighter plus-or-minus three-point margin, even WebArena remains inconclusive. RouterBench, BFCL v4, and tau2-bench could not establish equivalence at either threshold, but they also showed no advantage for the router.
Why should builders care about this finding?
If you are architecting an AI application right now, you probably face a routing decision. The vendor pitch is seductive: a routing layer sits in front of your model calls, classifies each prompt by difficulty, and routes it to the cheapest capable model. The savings compound across millions of calls. The problem is that this study, alongside earlier work from the same research group showing that many routing methods exhibit similar performance under unified evaluation, suggests the routing intelligence may not exist in the open-source tooling you can deploy today.
What this means for you concretely:
- Your routing layer may be a no-op. If your router defaults to sending most prompts to the same tier, you are paying infrastructure and latency overhead for a decision you could hardcode in a config file. Audit your router's actual selection distribution before assuming it is doing intelligent work.
- Cost savings may come from tier selection, not task targeting. The study shows that aggregate success rates track the proportion of tasks sent to the stronger tier. A router that happens to select 70 percent strong-tier and 30 percent cheap-tier will look different from one that selects 50/50, but the difference is composition, not intelligence.
- Benchmark claims from router vendors need a fixed-tier baseline. Any router evaluation that does not include content-free fixed-tier policies and share-matched controls is not measuring routing intelligence. It is measuring model selection frequency. If a vendor shows you a routing benchmark without an Always-Mid or Always-Strongest baseline, ask for one.
- Live evaluation matters. RouterBench replays logged outcomes, which means it cannot catch distribution shift between the logged data and your traffic. BFCL v4, tau2-bench, and WebArena executed real model calls, and the results were no more favorable to routers.
This connects to a broader pattern we have tracked: the gap between what AI infrastructure promises and what it delivers under controlled testing. As we noted in our coverage of LLM performance drift and benchmark variance, benchmark scores move for reasons that have nothing to do with model capability. Router benchmarks are even more vulnerable to this illusion because they conflate model selection with routing intelligence.
Should you rip out your router today?
Not necessarily. The study authors are careful to scope their claims to the specific configurations, candidate pool, and frozen benchmark samples they tested. They do not claim that routing as a concept is useless. They claim that these four open-source routers, under these conditions, do not demonstrate task-specific targeting that beats a fixed policy.
Several caveats narrow the result. The candidate pool had only two tiers. Routers that choose among five or seven models might show more variance, and that variance might translate into real savings. The benchmarks cover function calling, tool use, and web interaction, but not every domain. A router trained on your specific traffic distribution, with your specific cost function, might behave differently from a general-purpose open-source tool.
Still, the burden of proof has shifted. If you are building or buying a routing layer, the baseline you need to beat is not another router. It is a one-line config that sends everything to your mid-tier model. Here is what that looks like in practice:
# The baseline that beat three of four routers
# and matched the fourth within 0.003
MID_TIER_MODEL = "your-mid-tier-model-id"
def route(prompt: str) -> str:
return MID_TIER_MODELIf your router cannot measurably outperform that function on your own traffic, you have your answer. The study provides a protocol for running this test: lock your candidate pool, freeze your task set, record the selected-tier distribution, and run a share-matched permutation test against the fixed baseline.
The research also echoes a finding from a separate benchmark study, LLMRouterBench, which evaluated routing methods under a unified framework and found that several recent approaches, including commercial routers, fail to reliably outperform a simple baseline. Two independent research groups, using different evaluation frameworks, have now arrived at the same conclusion: the routing emperor may have fewer clothes than advertised.
What should you do before your next infrastructure decision?
Start with the audit. Pull the last 10,000 requests your router handled and check the tier distribution. If 95 percent went to the same tier, your router is already a fixed-tier policy with extra steps. If the distribution is more balanced, run the share-matched test: randomly assign the same proportion of tasks to each tier without looking at prompt content, and compare success rates. If the random allocation matches or beats your router, the routing logic is not adding value.
For teams building new pipelines, the pragmatic path is to start with Always-Mid or Always-Strongest as the default and add routing only when you have evidence it helps on your specific workload. This is the software engineering equivalent of the paper's finding: complexity without measured benefit is just cost.
For teams already invested in a routing layer, the paper does not say your investment is wasted. It says you cannot know whether it was worth it without running the right control. A fixed-tier baseline and a selected-tier distribution are the two controls the study authors identify as necessary. Without them, any apparent router advantage is uninterpretable.
The open question is whether proprietary routers, trained on production traffic at scale, can clear the bar that open-source routers failed to clear. Stripe's OpenRouter acquisition suggests that routing as a business is valued highly. Whether that valuation reflects genuine routing intelligence or simply the convenience of a unified API across model providers is a question this study cannot answer, but it is the question buyers should be asking.
The cost of complexity you cannot measure
A router that always picks the mid-tier model is free. It costs zero engineering time, adds zero latency, and introduces zero failure modes. A router that picks intelligently costs developer hours, adds a network hop, and can fail silently when its classification model drifts. The only reason to pay that price is measurable, task-specific advantage. This study looked for that advantage in four open-source routers across four benchmarks and found none. The routers were not broken. They were just doing what a config file could do, and charging you complexity for the privilege.
Sources
- arXiv: Task- and Session-Level Model Routing: A Common-Interface Hybrid Evaluation of Four Open-Source Routers Across Four Benchmarks
- arXiv HTML: Full HTML version with detailed results tables
- ACL Anthology: LLMRouterBench: A Massive Benchmark and Unified Framework for LLM Routing
- Data Today: Stripe's $7B OpenRouter buy makes AI model routing pay
