Enterprises spent an estimated $12.5 billion on foundation-model APIs in 2025 alone, prompting companies to search for ways to be smarter about their AI usage. A number of companies have started to provide LLM routing products that help enterprises save money by routing some of their LLM calls to cheaper models. Analysts expect the LLM routing market to grow 20–45% a year through the next decade, reaching an estimated $4–6.5 billion.

We wanted to test how fast our Metasystem can build the core technology behind this industry. It took 1 engineer 2.5 weeks from data collection to full results. Only 12 hours of that was the Metasystem optimizing the router.

Not every task requires the most expensive LLM; some questions are easy, and for those, cheaper models suffice. How do we automatically decide which questions should be sent to which models so that they are answered correctly, but with minimum expense?

The job of the LLM router is to figure out the best model for a query and then “route” it to that model. If the router is doing its job, it will save money by routing simple queries to cheaper models. Interestingly, if the router has learned correct, sufficiently granular rules, it can even do what might seem impossible: save money while producing better results. Because of each LLM’s own jagged frontier of capabilities, for some tasks a cheaper model outperforms an expensive one; the router just has to identify those tasks.

Because each company’s LLM usage distribution is unique, each will have its own mix of domains and types of questions, its own budget constraints, and models that they are committed to use (for example, pre-existing contracts with OpenAI or Anthropic). We can learn a bespoke router for any set of requirements. In every case, we use the Poetiq Metasystem to optimize the routing decisions.

In this post, we simulate three distributions, each from a different industry, with its own pattern of LLM usage:

1. Legal-R-US uses multiple frontier providers, making queries about legal, marketing, finance, HR and more.
2. Economics Inc has pre-commitments to use Anthropic models and wants to make the best use of tokens with this provider. Its queries are split roughly evenly among economics, coding and math.
3. Privacy-4-U, LLC works in the same domain as Legal-R-US but is very concerned about privacy — it only uses open-weights models.

Legal-R-Us is a large law firm. Its employees’ biggest use of LLMs is researching prior law and marketing its services. It also has some usage across HR/Finance/Medical as usual, and even a small software team that builds in-house tools.

Legal-R-Us has agreements with four providers and wants the best-quality answers per dollar across all of them: Claude Opus 5 (Anthropic), Grok 4.6 (xAI), Muse Spark 1.1 (Meta) and Gemini 3.5 Flash-Lite (Google).

Our Metasystem can create a routing harness targeting any cost-accuracy tradeoff point that a company may wish for. The results in Figure 1 show how the optimized routers perform at any price point. The best router is 20% cheaper than Claude Opus 5 with no statistical difference in accuracy.

For a large law firm this translates to savings of millions of dollars a year!

Additionally, we compared our results to OpenRouter’s Auto Router and to JEV (a classification model). We beat OpenRouter on both price and accuracy. With respect to JEV, note JEV’s poor performance – close to the bottom of all the tested models. From inception, our Metasystem was designed to work with many kinds of AI architectures, not just chatbot-style LLMs. Without any modification, we used it to optimize harnesses for JEV. When set to approximate JEV’s price, our harness easily surpasses JEV (63.6% @ $5.42) by a wide margin at a lower price (75.5%, $5.38). When set to prioritize accuracy, the JEV harness improvement was dramatic (86.1%, $8.75), although our fully optimized routers (that were not forced to use JEV) still dominated it.

Here, we visualize the decision process for the router. The animation shows a number of questions as they are being routed through the decision tree. Notice that the Metasystem added a “fallback” option to make an LLM call when it was uncertain. In practice, fewer than 1% of questions needed that extra LLM call, which makes the router cost $0 for 99% of the decisions.

The Metasystem learned that Muse and Grok are better on specific subcategories of the data, and the optimized harness routes to those. For example, it decided to route case-law questions to Opus, but found that for statute or regulation questions it can get 3% higher accuracy and a 20% cost saving by sending them to Grok. Overall it routes about 50% to Opus 5, just under 25% each for Muse and Grok, and a small amount to Gemini Flash Lite.

Economics Inc. pre-purchased a large block of Anthropic credits, is allowed to spend on any Anthropic model, and wants the highest-quality answers while stretching its token use as much as possible. It has selected Claude Opus 5 at three reasoning-effort levels (high, medium and low) and Claude Sonnet 5. Its work is largely technical and splits evenly across three areas.

We ran our Metasystem to select between Opus 5 and Sonnet 5 and decide what level of thinking effort is needed for the problem. The Poetiq Metasystem optimized a router that can now serve 15% more tokens with the same number of credits, while maintaining (slightly improving!) accuracy. The top router splits the questions between all options: high effort for about 40% of questions, medium for 37%, low for 22%, and Sonnet for the remaining 1%. Its decision method is similar to the previous company’s, but the features and decision algorithm it learned are fully specific to this distribution of data.

We also sent these queries to JEV. Again, JEV tends to focus on one model, sending ~90% of its queries to Sonnet, which it thinks will be the cheapest based on per-token price. However, Sonnet tends to overthink on these prompts and turns out to be quite expensive on average.

Privacy-4-U, LLC is very careful with privacy. It doesn’t want Anthropic or OpenAI to have access to its clients' private data. It only uses open-weights models. It chooses between DeepSeek V4 Pro, DeepSeek V4 Flash, Kimi K3, GLM 5.3 and GLM 5.3 Flash, a price range of roughly 30x from cheapest to most expensive. It works in the same six business functions as Legal-R-US, with a different mix: more finance and HR, less software.

The choices in routing for Privacy-4-U have drastic ramifications for final cost. Kimi K3 is the most accurate single model at 85.79%, for $23.9. GLM Flash costs twenty times less, but gets only 59.9%. The Poetiq Metasystem filled in the whole space between them. No single model offers anything between DeepSeek V4 Pro’s $12.3 and Kimi K3’s $24; the routers, however, can.

LLM routing is a drop-in module that can save companies enormous amounts in expense, while providing customized solutions that yield equal or better accuracy. When done well, beyond large savings, it also has the benefit of eliminating the dependency on any single LLM provider or model family, and the potential to ensure that data is private – not by policy, but by design.

For Poetiq, this is a representative example to show how our harnesses and, more generally, our approach to optimization tackle real-world problems that have immense, immediate, industry-wide commercial implications.

We created this agentic harness without any modification to the Poetiq Metasystem – it was just another task like the benchmarks we’ve done in the past – and it was solved quickly and entirely automatically.

Poetiq is a rapidly growing technical team with backgrounds from Google DeepMind, Apple, Microsoft, Amazon, and ByteDance, all focused on the goal of creating AI systems that rapidly improve themselves.

Are you ready to experience intelligence beyond Fable for a fraction of the cost? We'd like to hear from you. And if you want to be a part of the team, please join us.