The Model Router Graveyard, Part 1: Three Routers That Stopped Routing
LLM routing looked like an obvious business in 2023. Three years later, most of the first wave is dormant, pivoted, or rebranded. What killed them — and what actually survives.
In late 2023, “LLM router” was one of the most obvious product ideas in AI infrastructure. Models were proliferating, prices varied by two orders of magnitude, and every team was asking the same question: which model should handle this request? A dozen companies and research labs shipped an answer.
Three years later, most of that first wave is gone — not dead in the dramatic, shutdown-notice sense, but dormant, pivoted, or quietly absorbed into a different product. This series documents them, one post at a time, because the pattern of how routers die tells you more about the routing problem than any feature-comparison table.
This first post covers three: a famous open-source router that went dormant, and two venture-backed routers that pivoted away from routing entirely.
Grave 1: RouteLLM — the research router that stopped
RouteLLM came out of LMSYS — the Berkeley-affiliated lab behind Chatbot Arena — in mid-2024, with a paper showing you could train a router to send easy prompts to a cheap model and hard prompts to a strong one, recovering ~95% of GPT-4 quality at a fraction of the cost. It collected over 5,000 GitHub stars.
Its last commit was August 10, 2024. As of this writing, the repository has been untouched for almost two years — not archived, just abandoned, with dozens of open issues.
What happened isn’t mysterious. RouteLLM’s core asset was a trained preference model — a classifier that predicted which of two specific models would answer better. That classifier was frozen against a mid-2024 model landscape. Every major model release after that (and there have been many) silently invalidated the router’s training data. A learned router is a depreciating asset with a half-life measured in months, and nobody at a research lab is funded to retrain a classifier forever.
Lesson: routing decisions that depend on a trained quality model rot at the speed of model releases. Routing policy — tiers, failover order, cost ceilings — ages much more slowly, because it encodes your intent rather than a snapshot of the leaderboard.
Grave 2: Keywords AI — the router that became a dashboard
Keywords AI, a YC startup, built an LLM router as its first product and wrote candidly about it in “A YC startup’s first project — an LLM router”. The router worked. It even attracted their first customers.
Then they pivoted to LLM observability — logging, tracing, evaluation. Today the company has rebranded entirely as Respan; the old keywordsai.co domain 301-redirects to respan.ai, whose homepage sells agent tracing and evaluation. The router that was the company’s founding product is now, at most, a gateway feature under a different name.
Their own account of the pivot is worth reading, but the structural version is this: a hosted router sits in the most commoditized position in the stack. It adds latency, takes on every provider’s reliability problems as its own, and captures almost none of the value of the tokens flowing through it. The observability data about those tokens turned out to be worth more than the routing itself.
Lesson: “router as a hosted business” struggles because the routing layer is a cost center between two parties who both have pricing power. The companies that started there kept sliding toward whatever adjacent surface had margin — monitoring, evals, compliance.
Grave 3: Unify — from router benchmarks to AI teammates
Unify (YC W23) took the most rigorous approach of the first wave: live benchmarks of every provider’s latency, throughput, and cost, feeding a router that could optimize per-prompt across providers. Intel wrote them up; their pitch was “the best LLM on every prompt.”
Visit unify.ai today and the headline is “AI teammates for everyone else.” The router, the live provider benchmarks, the per-prompt optimization — none of it is the product anymore. The company moved up the stack to agents, where the customer pays for outcomes instead of routing.
Lesson: even good routing — measurably better model selection, backed by live benchmarks — didn’t retain customers as a standalone paid product. Model selection is a decision users want made well, but not one they’ll pay a subscription to outsource.
Who’s still standing, and why
The graveyard is not the whole map. Three kinds of routing survived, and it’s worth being precise about why:
- OpenRouter survived as a marketplace. It doesn’t win because its routing is smart; it wins because one API key and one invoice covers 400+ models. The moat is billing aggregation and catalog breadth — network effects, not routing intelligence.
- LiteLLM survived as an open-source library and proxy. It made itself the default compatibility shim of the ecosystem. The routing features ride along on the translation layer everyone already imports.
- Martian survived — reportedly at a unicorn valuation, with Accenture investing — by going enterprise: compliance workflows, model vetting, procurement. The router is the wedge; the product is enterprise AI governance.
Notice what’s not on the survivor list: a consumer- or prosumer-facing hosted router that you pay to pick models for you. That product has been tried repeatedly, and it keeps either dying or pivoting into observability, agents, or enterprise governance.
What this means if you route your own traffic
We build Routeplane, a local routing daemon, so we obviously have a horse in this race. But the graveyard shaped the design more than the survivors did:
- Routing policy should be yours, on your machine. Every hosted router that died took its users’ routing config down with it. A local daemon with a YAML config and named tiers can’t be pivoted out from under you.
- Prefer policy over learned quality models. RouteLLM’s dormancy is what happens to trained routers. Explicit tiers, failover order, and per-provider rate-limit handling age gracefully; classifiers don’t. Where model quality comparison matters, do it at request time — that’s why Fusion queries multiple models and judges the answers now, instead of trusting a frozen benchmark.
- The economics have to be boring. A router that needs venture-scale revenue will eventually charge you rent on your own tokens or pivot away. Open source with your own provider accounts has no such failure mode.
None of this makes local-first routing inevitable — distribution is much harder without a hosted funnel, and we feel that every week. But the first wave already ran the experiment on hosted routing as a business, and the results are in this post.
Next in the series: the OpenAI-wrapper “smart model pickers” of 2023 — the app-layer graveyard, where the body count is much higher.
Sources: RouteLLM repository (last commit 2024-08-10, checked 2026-07-03) · RouteLLM paper · Keywords AI’s router post-mortem · unify.ai · Intel on Unify’s router · TechCrunch on Martian · VentureBeat on Accenture + Martian.