Skip to content
Routeplane

Migrate from TensorZero

Migrating from TensorZero to Routeplane

Section titled “Migrating from TensorZero to Routeplane”

TensorZero is an open-source LLMOps platform — a Rust gateway bundled with observability, evaluations, optimization (fine-tuning, automated prompt engineering), and experimentation, all backed by a ClickHouse data warehouse and driven by a GitOps tensorzero.toml. Routeplane solves a narrower problem on purpose: an agent-native proxy with a single OpenAI-compatible surface, runnable as a local binary (BYOK, no infra).

This guide is for teams who adopted TensorZero mainly as a gateway — unified provider access, routing, retries, fallback — and want a leaner, agent-first surface without operating a database. If you lean on TensorZero’s optimization and experimentation loop, read What Routeplane intentionally doesn’t ship before you commit.

Routeplane Cloud (hosted endpoint, teams, audit) is on the Phase D roadmap. For now, self-host.

Why migrate?

Section titled “Why migrate?”
TensorZero Routeplane (self-hosted)
Runtime Rust gateway Rust (single static binary)
Production deps ClickHouse (observability store) + Docker/Compose None
Configuration tensorzero.toml — functions, variants, models, providers Provider keys + routing presets; no schema to maintain
Deployment Self-hosted only Local binary (Apache 2.0)
Design focus Full LLMOps loop: gateway + observability + evals + optimization + experimentation Agent-first proxy: MCP / ACP / Skills, server tools, presets
Agent protocol surface None (gateway is provider-facing) MCP, ACP, Skills, CLI — the product, not an add-on
License Apache 2.0 Apache 2.0

Two things worth highlighting

Section titled “Two things worth highlighting”

1. A proxy, not a platform

Section titled “1. A proxy, not a platform”

TensorZero is built around a feedback loop: route inference, store every call and its metrics in ClickHouse, then evaluate, fine-tune, and A/B test against that history. That’s powerful if you run that loop — and it’s why TensorZero needs a database and a typed config of functions and variants.

Routeplane deliberately stops at the gateway. There’s no tensorzero.toml to maintain, no ClickHouse to operate — you set provider keys and route by provider/model id. OpenTelemetry gives you spans and metrics for every request (the same OTLP export TensorZero offers), but they go to a backend you already run, not a warehouse the proxy manages.

2. Agent-native, single surface

Section titled “2. Agent-native, single surface”

TensorZero’s gateway is provider-facing — it unifies the upstreams. Routeplane adds the agent-facing half: an MCP gateway for tools, an ACP gateway for agent identity and dispatch, a server-tool loop, and structured outputs across providers. The local binary exposes the same OpenAI-compatible endpoint as any future hosted mode — your client code doesn’t change when you move between them.

Migration paths

Section titled “Migration paths”

From an OpenAI-compatible client

Section titled “From an OpenAI-compatible client”

If your app already talks to TensorZero’s OpenAI-compatible endpoint, migration is a base-URL swap plus dropping the tensorzero::… model prefix for a plain provider/model id:

<Tabs items={[‘Before (TensorZero)’, ‘After (Routeplane)’]}> 

import openai


client = openai.OpenAI(
    base_url="http://localhost:3000/openai/v1",
    api_key="not-used",
)


response = client.chat.completions.create(
    # TensorZero model / function reference
    model="tensorzero::model_name::openai::gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
)
```python import openai

Local: routeplane (BYOK via env vars) — see /get-started/quickstart

Section titled “Local: routeplane (BYOK via env vars) — see /get-started/quickstart”

client = openai.OpenAI( base_url=“http://127.0.0.1:4356/v1”, api_key=“not-used-in-local-byok”, # loopback proxy accepts placeholder; real key is on the daemon )

response = client.chat.completions.create( model=“openai/gpt-4o-mini”, messages=[{“role”: “user”, “content”: “Hello”}], )

</Tab>
</Tabs>


A TensorZero **function** (a named prompt template + schema with one or more variants) has no single equivalent — the routing half maps to Routeplane, the templating/variant half stays in your app or moves to a [preset](/features/presets). See the [feature mapping](#feature-mapping) below.


### From the gateway + ClickHouse stack


The infra migration replaces the Compose stack — gateway container plus ClickHouse — with one binary and no database:


<Tabs items={['Before (TensorZero Compose)', 'After (Routeplane local)']}>
<Tab value="Before (TensorZero Compose)">
```bash
# docker-compose.yml runs the gateway + a ClickHouse warehouse
export OPENAI_API_KEY=sk-...
export TENSORZERO_CLICKHOUSE_URL=http://chuser:chpass@clickhouse:8123/tensorzero
docker compose up   # gateway on :3000, ClickHouse on :8123
# config mounted from ./config/tensorzero.toml
 ```bash # Install from the v0.1.0 source tarball (see Installation docs) curl -L -O https://routeplaneapp.vercel.app/downloads/routeplane-v0.1.0-source.tar.gz tar -xzf routeplane-v0.1.0-source.tar.gz cd johnmwhitman-routeplane-*/ cargo install --path apps/routeplane

Set provider keys; Routeplane auto-detects on start

Section titled “Set provider keys; Routeplane auto-detects on start”

export OPENAI_API_KEY=sk-… export ANTHROPIC_API_KEY=sk-ant-…

Start the daemon

Section titled “Start the daemon”

routeplane init # writes ./routeplane.yaml routeplane start # listens on http://127.0.0.1:4356

</Tab>
</Tabs>


## Feature mapping


| TensorZero concept | Routeplane equivalent | Docs |
|---|---|---|
| `[models.*]` + `[models.*.providers.*]` in `tensorzero.toml` | Provider keys (auto-detected) + the model registry | [BYOK](/features/byok), [Models](/concepts/models) |
| `[functions.*]` (named prompt + schema) | App-side, or a routing [preset](/features/presets) | [Presets](/features/presets) |
| `[functions.*.variants.*]` (per-variant model) | Routing preset variants / model ids | [Presets](/features/presets) |
| `routing` / `retries` / `fallbacks` | Model fallback rules | [Model fallback](/features/model-fallback) |
| `load_balancing` across providers | Provider selection | [Provider selection](/features/provider-selection) |
| OpenAI-compatible `/openai/v1` endpoint | OpenAI-compatible `/v1` endpoint | `POST /v1/chat/completions` on the local daemon |
| Native `POST /inference` endpoint | OpenAI-, Anthropic-, and Google-compatible protocols | protocol docs in the [routeplane repository](https://github.com/johnmwhitman/routeplane) |
| ClickHouse observability + UI | OTLP traces & metrics to your own backend | [OpenTelemetry](/features/opentelemetry) |
| OpenTelemetry (OTLP) export | OpenTelemetry (OTLP) export | [OpenTelemetry](/features/opentelemetry) |
| Embedded structured outputs (JSON functions) | Structured outputs across all providers | [Structured outputs](/features/structured-outputs) |
| — (no equivalent) | MCP / ACP / Skills agent gateways | [Tools](/concepts/tools), [Agents](/concepts/agents) |


## What Routeplane intentionally doesn't ship


To set expectations honestly: Routeplane is a gateway, not an LLMOps platform. It does **not** ship TensorZero's optimization and experimentation loop — no built-in supervised fine-tuning, RLHF, automated prompt engineering, dynamic in-context learning, inference-level or workflow evaluations, or adaptive A/B testing, and no ClickHouse-backed UI for replaying historical inferences against alternative prompts.


If your workflow depends on that closed feedback loop — collect inferences and feedback, evaluate, optimize, experiment — TensorZero is the better fit and you should keep it for those workloads. Routeplane is the right move when the value you need from TensorZero is the *gateway* and the missing half is *agent infrastructure*: tool/agent gateways, server tools, and presets.


## Migration checklist


<Callout type="info">
**Before migration**
- [ ] List the providers and models you actually route through TensorZero (skip the rest)
- [ ] Separate gateway use from platform use — are you using evals / optimization / experimentation, or just routing?
- [ ] Note where `[functions.*]` templates and schemas live; plan to keep them app-side or move them to a [preset](/features/presets)
- [ ] Decide which providers you'll supply via env vars (BYOK)
</Callout>


<Callout type="success">
**Migration**
- [ ] Install the Routeplane daemon ([Installation](/get-started/installation))
- [ ] Export provider keys — Routeplane auto-detects on start
- [ ] Update client `base_url` to `http://127.0.0.1:4356/v1`
- [ ] Replace `tensorzero::…` model strings with `provider/model` ids
- [ ] Re-point OTLP export at your backend ([OpenTelemetry](/features/opentelemetry))
- [ ] Verify with a sample request
- [ ] Decommission the gateway container and ClickHouse if you no longer need the platform loop
</Callout>


## Next steps


<Cards>
  <Card title="Quick Start" href="/get-started/quickstart" description="Run Routeplane locally in under a minute" />
  <Card title="Comparison" href="/get-started/comparison" description="Side-by-side with OpenRouter, LiteLLM, and generic gateways" />
  <Card title="Agent features" href="/concepts/tools" description="MCP, ACP, skills, server tools" />
</Cards>


## Get help


- **Discord**: [Join the community](https://discord.gg/G3zVrZDa5C) for migration support
- **GitHub**: [Open an issue](https://github.com/routeplane/routeplane/issues)
- **Email**: contact@routeplane.app for enterprise migration assistance