The routing gateway
Your engineers already have coding agents, whether Claude Code, Codex, or Cursor. The open question is which models they should be using, and at what cost. mo eval systematically benchmarks the models with real repository tasks that run headless in your cloud, no harness switch required. The gateway is how the results are implemented: routing, metering, and budgets for the fleet's traffic, behind zero data retention.
The closed loop
A gateway that blindly forwards requests can't tell you whether the model on the other end is still the right one. mo closes the loop: the same gateway that routes and meters your agents' traffic also executes the benchmarks that decide where the traffic should go.
Benchmarking runs on a cadence you choose, not on every request.
Every test runs through the gateway, so its cost comes from the meter, not each contestant's self-report. The results become routing configuration: a route, a budget, a model choice. You apply them when you're ready, to the same traffic they were measured against. When a provider ships a change, re-run the suite. Routing is a measurement, not a preference.
Evals
mo eval runs headless in your cloud using your repository tasks, executed end to end by your coding harnesses, Claude Code included, through mo's gateway. Each candidate model's quality and metered cost are compared side by side, and adopting the best choice is a routing configuration change, instead of a migration.
Real tasks
Eval suites built from your real workloads, not synthetic leaderboards. Each task pins a repository at a known commit, an agent does the work, and acceptance tests gate the result. Scored against the frontier baselines you already pay for.
Metered sweeps
Models, harnesses, and configurations evaluated side by side through the gateway, so every test carries an authoritative per-task cost from the gateway's own usage records, not each contestant's self-report, and runs on approved routes, under the same budgets as daily work.
Repeatable
Verifiable against every new model release. Content-addressed test plans keep results reproducible, so test score movement over time is attributable to the model. The LLM judge is validated upfront against human-labeled samples before grading your workloads.
Why a harness gateway
Most AI gateways are built for a production application calling models on behalf of end users, so they lead with response caching, end-user rate limits, and content moderation. A coding-agent fleet has a different shape: every prompt is unique, the caller is your own engineer, and the questions are operational.
Typical app gateway
The caller is an application. Cache repeated completions, rate-limit end users, moderate user-facing output, cap spend per app. Good tools exist for this; it is not what mo is for.
mo's harness gateway
The caller is a developer's agent. Map developer-to-key attribution, track per-ticket costs, route models in real time, handle mid-session provider failover. This is the missing layer mo was built for.
Compare
The gateways teams most often evaluate are built for app traffic or general API access, and they solve real problems there. The comparison below is about coding-agent fleets.
| mo gateway | LiteLLM | Portkey | OpenRouter | |
|---|---|---|---|---|
| Built for | Coding-agent fleets, purpose-built: the callers are your engineers' harnesses | General-purpose proxy; the common DIY choice when a platform team fronts a dev fleet itself | LLM apps needing managed observability, guardrails, and semantic caching | Multi-model access for individual developers; agents point at it personally, unmanaged by the org |
| Data retention | Zero data retention enforced on every request; usage records are metadata-only | Self-hosted; logging is whatever you configure | Managed logging by default; self-host option | Hosted router; logging configurable, provider policies apply downstream |
| Cost attribution | Per task and per session, so a ticket has a cost, plus user, team, org, project rollups | Per key, team, and user | Per key and metadata | Per key |
| Runs your harness | Anthropic-native and OpenAI-compatible dialects; mo cc and mo pi launch Claude Code and Pi through it unchanged |
OpenAI-normalized endpoint; you wire each harness yourself (env vars per developer) | Unified API; you wire each harness yourself (env vars per developer) | OpenAI-compatible API plus Anthropic passthrough; the ori CLI configures Claude Code and Codex |
| Eval suites on your coding tasks | Agentic benchmark suites built from your repos and workloads: an agent does the task, tests gate the result. Re-run against each new model release | Proxies OpenAI's Evals API: prompt-level grading of stored completions, not agentic coding tasks | Guardrails and prompt tooling; no coding-task eval suites | Ori Eval: generated prompt evals from your app's LLM call sites, with CI regression gates; not agentic coding tasks |
| Deployment | Managed by Momento, or deployed into your VPC | Self-hosted open source; paid enterprise tier | Managed or self-hosted | Managed only |
Next step
Evaluate mo on your own repos. Run mo eval on your actual workloads, in your own cloud infrastructure. Your tasks, your cloud, results in days.
Or email hello@momentohq.com.