RouterFuel is a single async proxy layer in front of every model provider you use. Built on a Tokio and Axum network core, it multiplexes concurrent inference requests, negotiates provider failover in microseconds, and streams tokens back without adding measurable overhead.
cargo add routerfuel · zero telemetry sent off-cluster · MIT licensed core
p99 latency
41ms
req / sec
2,148
tokens / sec
18.4k
cache hit rate
37%
active providers
// architecture
Every RouterFuel deployment ships with the same resiliency core — no add-on modules, no separate billing for the parts that keep your inference layer alive.
Vector-matches incoming prompts against prior completions and serves near-duplicates from cache, cutting repetitive API spend by up to 40%.
Trips automatically on provider error spikes or latency drift, rerouting in-flight and new requests to a healthy fallback provider with no dropped calls.
Inject headers, rewrite request bodies, and track upstream server health parameters with a middleware chain that runs inline with zero added round-trips.
// pricing
Every tier ships the full resiliency core. Scale request volume as you go — no feature gating.
For side projects and prototypes.
$0/mo
For production workloads at scale.
$15/mo
For regulated and air-gapped environments.
Custom