A cost-minimizing reverse proxy that predicts token prices and quota exhaustion using dual Kalman filter families, then routes every request to the cheapest viable provider in real time. Price is the primary signal — argmin(effective_price).
argmin(effective_price) among viable providers — no hardcoded cascade,
no peak-hour routing directive. Peak hours affect routing only through the
price multiplier, never through an if-statement.
Two completely independent Kalman filter families operate per-provider, each tuned to its physics (ADR-002). They never share state. One estimates cost, the other predicts consumption — together they power the routing decision.
Estimates the smooth component of effective cost per million tokens. Peak multiplier, scarcity, and health are deterministic functions applied on top — they are NOT Kalman inputs (ADR-003).
Provider-agnostic predictor of token consumption and quota exhaustion.
Knows nothing about price, cost, peak hours, or health. The
will_exhaust() output feeds the lock decisions in Phase 1 & 2 of
best_key().
The Kalman-smoothed base rate gets multiplied by four deterministic functions (ADR-008). All multipliers are pure and side-effect-free. Any infinite multiplier makes the provider effectively unreachable.
1 + max(0, (used_pct − 50) / 50)clamp(ratio², 0.5, 3.0). Attracts traffic when underutilizing, slows when exhausting.A swim-lane view showing how one request moves through the system, with decision points, branch conditions, and the Kalman feedback loop that makes each subsequent prediction more accurate.
All providers ranked by converged effective price per million tokens. Flat-rate z.ai keys dominate — the Kalman filter amortizes the monthly fee across actual volume, driving effective rates to fractions of a cent.
| Provider | Type | $/M | Notes |
|---|---|---|---|
| ours | Flat €155/mo | $0.001 | Cheapest — high volume |
| friend | Flat shared | $0.029 | 21% penalty over ours |
| ollama | Flat $100/mo | $0.024 | No peak hours |
| openrouter | Per-token | $0.135 | deepseek-v4-flash |
| ppq | Per-token | $0.140 | deepseek-v4-flash |
| deepinfra | Per-token | $1.300 | Last resort only |
Flat-rate providers converge dramatically as volume amortizes the fixed fee. Per-token providers have published prices — no Kalman needed.
A pure, side-effect-free function exposed at GET /v1/dispatch_gate.
Governs whether hardware-dependent tasks should proceed. Checks hardware first,
then quota sufficiency with predictive margins, then price as informational.
| Task Type | Model | Budget Mult |
|---|---|---|
| mechanical | glm-4.5-flash | 0.25× |
| coding | glm-5.2 | 1.0× |
| research | glm-5.2 | 2.5× |
| review | glm-5.2 | 0.5× |
| docs | glm-4.5-flash | 0.5× |
Replay of 50,354 historical routing decisions consuming 527M tokens. The Kalman filters converged to rates 1,000× cheaper than seed estimates while maintaining 99.1% agreement with the seed-rate optimizer.
| Metric | Value |
|---|---|
| Decisions replayed | 50,354 |
| Tokens consumed | 527M |
| Live cost (actual spend) | $10,743.95 |
| Shadow cost (seed-rate estimate) | $11,996.63 |
| Converged replay cost | $0.55 |
| Converged vs Seed savings | 99.7% |
| Live vs Converged agreement | 31.8% |
| Seed vs Converged agreement | 99.1% |
| Routing under converged rates | 100% → "ours" |
The design is codified in Architecture Decision Records. These capture the why behind each non-obvious choice.
argmin(effective_price)..enable_live_routing, .key_disabled_ours. All shadow/logging
paths are wrapped in try/except — they can never break production routing.