Council of AIs · Budget-Driven Multi-Model Router Case Study | ProductGurus
Open Source · In Production

Council of AIs

Case study by Abhay Kumar · Updated July 2026

An open-source engine that replaces AI subscriptions with a single prepaid budget, routing each task to the cheapest model that can actually handle it. Trivial work goes to cheap models, high-stakes reasoning to premium ones, with spending limits enforced in code instead of trusted to a model.

My role
Architect · Verifier
Status
In production · MIT
Stack
Python · SQLite · MCP
AI providers
Claude · Gemini · GPT
trivial non-trivial stakes / graph touch volume / ambiguity failure User Task Bypass Gate Cheap Model Leader Gate Thinker · Premium Explorer · Mid Escalation Graphify SQLite Mem Budget Ledger
System architecture: every dollar and every routing decision is enforced in code, not left to a model.
The problem

Subscriptions charge by the seat, not by the task

Power users pay for two or three AI subscriptions and still hit message caps in the middle of real work. The pricing has no relationship to the value of the request: a premium model answers "what is 15% of 240" for the same effective cost as designing a system, and rationing the good model to conserve quota just produces worse results.

  • Message caps interrupt work mid-thought, regardless of how trivial the task is.
  • Long chat histories re-bill the same context on every turn.
  • Switching to a cheaper model means losing the conversation's memory.
  • You either overpay a premium model for trivial work, or under-serve hard problems with a cheap one.
The idea

One prepaid budget, spent by capability

Most tasks do not need a genius. A few do. So the system uses a premium model only for the few, makes that routing call automatically, and protects a fixed prepaid pool the whole time. A Leader routes each task, a Thinker handles high-stakes reasoning, Explorers handle volume work, a Ledger meters every token, and a memory layer carries context across sessions and providers so nothing is lost when the model changes.

What I built

A router, a ledger, and a memory that outlives the model

Gate-based router

A deterministic cascade decides the tier by what a task touches, not how it is phrased. Anything hitting a schema, auth path, or contract escalates to the premium model, regardless of how simple it looks.

Budget ledger

Atomic reservations stop two tasks spending the last dollar at once. A locked reserve floor, per-task caps, and an endgame mode that asks before touching the final reserves are all enforced in code, never left to a model.

Graphify memory

Decisions, constraints, and rationale live as human-readable, git-versioned files, with a rule that only premium models can author the high-stakes ones. Your context becomes an asset you keep, not something trapped in one vendor's history.

Council Chamber

A live view where a premium model writes a contract and reviews the work while a cheaper model implements it. The loop is bounded: one clarifying question, three review rounds, a hard budget cap, ending in approval or a clean-room escalation.

Under the hood

The hard part is judging difficulty cheaply

Deciding whether a task needs deep reasoning often requires deep reasoning, so asking a cheap model "is this hard" fails on exactly the tasks that matter. The router sidesteps that with observable gates, and a codebase graph tells it what a change actually reaches. A rename that looks trivial but touches a schema is escalated automatically.

// routing decision, logged per turn{
  "gate_fired": "Gate 1 (stakes: schema touch)",
  "tier": "thinker", "model": "claude-opus",
  "cost": 0.00884, "remaining_budget": 74.02
}
How it was built

Built by its own council

I did not hand-code it, and I did not let one model generate the whole thing in one shot. I ran the project the way the product runs a task, using the same economics it preaches: premium reasoning only where it changes the outcome, cheap execution for the volume.

T

Claude Fable 5 · Thinker

Acted as principal architect. Wrote the spec, red-teamed its own design (it found six holes in its escalation logic worth fixing), reviewed every batch of code, and caught bugs that only surfaced against live API traffic.

Cheap input tokens make an elite reviewer affordable
B

Gemini · Builder

Wrote the modules and the first pass of the tests, and iterated on them. Cheaper to run as the high-volume builder doing the bulk of the typing.

The build was itself a demonstration of the thesis
Outcome

A budget that cannot be blown, and memory you own

After several build-and-verify rounds, the system ships with routing that measured a 35x cost gap between tiers on a live run, a ledger that physically cannot overspend, and a memory layer that lets any model resume a project cold. The guardrails are tested adversarially: injection through tool output, overspending through concurrency, and tampering with verification all have tests that try to break them.

47
Passing tests, including adversarial
35x
Cost gap between tiers on a live run
3
LLM providers, swappable by config
Roadmap

What ships next

Sandbox

Container isolation for all untrusted execution, so agent-written code runs safely by default.

Learned routing

A local classifier trained on real routing outcomes, so the router improves from use instead of static rules.

Parallel trees

Opt-in parallel Explorer branches on high-stakes tasks, with the Thinker choosing the best result.

More case studies

See the rest of the work, or get in touch.