Everyone's Burning Millions on AI Coding. I Tried to Fix the Bill, and Built the Wrong Thing.

You've seen the headlines: the industry torching hundreds of millions of dollars in tokens on AI coding agents. My reaction was probably the same as a lot of engineers': there's got to be waste in there, and waste is a product.

So I built one. Hannah was a proxy you'd point your coding agent at. It would trim and compress the context being re-sent every turn, and route the easy work to a cheap local model while the hard work went to a frontier model. Classic middleman-takes-a-cut economics. On paper, obvious.

Then I benchmarked it properly, and the data quietly took the idea out back.

The setup

I built the same two real apps, a Next.js PWA and a mobile game, four ways: on Codex (gpt-5.5), on Claude (opus-4.8), on a local model, and through my router. Same tasks, real build-and-test verification on every run, actual token and dollar accounting.

Three things fell out, and none of them were what I expected.

1. "Subscription" coding agents are not free, they just report $0

The harness shows you nothing because it isn't pricing that path, but the API-equivalent bill is real: $0.48 to build the PWA on Codex, $0.95 on Claude. And the bill is dominated by output tokens ($25–30/M), not the input you're tempted to compress; cached input reads are nearly free ($0.50/M). A terser model beats a verbose one on the same task, even when it reads more context. The thing I built Hannah to optimise barely mattered.

2. The 10× lever isn't the model, it's the harness

This is the one that actually matters. Same model (gpt-5.5), same kind of build, two harnesses:

Harness	Input tokens / turn	Build cost
Native Codex CLI	~50–70k	~$5–11
Lean harness (Pi)	~4k	$0.22

That's ~15× more context shipped per turn, purely from the harness, before a single optimisation. The native CLI re-ships a big system prompt, full tool schemas, and accumulated history every turn; the lean harness ships a tiny prompt and four tools.

And no, caching doesn't save you. The native CLI cached well here (most of its input was cache-reads), but at enough volume even near-free reads add up. The harness sends more; caching only softens the blow.

3. Routing barely pays on real work

On a from-scratch build there's no big "easy slice" to offload; the planner sent almost everything to the frontier anyway, and the local model got one or two buggy steps. The routed run ended up the most expensive and the slowest: you pay a planner tax on top of frontier executors. Routing only wins when there's a large, separable, low-risk chunk, which complex builds don't have.

The conclusion: it's a setup choice, not a product

Across both tasks, one setup was consistently the cheapest, the fastest, and the only clean first pass:

Pi + Codex. $0.22–0.48 a build, 2–4 minutes, correct on the first try. Roughly an order of magnitude cheaper than the same build in a heavyweight native harness.

And it needs no proxy, no routing, no local model: the exact machinery I'd spent days building. The whole savings story collapses to two decisions you make once, for free: use a lean harness, pair it with a terse frontier model.

The proxy was a clever answer to a question a good default already solves. Two things did survive the cull, and I think they're the real lesson:

• Cost has to be visible: per-task, per-model, in real dollars. "It says $0" is how the bill gets to nine figures.
• You cannot trust an agent's own "done." A wall-clipping game with a dead control pad passed every "no console errors" check I had. A functional check (actually run it, screenshot it) caught the bug in seconds.

I'll publish the full benchmark and the Hannah code separately for anyone who wants to pull the numbers apart. But the one-line takeaway is cheap and worth more than the proxy ever was: before you optimise the middle, look at the edges.