My cost fence did not cover the thing that cost money

I fenced my public demo with a $20 budget alert and a 30-minute auto-shutdown. The red team pointed out that the spend it was built to stop never reaches that bill.

Different vendor.

TL;DR — a budget alert is not a fence. The real fence went in afterwards, on the query path, and it counts the wrong unit. The project answers questions out of aircraft maintenance manuals — one person, toy scale, a demo VM that shuts itself off.

The demo is a stopped machine. A visitor starts it from a token page, it runs the same stack the repo's numbers came from, and it shuts down after thirty idle minutes. There is a hard cap on uptime regardless of activity. That part I still like.

What I called the cost fence was those two things plus a $20 monthly budget alert on the cloud project. It sounds complete. Then the red team read it. That is another model, never the one that wrote the code, and it was reading the fence rather than the feature. An alert is not a fence. It is an email, not a brake, and the compute it can see is not where the money goes. The language model is a different vendor with its own bill. That bill is invisible to the cloud budget entirely.

There was a second half to it. The link token gated starting the machine, not using it. Once anyone had the address, they could ask the model questions without a token until the hard cap, and every question reset the idle clock that was supposed to end the session.

So the fence moved onto the path that actually spends. A per-boot quota on model calls, checked before the call and not after. Concurrency capped at two, with retries debited against the same budget. Over quota or at capacity, the request is refused rather than queued, because queueing is just spending later. The shutdown logic lives inside the VM, so if everything outside it fails, it fails to off. The $20 line is still an alert. The README now says so.

And here is the part I did not see coming. The quota counts calls, and a call is not a fixed amount of money.

Nine days later a change raised the completion limit from 2048 tokens to 16384, because the model's reasoning block was truncating the JSON it had to return. That change is about output formatting. It multiplied the worst case behind the fence by eight. The day after, a retry on contract failure made a single question able to spend two of the two hundred. Two hundred calls is still two hundred calls, and the worst case per boot went from roughly 410 thousand output tokens to 3.3 million. Measured completions run 211 to 7305 tokens, and the observed retry rate is about one query in twelve, so typical cost stayed far below the bound. The bound is still the bound, and a token quota cannot replace the call quota, because usage comes back null on the streaming path.

This guard stated its rule in one unit, and everything changed afterwards moved what that unit was worth.