Budget guards — what a Friday-night LLM bill taught me about treating model calls as untrusted services

The feature went live on a Friday. It was a small thing — a Claude-backed helper that processed a chunk of user-provided text into a structured response, the kind of feature you can demo in fifteen minutes and ship in a day. The prompt worked. The output was clean. I'd left the budget side of the implementation as "we'll add metering later."

Saturday morning, nothing seemed unusual. By Sunday night a single user had figured out a flow that triggered the helper a few hundred times in succession — not maliciously, just iteratively, because the feature let them iterate. The model dutifully served every request with no awareness that the bill was a real thing. Each call was cheap. Each call was also not the only call.

Monday morning, the API bill for the weekend had outpaced the rest of the company's monthly cloud spend. Nothing had broken. The feature worked exactly as designed. What I hadn't priced was that "as designed" included no upper bound on what a single user could do, multiplied by no upper bound on how many users could do it, multiplied by a per-call cost that looked rounding-error-small when I read it on the pricing page.

I shipped a kill switch by Monday afternoon. The lessons came over the following week, not in numbered-list form but as a stack of small realisations about what it actually means to call a paid external service from production code.

The first realisation: max_tokens is a cost ceiling, not a model limit

I'd been setting max_tokens to whatever the model's documented maximum was, on the theory that "more headroom is better." It is not. The right value is the longest output I'd be willing to pay for from this specific feature. A summarisation feature where outputs longer than 300 tokens are useless wants max_tokens: 400, not max_tokens: 8192. The 400 isn't a limit on the model's capability; it's a limit on the cost of one call. Five minutes of work, bounds the worst case.

This is the laziest of the guards. It also has the highest leverage, because it caps the cost of each individual call regardless of what goes wrong elsewhere. If everything else fails, the per-call ceiling holds.

The second: per-user caps, enforced before the call

The Friday-night situation wasn't a single expensive call. It was a single user making a hundred normal-sized calls. Per-call caps did nothing for that shape of failure — what I needed was a counter, keyed by user ID, that tracked token usage in the last hour and the last day, and refused to make the API call when the counter exceeded a threshold.

Two windows matter. The daily cap bounds the worst case over a day. The hourly cap is what stops a tight loop from burning through a daily cap in 90 seconds. Without the hourly window, daily caps are theatre — mathematically correct, operationally useless against the failure mode I'd actually seen.

The counter lives in Redis with a sliding window. Every call increments it. Every call also reads it before sending the request to the model, and gracefully refuses when the user has burned through their budget. The user gets a clear message; the company doesn't get a bill.

The third: a global kill switch I haven't thought of yet

Per-user caps assume per-user caps are right, which assumes I sized them correctly, which assumes I understood the cost shape of the feature correctly. None of those assumptions are bulletproof. The third guard is a single counter for the whole feature, summed across all users — a global daily ceiling. If today's total spend exceeds the configured number, the feature returns a graceful error for everyone until the counter resets.

This is the layer that catches the cases where I was wrong, not the cases where the user was. A prompt regression that doubles token counts on every call. A viral moment that ten-x's traffic overnight. A bug where a retry loop fires twenty times for one user input. The per-user math doesn't catch any of those — only the global cap does.

The ceiling sits in environment config so I can tune it without a deploy. Tuning it without a deploy is what makes it useful in practice. A kill switch behind a CI/CD pipeline isn't a kill switch.

The fourth: logging that includes cost, on every call

The thing I couldn't answer Monday morning was "where did the bill go." I had logs of which features were called, but no logs that connected a specific request to a specific cost. I rebuilt cost estimates by joining traffic counts against token counts against pricing tables, and the answer arrived three days late.

Now every Claude call writes a log row that includes tenant ID, user ID, feature name, model, input tokens, output tokens, cost in cents computed at log time, latency, and which tool (if any) was forced. The cost-in-cents column is the one that makes cost dashboards operationally useful — they're a SQL aggregate, not a join across pricing tables, which means I actually look at them instead of intending to.

The fifth: the calendar reminder I have to actually run

Anthropic's prices change. So do every other provider's. The fifth guard isn't code — it's a quarterly calendar reminder to re-check the pricing page and update the cost table in my logger. Skipping this quietly drifts the cost dashboards further from reality every month until they're useless. A drifted dashboard is the same problem as a drifted production metric: it makes the metric undecidable, and an undecidable metric is worse than no metric at all.

What I don't bother with

A few things I've seen recommended that I skip for early-stage AI features:

• Streaming costs to a third-party metering service is overkill before multi-million-dollar AI spend. A Postgres table and a Redis counter handle it fine for years.
• Optimising prompts for token efficiency before shipping is dead- weight engineering. The whole point of the budget guards is that I can ship without optimising prompts yet, then optimise against real data when the cost dashboard tells me where the cost actually lands.
• A "soft" cap that warns instead of blocking just defers the conversation. If the cap is the right number, it should hard-block. If it's not, change it. There's no third state worth building.

The principle

The shift in stance that took me longest to internalise — and the one the Friday-night bill made impossible to ignore — is that an LLM call is an external paid service, and the discipline you'd apply to any other paid third-party API applies here, just with a different shape:

• Timeouts (the per-call max_tokens ceiling)
• Rate limits (per-user windows in Redis)
• Circuit breakers (the global kill switch)
• Observability (cost-included logging)
• Maintenance (the pricing-update reminder)

None of these are clever. All of them exist precisely because the provider, however good their model is, doesn't know what your budget is and won't refuse to spend it on your behalf. Treating the call as untrusted — not in the security sense, in the cost sense — is what separates a prototype from a feature that lives in production.

Most of the work isn't model work. It's plumbing discipline applied to a new shape of dependency. The Friday-night bill was the cheapest way I'd have ever learned that.

---

The budget guards this post describes are part of the AI engineering surface I work on at the multi-tenant AI mirror-site platform.