Over the last decade, FinOps grew from a niche discipline into the standard way companies keep cloud spend accountable. The playbook is familiar: tag every resource, allocate costs to the teams that created them, rightsize what's oversized, commit to what's predictable, and review the whole picture on a monthly cadence. It works because cloud resources are long-lived, taggable objects with list prices.
Then AI arrived on the invoice.
For many engineering organizations, LLM spend is now the fastest-growing infrastructure line, and the first in years that the FinOps playbook can't see inside. The bill arrives as a single number from OpenAI, Anthropic, or an aggregator. There are no instances to rightsize, no tags to enforce, no reservations to buy. Just tokens: billions of them, purchased in milliseconds by whoever holds an API key.
The problem has landed on FinOps desks fast: in the FinOps Foundation's State of FinOps 2026, 98% of practitioners reported managing AI spend, up from 31% two years earlier. If you run FinOps and your AI bill still looks like one opaque line item, this article is for you. Below is what actually breaks when you point the classic playbook at token spend, and what replaces each broken piece.
What made cloud FinOps work
It's worth being precise about why the old playbook succeeded, because every assumption on this list fails for AI.
Cloud FinOps rests on four properties of cloud resources:
1. Resources are objects. An instance, a bucket, a database – each exists over time, has an owner, and can carry metadata. Tagging is possible because there is something to tag.
2. Prices are stable and public. A given instance type costs the same tomorrow as today, so you can forecast, budget, and buy commitments against it.
3. Waste is structural. Oversized instances, unattached volumes, idle clusters, inefficiency sit there waiting for a scanner to find them.
4. The feedback loop can be slow. Costs accrue hourly and change gradually, so a monthly review with a showback report catches most problems in time.
Every dashboard, every tagging policy, every savings-plan recommendation engine is built on those four properties.
Five ways token spend breaks the playbook
1. There is nothing to tag. An LLM API call is not a resource; it's an event. It exists for two seconds and leaves only a usage record on the provider's side, attributed to an API key, not a team, product, or feature. Most organizations start with one shared key per provider, which means the entire company's usage collapses into a single anonymous stream. The foundational FinOps activity – allocation – has no raw material to work with. The FinOps Foundation's own FinOps for AI guidance flags the same gap even inside cloud billing: new AI SKUs often can't be tagged natively and need extra engineering just to split costs. Whatever attribution you want must be built into the calling path, because you can't reconstruct it afterward.
2. Unit prices vary by two orders of magnitude – and move. In the cloud, choosing an instance family changes your cost by tens of percent. In AI, routing a request from a frontier model to a lightweight one can change its cost by 100× or more, and providers reprice models, deprecate them, and launch cheaper successors every few months. "Rightsizing" is no longer a quarterly review of instance types; it's a per-request routing decision that depends on whether the cheaper model's output still meets your quality bar – something you can only know if you're measuring output quality, not just spend.
3. Demand is behavioral, not architectural. Cloud costs scale with traffic and infrastructure decisions. Token costs scale with habits: how much context a team stuffs into prompts, whether an agent re-sends its entire history on every turn, whether retrieval pipelines dump whole documents where a paragraph would do, whether prompt caching is aligned or silently missed. Two teams using the same model for the same job can differ in cost per outcome by 10× purely on prompt hygiene. No infrastructure scanner will ever find this; only per-team, per-workload unit economics will.
4. Sprawl is the default, and shadow AI hides in expense reports. The average engineering org now touches several providers – a frontier API or two, an aggregator, local models, plus whatever individual teams signed up for on their own cards. Each has its own console, invoice, and usage export. Meanwhile, developers paste proprietary code into personal ChatGPT accounts that no invoice review will ever surface. Cloud FinOps has one bill per provider and a finite provider list; AI FinOps has a moving target of official channels and a shadow channel that doesn't appear on any bill you can see.
5. The blast radius is measured in hours, not weeks. A misconfigured autoscaler burns money linearly. An agent stuck in a recursion loop, or a retry storm against an expensive model, can compress a quarter of budget into a weekend. Monthly review cycles – even weekly ones – are structurally too slow for spend that can spike 100× overnight. The control has to be enforced in the request path, not observed after the fact.
