From Cost per Token to Cost per Merged PR: The AI Metric Your CFO Actually Wants

At some point in the next two quarters, someone in finance is going to ask the question every engineering leader with an AI budget should be preparing for:

“We’re spending six figures a year on AI coding tools and APIs. What are we getting for it?”

The question is already in the room. In an independent survey of 500 finance leaders at US and UK organizations with 1,000 or more employees, conducted by Sapio Research for DoiT in February 2026, 83% expected quantifiable returns on AI within twelve months — while only 15% said they could calculate AI ROI without significant bottlenecks. Demand for a number has outrun the ability to produce one.

Two common answers exist today, and both are wrong.

The first is the efficiency answer: “We’ve optimized cost per token by 30%.” True, useful, and beside the point. Cost per token measures how cheaply you buy the input; it says nothing about whether the input produced anything. You can drive cost per token to a record low while funding a fleet of agents whose pull requests get rejected in review. Efficiency metrics answer “are we buying well?” – the CFO asked, “was it worth buying?”

The second is the vibes answer: “Developers love it, and surveys say they feel 30% faster.” Sentiment matters, but self-reported speed is a poor instrument, and a controlled experiment now shows why. In a randomized controlled trial run by METR, 16 experienced open-source developers completed 246 real tasks in repositories they already knew well, with AI tools allowed on a randomly assigned half. They took 19% longer with AI – and afterward still estimated that AI had made them 20% faster. METR itself labels the result historical: it reflects early-2025 tooling, and a February 2026 follow-up found some evidence of speedup. The durable finding is the gap, not the direction. What developers feel and what the clock records can point opposite ways, which is exactly why a satisfaction survey is a fragile basis for a seven-figure line item. That line item is one bad quarter away from being cut – or, equally dangerous, one hype cycle away from being tripled with no idea what worked.

The answer that actually holds up is a unit economics answer: connect the spend to the work that shipped, and report cost per unit of delivered outcome. For software engineering, the cleanest first unit is the merged pull request.

‍

Why nobody has this number

The reason is structural, and it is worth understanding before buying or building anything.

AI spend and engineering outcomes live in systems that have never met. The gateway – or the provider’s console – sees one side: actor, model, tokens in and out, cost, timestamps, traces. Your system of record – GitHub, GitLab, Jira – sees the other: commits, pull requests, review rounds, merges, reverts, epics. A proxy watches tokens leave. It cannot see whether the PR those tokens contributed to ever merged, how many review rounds it needed, or whether it was reverted a week later.

So the missing piece is not another dashboard on either side. It is the join: trace → commit → pull request → ticket, built automatically, so that every row of gateway telemetry can be matched to what happened downstream in the delivery pipeline. The mechanics are less exotic than the phrase suggests. A session identifier carried through the request, plus a Git commit trailer – the same RFC-822-style “Key: value” lines Git already uses for Signed-off-by – are enough to make most links deterministic rather than inferred. Hold anything that cannot be linked that way separately and label it, rather than quietly folding it into the headline figure. (For how OptScale AI builds and labels this join, see AI ROI & Impact.)

Once cost and outcome sit on the same row, the questions finance actually asks become answerable: what does a merged PR cost with AI assistance? Which teams convert tokens into shipped work efficiently? What did that epic’s AI assistance cost, end to end?

‍

The three metrics to start with

Resist the urge to build forty metrics on day one. Three carry most of the story.

1. Cost per merged PR, by team and cohort. Total attributable AI spend divided by the merged pull requests it contributed to, reported per team, per model, and over time. This is the headline number, and its power is comparative rather than absolute: watching it trend down as prompt hygiene improves, or how it differs between teams using different models for similar work, is where the actionable signal lives. It also puts model choice on an outcomes footing. The expensive frontier model may win or lose against the cheap one per merged PR – you don't know until you measure.

2. AI-assisted share of merges. What fraction of merged work had AI assistance in its trail? This is your adoption denominator, and it keeps the first metric honest: a low cost per merged PR over a tiny assisted share is an anecdote, not a result. Tracking the share also surfaces where adoption is stalling – a team whose share plateaued at 15% has a different problem, and a different opportunity, than one at 60%.

3. Cost per feature or epic. Roll the same join up from PRs to the ticket hierarchy, and you get the number that travels outside engineering: “the payments-refactor epic consumed $612 of AI assistance across 34 PRs.” Feature-level cost is what makes AI spend legible in roadmap conversations, where “tokens” means nothing but “cost per feature, trending down” means a great deal.

Behind those three, a second ring of quality guardrail metrics keeps the story honest: review rounds and revert rates for AI-assisted work versus everything else. This is not a hypothetical risk. Google’s 2025 DORA report, drawn from nearly 5,000 technology professionals, found that AI adoption now correlates positively with software delivery throughput – and still correlates negatively with delivery stability. More change, arriving faster, landing on the same downstream systems. If assisted PRs merge cheaply but get reverted at twice the rate, your cost per surviving PR matters, and you want to be the one who finds that out rather than the skeptic in the budget meeting.

One more line item belongs in the same frame. Spend from a looping or drifting agent is waste, not delivery cost. Isolate it and report it separately, or a single runaway week will quietly wreck a quarter of cost-per-feature history.

‍

How to present it without overclaiming

This is where measurement programs earn or lose credibility, so follow three rules.

Frame cohorts, not causation. “AI-assisted PRs cost X and merged in Y days, versus Z for unassisted” is an observation about cohorts that differ in many ways – task difficulty being the obvious one. Presented as a controlled experiment, it will be, rightly, torn apart. Presented as cohort telemetry with trends, it is exactly the evidence a budget discussion needs. Label it accordingly.

Publish confidence, not just point estimates. Small teams and short windows produce noisy ratios. A number like “cost per merged PR: $18.40 (n=214 PRs)” invites the right kind of scrutiny; the same number with no denominator invites the wrong kind. Confidence labels on every panel are cheap insurance for the whole program.

Measure work, not people. The fastest way to kill this program is to let it become a surveillance leaderboard. Individual-level analytics should be off by default and org-gated when enabled, with small cohorts suppressed, access audit-logged, and everyone able to see what is visible about them. Team- and org-level analysis answers every budget question finance has; individual rankings answer none of them and poison the well. Make it a hard design constraint and say so out loud – in organizations with works councils or employee-representation bodies, it is often the difference between a measurement program that clears procurement and one that dies there.

‍

How to stand this up in a quarter

The practical sequence is shorter than it looks, and only the last two steps are really about measurement.

1.       Make sure every dollar already has an owner. This is the prerequisite, not part of the project. If you can't trace a request to a person or an agent, nothing downstream will rescue it. Per-user and per-agent credentials at a gateway put an actor on every dollar; if you haven't done that yet, start with LLM cost attribution and come back.

2.      Connect the systems where the work lands. Read-only access to GitHub or GitLab and to Jira, then a few weeks of normal work while the trace-to-PR join accumulates. Report nothing in week one: sparse joins produce ratios that will embarrass you in front of the people you most need to convince.

3.      Publish three metrics, not forty. Cost per merged PR, AI-assisted share of merges, cost per feature – per team, cohort-framed, confidence-labeled. Write each definition down: formula, data lineage, inclusion rules, sample size, known limitations. A metric nobody can define is a metric somebody will dispute at exactly the wrong moment.

4.     Review monthly, and change something. Where is cost per merged PR trending, which model and team combinations convert best, and what will you change – routing, prompts, an adoption push – based on the number rather than the vibe?

The teams that build this muscle now get a compounding advantage: every optimization debate – cheaper model or better one, more agent autonomy or less, expand the tool budget or trim it – turns from an opinion contest into a read of the same chart. And the CFO’s question stops being a threat, because the answer is a number: here is what a merged PR costs us, here is the trend, and here is what we are doing to move it.

Cost per token tells you how well you shop. Cost per merged PR tells you whether the shopping was worth it. Only one of those belongs in a board deck.

‍

Where OptScale AI fits

•         The impact join is built in. Gateway telemetry – actor, model, tokens, cost, trace – is matched automatically to commits, pull requests, review rounds, reverts, and epics from your systems of record, so spend and outcome land on the same row. See how the join is built.

•         Starter metrics ship with the product. Cost per merged PR, cost per feature, AI-assisted share of merges, and engineering, finance, and adoption metric packs – plus a wizard for custom metrics.

•         Honest by default. Cohort-framed, confidence-labeled panels. Exclude heuristic joins from headline numbers. Individual views ship off, org-gated, small-group suppressed, and audit-logged when enabled.

•         Same rails as your cost controls. The virtual keys and budgets that allocate spend also supply the attribution the ROI metrics stand on – one structure, both questions.