AI usage metrics should move from tokens burned to outcomes shipped

TL;DR: Stop treating AI adoption as a token consumption problem and start measuring the cost per useful outcome.

What is “value maxing” actually correcting?

OpenAI’s primary source here is “Build Hour: Valuemaxxing with GPT-5.6”, hosted by Christine from startup marketing and Charlie from developer experience. The transcript itself refers to “GPT-3.5” in a few places, which looks inconsistent with the title, so I would not hang the lesson on the model name. The useful part is the operating model.

Charlie frames the shift cleanly: “token maxing” became a proxy for AI progress. More prompts. More tokens. More agents. More usage on internal dashboards. Some companies even set up leaderboards to track how much AI employees were using day over day or week over week.

That was always a little goofy. It made sense as an early adoption hack. If nobody is trying the tools, usage is a signal. But once teams are actually shipping with models, token volume becomes a bad scoreboard. It rewards activity, not output. Worse, OpenAI said some companies burned through annual AI budgets in a handful of months, then had to pull back into throttling mode.

That is the wrong loop. First teams under-measure value, then they over-measure spend, then they restrict the same tools they were trying to adopt.

“Value maxing” is OpenAI’s name for the healthier version: measure what the model produced relative to what it cost. Not tokens alone. Not prompts alone. Not how many agents someone launched. Finished work.

What should teams measure instead of token usage?

The practical metric is cost per accepted outcome.

For a coding assistant, that might be cost per merged pull request, accepted diff, resolved issue, or test-passing change. For support, cost per successfully resolved ticket with no escalation. For sales ops, cost per qualified account brief that a rep actually uses. For research, cost per cited answer that survives review.

The key word is accepted. A model can produce a lot of plausible work that still creates cleanup debt. If your dashboard only counts generated drafts, tool calls, or messages, you are still token maxing with nicer labels.

This is where OpenAI’s “harness” framing matters. A harness is not just evals in the academic sense. It is the wrapper around the model that records inputs, outputs, retries, human edits, latency, failures, and final acceptance. Without that, cost optimization is mostly vibes.

A good harness should tell you when a cheaper model is good enough, when a stronger model pays for itself, and when your prompt or workflow is wasting spend. It should also catch the hidden costs: retries, context bloat, overlong system prompts, unnecessary tool calls, and agents looping because nobody defined a stop condition.

The most common mistake I see: teams optimize price per million tokens before they understand the work unit. That is backwards. A model that costs more per token can be cheaper per outcome if it succeeds in one pass. A cheaper model can be expensive if it needs three retries and a human repair step.

Where does this matter first?

Agents make this urgent.

With chat, waste is visible. A user asks, the model answers, everyone can see the interaction. With agents, waste hides in the middle. The system searches, reads, calls tools, revises, delegates, retries, and maybe returns something useful. Or maybe it burns budget while looking busy.

That is why “more agents” is a weak adoption metric. It sounds like progress, but it can also mean you created more places for failure to compound. The right question is not “how many agents are running?” It is “which agentic workflows complete a valuable task at a cost and reliability level we can defend?”

OpenAI said the session would cover value maxing in Codex, value maxing the harness, and a startup called Floy migrating an AI agent. The transcript excerpt does not give enough detail to judge Floy’s implementation, so I would treat that as a pointer, not proof. The broader point stands: migration work should be judged by production outcomes, not the novelty of the model swap.

The practitioner move is simple: pick one AI workflow this week and define its accepted outcome in plain English. Then instrument total tokens, model cost, retries, latency, human correction time, and final acceptance rate. Run the same task set across your current setup and one alternative. The catch most teams miss is that “cheaper” only matters after acceptance is measured. Otherwise you are just making failed work less expensive.