60% of agentic AI costs go to refining answers, McKinsey finds

Key Takeaways

• Nearly 60% of agentic AI operating costs stem from verifying and refining responses, not generating initial answers
• Agentic AI tasks can consume 1,000x more tokens than standard chat-based AI, making per-token pricing obsolete for budgeting
• McKinsey identifies six cost drivers CFOs must track: context length, response refinement, cost variability, reasoning overhead, orchestration, and information structure

Most of what enterprises spend running AI agents goes toward fixing and validating the agent's own output. A new McKinsey report puts the figure at 60% of operating costs, a finding that reframes how CFOs and CIOs should budget for agentic systems as they scale beyond pilots.

The consultancy argues that the first two years of generative AI focused on access, experimentation, and deployment. The next phase will be about proving financial sustainability. "The decision to scale an agent is increasingly becoming a complex and fast-changing economics decision, not a technical one," the report states.

Why do AI agents cost so much to run?

McKinsey identifies six factors driving agentic AI operating expenses. The biggest surprise is that generating the first answer is the cheap part. Refining that answer through iterative loops, validation steps, and error correction eats 60% of the total bill.

Context length is another major culprit. Agentic tasks can consume nearly 1,000 times more tokens than a conventional code-reasoning or chat query. At that scale, per-token pricing becomes meaningless for enterprise budgeting. "Per-token pricing has stopped being a useful measure for what enterprises actually pay for gen AI," the report notes.

The third factor is cost variability. Autonomous agents don't follow a predictable path. The same task can incur different expenses depending on which tools the agent selects, how many reasoning steps it takes, and how many retries it needs. This makes forecasting difficult and surprises common.

Where are enterprises wasting money?

McKinsey flags a pattern: companies apply advanced reasoning models to simple tasks that don't need them. Extended reasoning adds value for complex, multi-step problems. For routine queries, it just burns compute without improving outcomes.

Agent orchestration matters too. How AI agents coordinate with tools, external models, and each other determines cost efficiency. Poor task allocation and redundant coordination inflate expenses without changing business results.

Finally, information structure affects token consumption more than most teams realize. Prompt design, context length, formatting, and even language choice alter costs. Non-English text gets fragmented into more tokens per unit of meaning, so identical conversations cost more in some languages than others.

What should CTOs and CFOs track?

The report highlights a gap: many companies deploying agentic AI lack systems to track the business impact of AI-driven decisions. CFOs and CIOs increasingly demand evidence that AI investments deliver tangible returns, but the measurement infrastructure hasn't caught up.

McKinsey's framing suggests enterprises need cost observability at the task level, not just the model level. Knowing your monthly API spend tells you nothing about which agent workflows are efficient and which are bleeding money.

Teams building agentic systems with tools like n8n, Zapier, or Make should consider how orchestration choices affect inference costs downstream. Workflow automation platforms can route tasks to the right model tier, but only if cost tracking is built in from the start.

Does the 60% figure hold across industries?

The McKinsey report doesn't break down costs by sector, so the 60% figure appears to be an aggregate. Industries with higher accuracy requirements, like healthcare, finance, and legal, likely spend more on validation loops. Consumer applications with higher error tolerance may spend less.

The underlying dynamic remains: agents iterate until confidence thresholds are met, and iteration costs money. Enterprises that can lower their confidence thresholds, or accept more agent errors, will run leaner. Those that can't will pay the premium.

Frequently Asked Questions

What percentage of agentic AI costs go to response refinement?

According to McKinsey, approximately 60% of agentic AI operating costs are spent on verifying and refining AI-generated responses, rather than generating the initial answer.

Why is per-token pricing no longer useful for enterprise AI budgeting?

Agentic AI tasks can consume nearly 1,000 times more tokens than conventional chat queries due to extended reasoning chains and iterative refinement, making per-token costs unpredictable at scale.

What are the six cost drivers of agentic AI systems?

McKinsey identifies: long-lived context, response refinement, autonomous cost variability, advanced reasoning on simple tasks, agent orchestration inefficiency, and information structure (prompt design, language, formatting).

How does language choice affect AI agent costs?

Non-English text gets fragmented into more tokens per unit of meaning, so the same conversation costs more in some languages than in English.

What should enterprises track to control agentic AI costs?

Cost observability at the task level, not just monthly API spend. Companies need to identify which agent workflows are efficient and which consume disproportionate resources.

Source: Tech-Economic Times / ET