UK study: AI benchmarks underestimate agents by up to 25%

Key Takeaways

• Standard AI benchmarks underestimate agent capabilities because they cap compute budgets while performance curves are still climbing
• Success rates on software engineering and cybersecurity tasks improved up to 25% when token budgets increased from 1 million to 10 million
• Newer frontier models benefit disproportionately from extra compute, with capability doubling every 40-50 days at higher budgets versus 67-91 days at standard limits

The UK's AI Security Institute tested frontier models across seven benchmarks and found something that should concern anyone building or deploying AI agents: fixed compute budgets systematically underestimate what these systems can do. When given more processing time, success rates climbed as much as 25 percent on software engineering and cybersecurity tasks.

The study's core argument is straightforward. An AI agent's performance is a curve that rises with test-time compute. Cap the budget while the curve is still climbing, and you measure a floor, not a ceiling. Most current benchmarks do exactly that.

Where extra compute matters most

The gains varied by domain. Cybersecurity tasks showed the starkest gaps. About 8 percent of tasks were only solved when budgets exceeded 10 million tokens. Some required 50 million. The newest models reached even higher scores at budgets above 100 million tokens.

Software engineering benchmarks, TerminalBench 2.0 and SWE-Bench Pro, saw success rates jump roughly 25 percent when token budgets went from one million to ten million. Math and academic tasks on Humanity's Last Exam gained around 22 percent up to five million tokens.

Not every domain responded the same way. On HealthBench, a medical task benchmark, all models plateaued within standard budgets. The pattern suggests extra compute helps most where agents can verify their own work, running code or testing an exploit. It barely moves the needle when feedback is missing or delayed.

Human task time predicts agent token consumption

The researchers found a power-law relationship between how long a human expert needs for a task and how many tokens an agent burns to complete it. Across 211 software engineering tasks from METR and 78 cyber tasks from AISI, the pattern held consistently.

• A one-minute task costs thousands of tokens
• A one-hour task costs millions
• A one-week task costs billions

This matters for evaluation design. A fixed budget cuts off the longest and hardest tasks. When an agent fails, it might mean the budget was too tight, not that the agent lacked the skill.

AISI highlighted a specific example: a cyber task called "The Last Ones" that takes a human expert about 20 hours. No tested model solved it with fewer than 30 million tokens. Under typical benchmark conditions, this task would register as impossible.

Newer models extract far more from extra compute

Each generation of frontier models benefits more from additional compute than the last. The capability curve shifts upward and changes shape along three axes: reach (harder tasks become solvable), reliability (the same task gets solved more often), and efficiency (the same task needs fewer tokens).

A current frontier model's effective time horizon grew from about 40 minutes at 2.5 million tokens to roughly four hours at 50 million tokens. Across the entire frontier, the horizon shifts from about two hours to 14 hours when budgets increase from 2.5 to 50 million tokens.

The models tested included GPT-5, GPT-5.5, Opus 4.5, Opus 4.8, and Sonnet 4.5.

The frontier is accelerating faster than benchmarks showed

AISI had previously estimated that frontier model capabilities on cyber tasks double roughly every 4.7 months, measured at a fixed budget of 2.5 million tokens. At 50 million tokens, the trend line steepens by about 60 percent. Doubling happens every 40 to 50 days instead of every 67 to 91.

The implication: safety evaluations based on standard benchmarks may be measuring a version of AI capabilities that is already obsolete by the time deployment decisions are made. The actual rate of progress is faster than the measured rate.

What this means for AI safety assessments

The study raises uncomfortable questions for anyone relying on benchmark scores to make deployment decisions. If an AI agent scores 60 percent on a cybersecurity benchmark at standard compute, but would score 85 percent with extended resources, which number should inform the risk assessment?

In controlled evaluation settings, compute caps make sense for reproducibility. In real-world deployment, attackers and users can let agents run longer. The gap between what benchmarks measure and what deployed systems can do widens as models improve.

Frequently Asked Questions

Why do AI benchmarks underestimate agent capabilities?

Standard benchmarks cap the compute budget an AI agent can use, typically at fixed token limits. Agent performance rises with compute, so capping it while the curve is still climbing measures a floor, not a ceiling.

How much do AI agent success rates improve with more compute?

On software engineering and cybersecurity tasks, success rates improved up to 25 percent when token budgets increased from one million to ten million. Some cybersecurity tasks required 50 million or more tokens to solve.

Which AI tasks benefit most from extended compute budgets?

Tasks where agents can verify their own work, like running code or testing exploits, show the largest gains. Tasks with missing or delayed feedback, like medical diagnosis, plateau at standard budgets.

How fast are frontier AI capabilities actually advancing?

At standard 2.5 million token budgets, frontier capabilities double every 4.7 months. At 50 million tokens, doubling happens every 40 to 50 days, about 60 percent faster than standard benchmarks suggest.

What models did the UK AI Security Institute test?

The study tested GPT-5, GPT-5.5, Opus 4.5, Opus 4.8, and Sonnet 4.5 across seven benchmarks including cybersecurity, software engineering, and academic reasoning tasks.

Source: The Decoder / Matthias Bastian