GitHub found better tools made Copilot code review worse

Key Takeaways

• More powerful tools don't automatically improve AI agent performance
• GitHub had to rethink how Copilot accesses and processes code context
• The fix required balancing tool capability against information overload

GitHub's engineering team made a counterintuitive discovery while improving Copilot's code review capabilities: giving the AI agent access to better, more powerful tools actually made its reviews worse. The finding challenges a common assumption in AI development, that more capability equals better results, and forced the team to rethink their approach entirely.

Napalys Klicius, a Software Engineer at GitHub who works on agentic systems, detailed the problem and solution in a recent blog post. His background spans model checking, low-level C++ drone systems, and static analysis. Now he focuses on teaching agents to inspect code without getting lost in the process.

Why more tools created worse reviews

The core issue stems from how AI agents process information. When GitHub expanded the tools available to Copilot during code review, the agent gained the ability to explore more of the codebase, fetch additional context, and examine related files. In theory, this should produce more informed, accurate feedback.

In practice, the agent got lost. More tools meant more pathways to explore, more context to weigh, and more opportunities to chase irrelevant details. The AI would spend its reasoning capacity navigating the expanded toolset rather than focusing on the actual code changes under review. Reviews became slower, less focused, and paradoxically less useful despite having access to more information.

This mirrors a problem human reviewers face. A senior engineer reviewing a three-line change doesn't need to understand the entire repository. They need the right context, delivered efficiently. Flooding them with every tangentially related file makes the review harder, not easier.

How GitHub actually fixed Copilot code review

The solution required GitHub to constrain what Copilot could access and when. Rather than giving the agent free rein over an expanded toolset, the team built guardrails that guide the agent toward relevant context without overwhelming it.

The approach balances capability against focus. Copilot still has access to powerful tools for understanding code, but the system now manages how and when those tools get deployed. Think of it as giving a reviewer a well-organized desk rather than dumping the entire filing cabinet in front of them.

This required significant iteration. Klicius and the team had to identify which contexts actually improved review quality and which just added noise. The answer wasn't always obvious. Some information that seemed essential turned out to distract the agent, while other data that appeared tangential proved crucial for accurate feedback.

The broader lesson for AI agent development

GitHub's experience highlights a challenge facing the entire AI tools industry. As companies race to add capabilities to their AI systems, they're discovering that raw power doesn't translate directly to utility. The models powering these tools have finite context windows and reasoning capacity. Every piece of additional information competes for attention.

This matters for engineering teams evaluating AI code assistants. A tool with more features isn't necessarily better. What counts is whether those features are deployed intelligently, at the right moment, with the right constraints.

The same principle applies to developers building their own AI-powered tools. If you're integrating language models into your workflow using platforms like Vercel or managing AI-assisted projects in ClickUp, the lesson holds: more capability requires more careful orchestration.

What this means for Copilot users

GitHub Copilot now serves over 1 million paying subscribers, with GitHub claiming that 46% of code on the platform is written with AI assistance. Code review represents a natural expansion. Developers spend an estimated 20-30% of their time reviewing others' code, making it a significant productivity bottleneck.

The improvements GitHub describes should translate to more focused, actionable review comments. Rather than getting generic observations or tangential suggestions, developers should see feedback that demonstrates understanding of the specific change and its context.

Whether this closes the gap with human reviewers remains an open question. AI can catch certain classes of issues, syntax problems, common anti-patterns, potential bugs, more consistently than tired humans at 5pm on a Friday. But code review also involves judgment calls about architecture, maintainability, and team conventions that require deeper context than any tool currently provides.

Frequently Asked Questions

Why did better tools make GitHub Copilot code review worse?

More powerful tools gave the AI agent too many pathways to explore. It spent its reasoning capacity navigating the expanded toolset rather than focusing on the actual code changes, producing slower and less focused reviews.

How did GitHub fix Copilot's code review quality?

GitHub built guardrails that guide the agent toward relevant context without overwhelming it. Rather than free access to all tools, the system now manages how and when tools get deployed based on what actually improves review quality.

Can AI code review replace human reviewers?

Not entirely. AI catches syntax problems, common anti-patterns, and potential bugs consistently, but code review also involves judgment about architecture, maintainability, and team conventions that require deeper context than current tools provide.

How many developers use GitHub Copilot?

GitHub Copilot has over 1 million paying subscribers. GitHub reports that 46% of code on the platform is now written with AI assistance.

Source: The GitHub Blog / Napalys Klicius