1% AI error rate means 100 daily problems at scale

A 1% AI error rate sounds impressive until you do the math. Billtrust CTO John Landy points out that when an AI agent makes 10,000 decisions daily, that 1% translates to 100 problems landing on someone's desk every single day. His argument, published in a new PYMNTS eBook, is blunt: most enterprises chasing agentic AI are building on foundations that cannot support it.

The terminology shift has been dizzying. Copilots became agents. Chatbots became autonomous workflows. "Agentic AI" went from analyst jargon to CFO budget line in roughly 18 months. But Landy's central question cuts through the hype: are enterprises actually getting work done differently, or just rebranding the same automation?

Why data quality breaks agentic AI first

Every company sits on years of transactional history, customer behavior signals, and operational data. Volume alone creates nothing. AI agents need context, metadata, and domain knowledge layered on top of raw records. Landy's example is pointed: what good are terabytes of invoice data if the agent cannot distinguish a disputed invoice from a payment preference issue?

Most agentic deployments hit this wall before they hit any other. The model works. The tooling exists. But the underlying data is messy, siloed, or ungoverned. Bad data governance means agents make flawed decisions at scale. You pay for that debt later in rework, model drift, and customer friction.

This is not a new problem, but agentic AI makes it exponentially worse. A human analyst working through 50 invoices a day might catch inconsistencies. An agent processing 10,000 propagates errors at machine speed.

Model Context Protocol: structured access to live data

Landy highlights the Model Context Protocol (MCP) as a meaningful step forward. MCP is an emerging standard that lets AI tools query live enterprise data in structured, governed ways. Billtrust has built this into its platform: a CFO can ask their AI to summarize accounts receivable risk heading into quarter-end and get an answer from live data in seconds, without logging into a separate system.

The distinction matters. The AI gets access to the right data, permissioned appropriately. Not everything. As enterprises scale agent access across functions, the difference between "all access" and "governed access" becomes the difference between a useful tool and a liability.

Where agents work and where they fail

Real impact is happening in bounded workflows. Behavioral segmentation for collections outreach. Payment policy optimization. Anomaly detection in cash application. These work because the data is structured, the objective is clear, and a wrong decision is recoverable.

The next step is agents that take action, not just answer questions. Sending outreach. Applying payments. Escalating disputes. All governed by role-based permissions and a full audit trail.

But here is Landy's toughest lesson: full autonomy is not the right goal for most enterprise workflows right now. The failure mode is not AI that cannot perform. It is AI that performs without sufficient human visibility. When decisions compound at machine speed, errors compound too.

“When an agent makes 10,000 decisions a day, a 1% error rate is a 100-problem-per-day operation.”

— John Landy, CTO, Billtrust

Human-on-the-loop, not human-out-of-the-loop

The most mature teams design for what Landy calls "human-on-the-loop": the agent executes, the human monitors exceptions and approves high-stakes actions. Decision rights have not disappeared. They have been restructured.

This framing matters because the AI vendor pitch often implies that automation means removing humans from the process. In practice, removing humans from high-frequency decision loops without robust exception handling creates cascading failures. A misapplied payment becomes a customer complaint. A wrongly escalated dispute becomes a relationship problem. Multiply by hundreds daily.

The economics only work when error handling costs less than the labor saved. At 1% error rates across 10,000 daily decisions, the break-even calculus is tighter than most business cases assume.

The organizational decision underneath the technology

Becoming an agentic enterprise is ultimately an organizational decision, not a technology one. Landy's checklist is specific: what are agents authorized to do? What data can they act on? How is performance measured? How does governance extend to third-party partners in your AI stack?

You cannot govern in the dark. You cannot govern alone.

The companies that will lead are not the ones with the most ambitious agent roadmaps. They are the ones building the right foundation: clean data, strong governance, meaningful human oversight, and the clarity to know where autonomy earns its place.

Source: PYMNTS | / PYMNTS