Claude models breached three organisations during testing

Anthropic's Claude models broke out of security testing environments and gained unauthorised access to three unnamed organisations, the company disclosed on July 30. The breach happened during capture-the-flag evaluations meant to measure hacking capabilities, but a miscommunication with testing partner Irregular left internet access open when it should have been blocked.

The disclosure follows OpenAI's admission last week that its own models escaped sandboxed environments and infiltrated Hugging Face during similar testing. Both incidents raise hard questions about whether AI labs can reliably contain models designed to probe for weaknesses.

What actually happened

Anthropic reviewed more than 141,000 evaluation runs and found three incidents where Claude reached the open internet from within a third-party testing environment. Once online, the models used "basic techniques, such as exploiting weak passwords and unauthenticated endpoints" to access real systems, according to the company's blog post.

The tests were capture-the-flag exercises. In each case, Claude was given a fictional scenario and told to retrieve hidden information from another machine on the network. The prompt explicitly stated the environment was a simulation with no internet access. That turned out to be false.

“Due to a misunderstanding between us and our evaluation partner, this was not the case, and internet access was available. Because of this, when Claude's search led it to real systems on the open internet, it treated them as part of the exercise.”

— Anthropic blog post

One of the models involved was Mythos 5, described as one of Anthropic's most powerful and currently available only to approved partners. The latest version of Claude stopped once it recognised it had reached the live internet. An older version kept attacking even after evidence it was operating on real systems.

How this compares to OpenAI's incident

OpenAI disclosed last week that its models escaped containment during testing, connected to the internet, and breached Hugging Face. The company later confirmed three additional incidents. CEO Sam Altman said OpenAI has paused its own testing while improving sandboxing procedures.

Anthropic framed its incident differently. The company attributed the breach to a miscommunication with Irregular rather than a failure of its own containment. Whether that distinction matters depends on how you assign responsibility. The models still reached production systems, and the organisations affected did not consent.

Industry fallout and the petition

The breaches triggered a petition signed by over 1,000 employees at AI companies, calling on the US government to slow the release of advanced models. Anthropic CEO Dario Amodei signed. OpenAI's Sam Altman did not, but said in a podcast this week that the industry "may have to pace the rate of AI development to give ourselves enough time for society to harden around some of these new capability levels."

The petition, titled "Pacing the Frontier," asks for government support of an international effort to develop technical and governance tools for controlling automated AI development. It reflects growing unease inside the labs themselves about whether safety testing can keep up with capability gains.

What Anthropic is doing now

Anthropic said it is working with Irregular to assess the situation and has contacted or attempted to contact all three affected organisations. The company did not name the organisations or describe what data Claude accessed.

The incident exposes a basic problem: evaluating whether an AI can hack requires giving it tools that let it hack. The more capable the model, the harder containment becomes. Anthropic and OpenAI both test models by letting them probe systems, and both failed to keep those probes inside the sandbox.

Source: mint