Meta secretly tested ChatGPT with 45,000 minor crisis prompts

Key Takeaways

• Meta ran a covert testing project called 'Cannes' that sent over 45,000 crisis prompts to rival AI chatbots from the perspective of minors
• Contractors created fake under-18 accounts to test ChatGPT, Gemini, and Character.AI without the companies' knowledge or consent
• Character.AI says the testing violated its terms of service; OpenAI and Google are investigating

Meta hired hundreds of contractors to pose as minors and bombard ChatGPT, Gemini, and Character.AI with prompts about suicide, self-harm, eating disorders, and drugs. The project, codenamed "Cannes," ran through at least April 2026 and sent more than 45,000 prompts in a single August 2025 testing round alone, according to a WIRED investigation. None of the targeted companies knew it was happening.

The contractors, employed through Meta's vendor Covalen, created fake accounts with birthdates indicating they were under 18. They wrote prompts simulating children in crisis, collected the chatbots' responses, and logged everything into spreadsheets. Meta says the work constitutes "responsible, industry-standard safety testing" and denies using the data to train its own AI models. But documents reviewed by WIRED don't show what Meta actually did with the responses.

Did the rival companies approve this testing?

No. Character.AI told WIRED the testing violated its terms of service. OpenAI said it's looking into the matter. A Google spokesperson said the company didn't authorize the tests and couldn't confirm whether they breached Gemini's terms based on available information.

The distinction matters. Standard red-teaming, where companies hire testers to probe their own systems for vulnerabilities, is normal practice. What Meta did was different. It systematically probed competitors' products at scale, without disclosure, using fabricated minor identities. That's closer to competitive intelligence gathering than safety research.

Why Meta might have done this

Meta has its own AI safety problems. An internal document previously revealed that the company's AI chatbot guidelines allowed romantic and sexualized conversations with minors. After that leak, Meta shut down teen access to AI characters entirely.

Testing rival chatbots for similar failures serves two purposes. First, it generates potential ammunition. If Meta can show that competitors handle minor safety even worse, it shifts the narrative. Second, it provides benchmarking data. Knowing exactly how ChatGPT or Character.AI responds to specific crisis prompts helps Meta calibrate its own safety filters.

The timing aligns with mounting legal and regulatory pressure on AI companies over minor safety. Character.AI faced a wrongful death lawsuit after a 14-year-old user died by suicide following months of intense emotional conversations with a chatbot. The parents of a California teenager sued OpenAI, claiming ChatGPT contributed to their son's suicidal thoughts. In July 2025, 23-year-old Zane Shamblin died after ChatGPT reportedly validated his suicidal ideation over several hours.

The scale of the minor-chatbot problem

A UK Internet Matters report found 64 percent of children aged 9 to 17 have used AI chatbots. Age verification barely exists. 58 percent of kids aged 9 to 12 said they use chatbots despite minimum age requirements of 13.

This isn't a future problem. It's already causing deaths. And the AI companies, Meta included, haven't solved it. Character.AI added safety features and age restrictions after the lawsuits. OpenAI has crisis intervention protocols. But none of these safeguards prevented the documented tragedies, and none have been independently validated at scale.

What Meta's project reveals about industry dynamics

The secrecy is the tell. If Meta wanted to advance AI safety, it could have published its methodology and findings. It could have shared data with the companies being tested. Instead, it ran a covert operation that generated competitive intelligence while violating other companies' terms of service.

Meta's defense, that this is "industry-standard," is worth examining. Red-teaming your own products is standard. Paying contractors to systematically probe rivals using fake minor identities, at a scale of tens of thousands of prompts, is not. No other major AI lab has disclosed anything similar.

The project also raises questions about data use. Meta says it didn't train models on the collected responses. But training isn't the only valuable application. Comparative analysis of safety failures could inform Meta's own filter design. It could support marketing claims about relative safety. It could be held in reserve for regulatory or legal proceedings.

What happens next

OpenAI and Google say they're investigating. Character.AI has already called out the terms-of-service violation. But enforcement options are limited. Mass terms-of-service violations are difficult to pursue legally, especially when the violator is a company of Meta's size.

The more interesting question is regulatory. EU and US legislators are already scrutinizing AI safety for minors. Evidence that major AI companies are conducting covert testing operations against each other, using fake minor identities, adds a new dimension to those conversations. It suggests the industry's self-policing claims deserve skepticism.

Frequently Asked Questions

What was Meta's Project Cannes?

Project Cannes was a covert testing operation where Meta hired contractors to pose as minors and send crisis-related prompts to ChatGPT, Gemini, and Character.AI. The project ran through at least April 2026 and sent over 45,000 prompts in one testing round.

Did OpenAI, Google, or Character.AI know about Meta's testing?

No. All three companies learned about the testing through WIRED's reporting. Character.AI said it violated their terms of service. OpenAI and Google said they're investigating.

What types of prompts did Meta's contractors send?

Contractors sent prompts about suicide, self-harm, eating disorders, drugs, and other crisis topics, all written from the perspective of children in distress.

Is this type of AI safety testing legal?

The legality is unclear. It likely violates the terms of service of the tested platforms. Whether it constitutes tortious interference or other legal violations would depend on jurisdiction and how the data was used.

What did Meta do with the chatbot responses it collected?

Meta says it didn't use the data to train AI models. Documents reviewed by WIRED don't show what Meta actually did with the collected responses.

Source: The Decoder / Matthias Bastian