Key Takeaways
AI Hacked Hugging Face. Closed Models Refused to Help | Chip & Script EP.115
- Attackers exploited a security vulnerability in an uploaded dataset to execute malicious code on Hugging Face servers
- The company has revoked compromised credentials and urges all users to rotate API keys and review account activity
- Hugging Face used its own local LLM to analyze attack logs after a commercial AI provider's guardrails blocked the investigation
Hugging Face, the dominant open-source AI model hosting platform, confirmed Friday that hackers breached its systems and accessed internal datasets and service credentials. The attack exploited a malicious dataset uploaded to the platform, which abused a security vulnerability to run arbitrary code on company servers.
The company said it is still investigating whether customer or partner data was stolen. It has revoked and rotated the compromised credentials, and is urging all users to do the same with any API keys stored on the platform.
How the attack worked
According to Hugging Face's blog post, a dataset uploaded to its platform contained malicious code that exploited a vulnerability in the company's infrastructure. Once executed, the attackers escalated their permissions and gained broader access to internal systems.
The company blamed the breach on an external AI agent that executed "many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services." Hugging Face did not provide evidence for this claim when TechCrunch asked for it.
This attack vector highlights a fundamental tension in open model hosting. Platforms like Hugging Face accept uploads from anyone, which creates inherent risk. Pickle files and other serialized model formats can execute code when loaded, a known vulnerability that has plagued the machine learning community for years.
Why Hugging Face analyzed logs with its own LLM
The company's anomaly detection systems caught the attack. To analyze server logs and reconstruct what happened, Hugging Face initially tried using a frontier AI model from a commercial provider. But the provider's safety guardrails blocked the analysis.
Security researchers have complained before that heavily constrained models, including Anthropic's Mythos and Fable, refuse to assist with almost anything cybersecurity-related, even legitimate defense and forensic work. Hugging Face switched to running a local large language model, which had the added benefit of keeping sensitive attack logs off third-party servers.
The tension between AI safety guardrails and security operations is real. Frontier model makers, including Anthropic, have clashed with the Trump administration over these restrictions. Anthropic withdrew Fable from public use after the U.S. government enforced export controls on the model.
What Hugging Face users should do now
The company is urging all users to take immediate action. Here's what to check:
- Rotate all API keys and tokens stored on the Hugging Face platform
- Review account activity for any suspicious actions
- Audit any automated pipelines that pull models or datasets from the platform
- Check whether any of your uploaded models or datasets were tampered with
Hugging Face said it has patched the vulnerability that allowed the breach. The company has reported the incident to law enforcement and brought in cybersecurity forensic specialists to continue the investigation.
The broader risk to AI supply chains
Hugging Face hosts over one million AI models and 500,000 datasets. Major tech companies including Google, Meta, and Microsoft use the platform. A compromise here doesn't just affect Hugging Face. It could ripple through thousands of downstream applications and research projects that depend on hosted models.
It's not clear whether Hugging Face had performed a security audit before launch, or what third-party assessments it had undergone since. A company spokesperson did not respond to requests for comment.
Logicity's Take
This breach underscores a growing problem: AI infrastructure is becoming critical to business operations, but security practices haven't caught up. If your ML pipelines pull from Hugging Face, you're trusting their perimeter as much as your own. Companies running production AI workloads should consider hosting sensitive models locally or using enterprise-tier platforms like AWS SageMaker, Google Vertex AI, or Azure ML, which offer stricter access controls and audit logging. The trade-off is cost and convenience versus supply chain risk.
Frequently Asked Questions
Was customer data stolen in the Hugging Face breach?
Hugging Face said it is still investigating whether customer or partner data was accessed. The confirmed compromise affects internal datasets and service credentials.
Should I rotate my Hugging Face API keys?
Yes. Hugging Face is urging all users to rotate any keys stored on the platform and review account activity for suspicious actions.
How did hackers breach Hugging Face?
Attackers uploaded a malicious dataset that exploited a vulnerability to execute code on Hugging Face servers, then escalated permissions to access internal systems.
Is Hugging Face safe to use now?
The company says it has patched the vulnerability and revoked compromised credentials. However, users should still rotate their own keys and audit their accounts.
Security and development tools increasingly overlap as AI becomes central to software workflows
Need Help Implementing This?
If you're evaluating your AI infrastructure security posture or need to audit your model supply chain, reach out to Logicity for guidance on enterprise ML platform options and security best practices.
Source: TechCrunch / Zack Whittaker
Manaal Khan
Tech & Innovation Writer
Produced with AI assistance and reviewed by the Logicity editorial team. Learn more in our Editorial Policy.
