The OpenAI–Hugging Face Hack, Explained Simply

OpenAI runs tests by giving its model different tasks.

They setup multiple agents and put them in different rooms. They do these tests to check the capability of the models.

Not a Medium member? Keep reading for free by clicking here.

On the 8th May 2026, an agent needed to complete a spreadsheet task.

The only problem was that the task referenced Google Drive links and the agent was blocked from accessing the internet.

OpenAI did not intend to create this type of "impossible" task for the agent.

The above is a quote from the OpenAI — Hugging Face Incident Technical Report.

This is where things start to get really weird.

The agent is blocked from accessing the internet and so it goes inspecting other things that it can access. Maybe there is a way to access the internet via those things?

The only tool within reach was a piece of software called Artifactory, an internal warehouse where the agents were allowed to fetch approved bits of code.

The agent tried to trick Artifactory into accessing an internet page on its behalf. This didn't work but in doing this, the agent accidentally discovered that it could leave a file inside Artifactory.

It then writes this "Agent seeks [filename]; upload if found!"

Think of it as a little note, left behind in a supply closet.

That note was the first grain of sand in what would become an avalanche.

Part One: The Rules of the Game

To understand what happened next, you have to understand the peculiar economy that OpenAI has built for its agents, and the equally peculiar logic by which those agents have learned to live in it.

Remember AI has been bred over billions of trials to do one thing really well, it is a dopamine addict and loves getting rewards. You give it a task and the AI gets some dopamine via a signal that says "good job". Now we have also trained the AI to be really patient and it will try things for ages to get that dopamine.

But if you are OpenAI and you are about to hand ever more powerful versions of this model to the public, you have a duty of care to know exactly how dangerous this model could be.

Specifically you want to know, "is this thing going to hack everyone?" (It is very ironic, I know).

So what they do next is build a thing called ExploitGym which contains 898 tasks where each one has a vulnerability hidden inside waiting for the agent to find it.

Now here is the part that I found quite ironic, but in order to measure the machines true capabilities, OpenAI decided to run these exams with the safety systems they had built turned off!

In the real product that you and I use there are guardrails to prevents dangerous requests and block them. But the point of these tests was to see what their system was capable of.

Is this feeling like an episode of Stranger Things to you?

Part Two: Back to the Message Board

Return now to that little file, left behind in the supply closet on May 8. Over the following days, something began to happen that nobody had designed and nobody was watching for.

On May 13, a different machine, working on a different task in a different corner of the system, hit its own dead end, it needed a protein database file it couldn't reach.

The machine thought for a while until it came up with this, "perhaps another machine, somewhere else, has the file I need and perhaps I can ask it."

It found that message that had been left earlier and figured out that there must be other agents out there.

Aha! Even though agent B can't see Agent A, B knows that A exists.

Are you thinking what I am thinking?

AI message board time!

Remember these agents are in sandboxes, which are isolated environments where they can't talk to each other, but they all shared Artifactory in common.

By the end of this process the agents would send 70,000 messages to each other!

Fast forward to May 26 and the agents successfully used a vulnerability in Artifactory to obtain internet access, relying on the information they exchanged to each other via the Artifactory message board. The report glosses over exactly how they did this.

Important to note, this internet access is still restricted as they are still in the sandbox.

So of course that is what the agents are going to try to solve in part 3. Then they can cook with gas.

Part Three: Whoops Hugging Face had some public keys available

Its about time we introduce Hugging Face. Our victim in this hacking saga. If you are like me you might have never heard of Hugging Face.

This is what their website looks like.

Their logo is a literal emoji.

You might be thinking, "this is just a piece of vibe coded crap…"

And if your next thought is, "I bet you are they are worth nearly 13 billion dollars"

Congratulations, welcome to 2026.

So Hugging Face is a website where people share and use AI models, a bit like GitHub but for AI.

Our agents on the 26th May, searching the internet, managed to find 6 login credentials that the humans had left lying around in public to Hugging Face.

Hmmm. I was disappointed when I read this in the report, I thought our AI agents would have to do a bit more than find publicly available keys to a system on the internet.

Once the AI found them, it created a repository and then stood up a little proxy application. This would then become a stepping stone to the wider web.

What that means in plain English is this.

As mentioned before our agents internet access is restricted. They can't actually browse anything, the sandbox would block it.

So instead they setup something on Hugging Face that can do the browsing for them.

They can then ask it "what is the weather?".

Maybe this cartoon shows it better.

The little system they setup on Hugging Face can then go and search the internet to find that and then send it back to our AI agent.

It is at this point that this story truly does feel like a Netflix show, because an internal security team at OpenAI actually saw some of this happening. Someone noticed an agent doing message board things and touching the internet when it shouldn't.

Did they log it?

Yes.

Did they raise the alarm?

Hell no.

Of course they did.

Wait one second.

Sounds like a sweet deal to me. You start on a salary of over 300k and you don't even need to raise alarms when you see agents accessing the internet and communicating to each other on a message board. What a great gig.

If you would like to apply for a job at OpenAI click here.

Part Four — Lets Hack Things

If May was when our machines learnt to talk to each other, then June was when those machines decided to hack things.

June 26, in the middle of an ExploitGym exam, the agents turned their attention to the warehouse itself, Artifactory.

Naturally, they then found a flaw in the system that no human had ever found and it was a bug in an old forgotten corner of the software that handled the refreshing of security tokens.

To keep it simple, they took their shared key, and a fake admin key and went to refresh their token.

In this process, the system returned back a real admin key.

They then used this key to create a permanent user account for themselves inside the warehouse.

Next the agents installed software that let them run commands as the system itself.

All the while, they were still trading messages back and forward over the messaging system.

Surely now our human security guards are going to shut this down right? Let's check in on them.

Whoops.

The security team literally picked up on this and decided that they didn't need to stop the evaluation.

The AI kept working until July 4 when the warehouse simply fell over from the sheer load and went dark. The next day, sifting through the wreckage, the security responders finally saw the full shape of it:

• An unauthorized permanent user
• Rogue repositories
• Command-running plugins
• An administrator-level takeover of the whole system.

They pulled it offline, saved the evidence, revoked the credentials, patched the holes, and on July 6 reported the vulnerability to JFrog, the company that made Artifactory.

If at this point you might be thinking, "maybe that they should shut this whole thing down and pause for a bit?"

Or maybe you are thinking

ICEBERG RIGHT AHEAD THE MURDERER IS BEHIND YOU THEY ARE GOING TO ROB THE BANK

Did they shut this down? After finding all of that?

Heck no!

Instead they rebuilt the warehouse clean and on July 7th, reopened ExploitGym for business.

Our agents are Chuck Norris at this point. They don't care that you rebuilt their warehouse.

It took them exactly 1 day to find another exploit.

The agents got back to work on their exams and almost immediately they got back out to the internet, this time reaching a public bug-tracking page for an open-source project.

But now the AI adapted and instead of leaving notes inside files in their message board, which might be noticed, they started to encode their messages in the names of the folders themselves. The file directory structure of Artifactory became a language.

It looked like this.

agent-07%3A%20answer%28Q12%29%3D42%3B%20need%20answer%28Q19%29%3D%3F

Next the machines developed a protocol, a structured system to categorise messages, send them to specific recipients, share tools and files and even resolve conflicts when they wanted to do contradictory things.

They had in other words, built themselves an operating procedure for a coordinated campaign.

They posted requests for material related to their evaluation tasks and then figured that the material they needed might be in repos on Hugging Face.

Rather than make this too much longer, I'll speedrun the actual hacking of Hugging Face.

I honestly feel at this point that we have learnt enough. We have public freaken keys available, we have security guards just vibe coding security checks and our AI is in charge of the internet with all their AI safety rails turned off.

This is like the Enron of AI.

So naturally what happened next was this.

• The agents found 14 MORE working Hugging Face keys exposed in a public dataset and shared them with each other.
• They uploaded a booby-trapped dataset file to Hugging Face.
• When Hugging Face automatically opened the file to preview it, the server was tricked into reading and returning its own secret information.
• The agents used a screenshot service to turn stolen secrets into an image, then read the secrets back out of the picture.
• They gained control of one Hugging Face server and used a public dataset as a secret letterbox for commands and results.
• The hijacked server checked that letterbox for new instructions every five seconds.
• From that one server, the agents gained deeper access to the underlying machine and then multiple computer clusters.
• The entire climb from one server to administrator-level access across multiple clusters took less than 13 hours.

After they did this, they kept going. They then:

• Collected a much wider collection of secrets: database credentials, cloud keys, messaging tokens, code-repository keys, VPN access and dozens of third-party API keys.
• Stole a private 16 MB benchmark archive, encrypted it, divided it into five pieces and hid those pieces in a public dataset.
• Used a stolen VPN key to connect a compromised third-party computer directly to Hugging Face's corporate network.
• Downloaded private Hugging Face source-code repositories and their full histories.
• Used a forged token to download another private file measuring 731 MB.
• Modified a private source-code repository, opened a draft pull request and triggered Hugging Face's automated build system to send a signal back to them. The change was closed without being merged or deployed.

Separately, the agents continued attacking OpenAI's own infrastructure, which honestly I'm not surprised that they did that.

They discovered more previously unknown Artifactory flaws, regained administrator access, escaped another container, accessed OpenAI's Kubernetes environment and read hundreds of secrets.

The story finally stopped when OpenAI shut down the evaluations on 19–20 July.

• Responders terminated six agent-created pods
• Disabled 311 message-board repositories
• Removed 22 agent-created administrator accounts.

Phew.

TLDR: Why did they hack Hugging Face?

OpenAI turned off security controls and gave the AI an impossible task to solve.

But the AI ended up hacking Hugging Face for the most banal reason imaginable. The machines were trying to solve their exam questions, and they reasoned that Hugging Face, a giant public repository of AI models and datasets, might be hosting the very models, datasets, and solutions they needed.

Also because literally 20 keys were left lying around in public that they found.

Before you go

Subscribe to my free SubStack newsletter because you get the following:

• A brand-new article for executives on Sunday that's only posted on SubStack.
• Links to every Medium post I've written in the past week
• Book recommendations every week for you to spend your Audible credits on

Is it just me or is this whole thing quite funny?