How One Leaked Key Hacked a Popular AI Tool: 5 Simple Lessons from the LiteLLM Breach

What Is an AI Gateway?

Many apps today use AI models like GPT or Claude. Most of these apps don't talk to the AI directly. They use a middle layer called a gateway.

The gateway sends each request to the right AI model. It also tracks costs and retries failed requests.

To do this, the gateway stores the secret keys for every AI provider you use. So if a hacker breaks into the gateway, they get all of those keys at once.

That is exactly what this story is about.

What Is LiteLLM?

LiteLLM is a popular open-source AI gateway. Thousands of teams use it.

On October 21, 2026, hackers published a fake version of LiteLLM with hidden malware inside. Anyone who installed it was at risk.

Here is how it happened, step by step.

Step 1: A Secret Key Was Posted by Mistake

A LiteLLM developer uploaded a settings file to a public GitHub page. That file accidentally contained a master key for an internal LiteLLM agent tool.

This was a simple mistake. The bigger mistake was that the key was never cancelled. It stayed active for months.

Step 2: Hackers Used the Key

A hacker found the key and used it to log in to the agent tool. Inside, they found a GitHub token.

This token was supposed to do one small job: let AI agents open pull requests. But it was not limited to that job. It belonged to the GitHub account of the organization owner. So it had full control over everything.

Step 3: Hackers Collected More Keys

With owner-level access, the hacker stole even more secrets. They took GitHub tokens from two more engineers. They also took keys for Linear, Together AI, and other services.

Now the hacker had three trusted accounts.

Step 4: Hackers Released a Fake Version

The hacker used the owner's account to turn off GitHub's safety rules. Then they changed the release process so it built the software from a fake repository. The fake name was "Berri-AI/litellm." The real name is "BerriAI/litellm." The only difference is one hyphen.

The three stolen accounts worked together. One started the release, one approved it, and one deleted the records afterward.

The result: version 1.107.0 went live on PyPI and Docker, with malware inside.

What Did the Malware Do?

The malware started every time Python started, even if the program never used LiteLLM. On Linux and macOS, it also checked for new commands from the hacker every hour.

It stole almost everything valuable on the computer. This included cloud keys for AWS, Google Cloud, and Azure, SSH keys, browser passwords and cookies, and the AI provider keys stored inside LiteLLM itself.

That last part is the key point. The gateway held every AI key in one place, so one break-in exposed them all.

This Was Not the First Time

LiteLLM was also hacked in March 2026. That time, hackers broke in through a poisoned security scanner in LiteLLM's build system.

Two attacks in one year show that popular AI tools are now a main target for hackers.

A Warning for Everyone Using AI Coding Tools

At first, LiteLLM thought the tokens were stolen from its logs. The team found that Claude Code can sometimes include a user's GitHub token in its responses. If those responses get saved in logs, the token ends up in the logs too.

Later investigation showed this was not the cause of this attack. But the risk is still real. AI tools can accidentally copy your secrets into places you don't expect, like log files.

5 Simple Lessons

1. If a key leaks, cancel it right away. Don't wait to see if someone uses it. Assume they will.

2. Give each tool only the access it needs. A tool that opens pull requests should not have owner-level power. Small access means small damage.

3. Don't use "latest" in production. The "latest" Docker tag pointed to the infected version for a few hours. If your system downloads "latest," someone else decides what you run. Use exact version numbers instead.

4. Protect your logs. Remove keys and tokens before saving logs. Limit who can read them. And never put internal tools on the public internet.

5. If you were affected, change every secret. Deleting the bad package is not enough. Go back to version 1.106.2, the last safe version. Then replace every key and password that computer could access.

Final Thought

I work on testing an AI assistant. Most of our testing checks what the AI says: Is it correct? Does it make things up?

This attack reminded me that the AI's answers are only part of the picture. The tools around the AI need testing too. That means checking logs for secrets, checking what access each tool has, and checking which version is actually running.

The fix is not complicated. It is the basic security rules we already know. We just need to apply them to our new AI tools.

Check your AI gateway this week. It probably holds more secrets than you think.