Your AI Model Is Only as Safe as Its Supply Chain

Be honest. How many times have you copied a model name from Hugging Face, pasted it into from_pretrained(), and hit run without a second thought?

Same here. It's the normal workflow now. Someone else spent weeks and a pile of GPU money training a model, they shared it for free, and you get to build on top of it in an afternoon. That's genuinely great.

But I recently went down a rabbit hole on AI supply chain security, and it changed how I look at that one innocent line of code. So this post is me walking through what I learned, in plain language, without the jargon soup.

First, what even is a supply chain?

Forget software for a second and picture someone building a house.

They don't make their own bricks. Bricks come from one supplier, timber from another, wiring from a third, pipes from a fourth. And each of those suppliers buys raw materials from someone else. That whole web of "who depends on whom" is the supply chain.

Here's the uncomfortable part: the house is only as safe as its worst component. You can lay the bricks perfectly, but if the wiring is bad, the whole place can still burn down.

Software is exactly the same. A normal Python project can pull in dozens, sometimes hundreds, of third-party packages. And when you run pip install, you aren't only trusting the packages you wrote in requirements.txt. You're also trusting every transitive dependency, meaning the packages your packages quietly pulled in on their own.

GitHub - DhanushNehru/python-for-lazy-devs: A collection of awesome Python frameworks, libraries… A collection of awesome Python frameworks, libraries, software, programs, automation scripts and resources. …

That's a long chain of trust. Every link is a place where something can go wrong.

Why attackers love supply chains

Supply chain attacks don't break through your defences. They walk around them. Instead of attacking your app directly, the attacker poisons something your app already trusts.

Two examples most of us remember:

• SolarWinds (2020). Attackers slipped malicious code into the build process for the Orion product. Customers installed what looked like a normal, legitimate update, and it carried a backdoor. Roughly 18,000 organisations were hit, US government agencies among them.
• Log4Shell (2021). A critical bug in Log4j, a logging library that is basically everywhere in the Java world, allowed remote code execution. Millions of applications were exposed.

What gets me about both is that the victims didn't do anything wrong. They installed trusted software from trusted sources. The problem happened upstream, before the software ever reached them.

And that's the scary maths of it. Compromise one popular component, and you get a foothold in everything that depends on it. One hit, thousands of victims.

Where AI makes it weirder: models aren't just code

Traditional supply chains are mostly about code. Libraries, frameworks, packages, all written by humans you can (in theory) read.

AI adds a completely different kind of thing to the chain: the trained model. And a model is really four things bundled together:

1. An architecture, the structure of the neural network.
2. Training data, sometimes millions of examples.
3. A training process, the settings and hardware used.
4. Serialised weights, the learned parameters saved to a file.

The weights are the part I keep thinking about. The best way I can describe them is a video game save file. When you save a game, the file stores everything about where you are: your position, your inventory, the quests you've finished. Model weights do the same for everything the model "learned".

Now imagine downloading a save file from a random stranger online. It loads fine. Your character has the right stats. The game plays normally. But that stranger had access to every single byte of the file, and they could have tucked something extra in there that you'll never notice. Until the day it matters.

That's not just a metaphor, by the way. Model files saved in Python's pickle format store serialised objects, and those objects can execute code the moment you load the file. You didn't run a script. You just loaded a model. And something ran anyway.

The fine-tuning trap

Almost nobody trains a model from scratch anymore. We grab a pre-trained model and adapt it to our own problem. That's transfer learning, and the most common way to do it is fine-tuning: you keep nudging the model's weights using a smaller dataset that's specific to your task.

It's a massive time saver. Hours instead of weeks.

But think about what you're actually doing. You're building your product on weights trained by someone you don't know, on data you haven't seen, using a process you can't verify.

Here's the bit that surprised me most: research has shown that backdoors planted during pre-training can survive fine-tuning. Your fine-tuning changes how the model behaves on your task. It doesn't scrub out what was baked in earlier. So if someone poisons a popular base model, they don't compromise one app. They compromise every app that fine-tunes from it.

LoRA makes this even more interesting. LoRA lets people share small adapter files that plug into a base model and change its behaviour. Super handy. But now you have two things to trust instead of one: the base model and the adapter. A perfectly clean base model with a malicious adapter attached is still a compromised model.

The four pieces of an AI supply chain

When I broke it down, an AI supply chain has four main parts. Each one comes with its own trust relationships and its own ways to get attacked.

• Models. Pre-trained weights and architectures, usually from Hugging Face Hub, TensorFlow Hub or PyTorch Hub. Think bert-base-uncased or gpt2.
• Datasets. The data used to train and evaluate models, from places like Hugging Face Datasets, Kaggle or academic repos. ImageNet and Common Crawl are the famous ones.
• Frameworks. The ML libraries that train and run models, like PyTorch, TensorFlow and scikit-learn, installed from PyPI, conda or GitHub.
• Dependencies. All the supporting packages those frameworks lean on, like NumPy, SciPy, Pillow and tokenizers.

If you drew this as a diagram, every arrow would be a trust relationship. Your app trusts the framework. The framework trusts the package registry. The registry trusts whoever uploaded the package. Break any one of those and the whole thing falls over.

One bad component hiding among thousands of good ones. That's all it takes.

Downloading a model vs calling an API

Not everyone uses AI the same way, and the risks shift depending on which camp you're in.

Camp 1: you download the model file

You pull the weights from somewhere like Hugging Face and run them on your own machines. Hosting, inference and security are all on you, and everything you load crosses straight into your environment.

The file format matters a lot here:

• .pkl, .pt, .bin use pickle under the hood. Loading them can execute arbitrary code. This is the dangerous one.
• .safetensors only stores raw weight values. No executable code, so the load-time exploit is gone.
• .h5 is the Keras format. Not pickle, but it can still carry executable code inside the model's layers.
• .gguf is what most people use to run local LLMs like LLaMA, Mistral and Qwen. It isn't pickle-based, so no serialisation exploit, but backdoored weights are still very much possible.

One more thing about GGUF. These files are nearly always quantised, meaning the weights have been squeezed down from 32-bit floats to 4-bit or 8-bit integers so huge models can run on normal hardware. If someone else did that compression for you, that's another step in the chain you never verified. You're trusting the original training and every transformation the file went through before it landed on your disk.

Camp 2: you call a model through an API

More and more teams just call OpenAI, Anthropic, Google, or an aggregator like OpenRouter. You send a prompt, get a response, and never touch a model file.

Feels safer, right? No pickle files to worry about. But the supply chain didn't disappear. It just moved somewhere you can't see:

• You can't inspect the weights. The provider controls them.
• Training data is often undisclosed or described vaguely.
• The provider may fine-tune the model without telling you.
• The model behind the same endpoint can change, whereas with downloads you can pin an exact file hash.
• Prompt templates copied from untrusted sources can change how the model behaves.
• It all runs on the provider's shared, multi-tenant infrastructure, not yours.

With an API, the only thing crossing into your system is a JSON response. But behind that response sits the provider's entire pipeline, and you have no way to verify any of it yourself.

Where things actually go wrong: four layers

So where can an attacker step in? For the download approach, I find it easiest to think in four layers. An attacker doesn't need to own the whole chain. One checkpoint is enough.

1. The model layer

This is the part that's unique to AI, and it splits into three levels:

• Serialisation level. Code hidden inside the file format that runs the moment the file loads.
• Architecture level. Malicious logic built into the model's layers that runs on every single prediction.
• Weights level. Learned values tweaked just enough that the model misbehaves on specific inputs.

The key lesson: converting a model to a safe format and stripping executable code only kills the first one. Architecture and weight attacks are completely untouched. You need to inspect at every level.

2. The dependency layer

The packages your ML project relies on. Classic tricks live here: dependency confusion, typosquatting (think a package named one letter off from the real one), and malicious packages with polished, professional descriptions hiding malware underneath.

3. The data layer

Poisoned training data is really hard to spot. Researchers have shown that swapping as little as 0.1% of a dataset with crafted samples can plant a reliable backdoor, without any noticeable drop in accuracy on normal data. If the goal is just to break the model (denial of service), it can take as little as 0.001%.

Let that sink in. A tiny fraction of bad data, and your metrics still look perfect.

4. The infrastructure layer

The systems that host and ship all this stuff. Attackers break into model or package repositories, sneak steps into CI/CD pipelines, or steal a maintainer's credentials and push a malicious update under a name everyone already trusts.

When the layers stack up

These don't happen in isolation. Picture this chain:

1. An attacker takes over a Hugging Face account. (infrastructure)
2. They upload a backdoored model with malicious pickle code. (model)
3. The repo includes a requirements.txt pointing to typosquatted packages. (dependency)
4. A poisoned dataset ships right alongside it. (data)

One download. Every layer compromised at once.

So, what now?

I'm not saying stop using pre-trained models. That ship has sailed, and honestly the open model ecosystem is one of the best things about this field.

But I've started treating from_pretrained() the way I treat curl | bash from a random website. A few habits I'm trying to stick to:

• Prefer .safetensors over pickle-based formats whenever there's a choice.
• Pin exact versions and file hashes instead of grabbing "latest".
• Check who actually published a model or adapter before using it, not just how many downloads it has.
• Treat third-party quantised files and LoRA adapters as their own trust decision.
• If you rely on an API, keep an eye on model version changes and don't blindly copy prompt templates from the internet.

The main takeaway for me is simple: when you use a model, you're trusting everyone who touched it before you. It's worth knowing who those people are.

If you want to go deeper, try this: spin up a throwaway sandbox, download a pickle-based model and look at what's actually inside the file before you load it. It's a very different feeling once you've seen it yourself.

Thanks for reading! If this was useful, drop a clap or leave a comment. I'd love to hear how your team handles model downloads.

**All About AI ** All About AI · Explores the fascinating world of AI · 36 stories on Medium

If this brought value then consider supporting or sponsoring. Follow the journey on X, Instagram, Github or Youtube