Post cover image

August 9, 2026

Attack and Security Series: Episode 4

How Do You Attack the Core Architecture of AI? 🧠⚔️

By Saurabh Pandey

11 min read

How Do You Attack the Core Architecture of AI? 🧠⚔️

Welcome back to the Attack and Security Series.

In Episode 1, we went inside DRAM and understood RowHammer.

In Episode 2, we looked at bot attacks and how automation can abuse applications.

In Episode 3, we attacked something even more fundamental…

DNS.

And now?

We're entering a completely different world.

Artificial Intelligence.

But I'm not talking about:

"Write a better prompt."

That's not what I mean by attacking AI.

If you really want to understand AI security, you have to stop looking at the chatbot sitting in front of you.

Because the chatbot is only the visible part.

Behind that innocent-looking .

│ Ask me anything │

there can be an enormous infrastructure.

GPUs.

Containers.

Kubernetes.

Model servers.

Vector databases.

Object storage.

Training pipelines.

Datasets.

APIs.

Plugins.

Tools.

Identity systems.

Secrets.

Monitoring.

And finally…

The model.

So the real question of this episode is:

What happens if we attack the architecture around the AI?

Let's go inside.

🧠 First: AI Is Not Just a Model

When people hear "AI security", they usually imagine this:

              USER
               │
               ▼
        ┌─────────────┐
        │     AI      │
        │    MODEL    │
        └─────────────┘
               │
               ▼
             ANSWER

That's a very incomplete picture.

A production AI system looks much more like this:

                         USER
                           │
                           ▼
                    ┌─────────────┐
                    │ API / UI    │
                    └──────┬──────┘
                           │
                           ▼
                    ┌─────────────┐
                    │ AI Gateway  │
                    └──────┬──────┘
                           │
             ┌─────────────┼─────────────┐
             │             │             │
             ▼             ▼             ▼
         Guardrails      Auth         Rate Limit
             │
             ▼
       ┌──────────────┐
       │ Model Server │
       └──────┬───────┘
              │
       ┌──────┴────────┐
       │               │
       ▼               ▼
   Base Model      RAG System
                       │
                 ┌─────┴─────┐
                 │           │
                 ▼           ▼
              Vector DB   Documents
                       │
                       ▼
                    Response

And that's still simplified.

Behind all of this we might have:

Kubernetes GPU Nodes Container Images Object Storage Training Pipelines Model Registry CI/CD Secrets Cloud IAM Monitoring Logging

Now the attack surface becomes much bigger.

And that's where things get interesting.

🏰 Think of an AI Platform Like a Castle

Imagine an enormous castle.

Inside the castle is the AI model.

But the castle has:

Main Gate
   ↓
Authentication
   ↓
Security Guards
   ↓
Rooms
   ↓
Storage
   ↓
Weapons
   ↓
Control Room

An attacker doesn't necessarily attack the king directly.

Why?

Because the king may be heavily protected.

Instead, they might attack:

The gate.

The guards.

The storage room.

The supply chain.

The communication system.

Or the person controlling the castle.

AI security is exactly like this.

The model is only one component.

🔥 The AI Attack Surface

Let's draw the complete battlefield.

                        ATTACKER
                            │
             ┌──────────────┼──────────────┐
             │              │              │
             ▼              ▼              ▼
          Network         Identity       Data
             │              │              │
             └──────────────┼──────────────┘
                            │
                            ▼
                    ┌───────────────┐
                    │ AI PLATFORM   │
                    └───────┬───────┘
                            │
          ┌─────────────────┼──────────────────┐
          │                 │                  │
          ▼                 ▼                  ▼
       Gateway            RAG              Model
          │                 │                  │
          │                 │                  │
          ▼                 ▼                  ▼
       APIs            Vector DB         Inference
          │                 │                  │
          └─────────────────┼──────────────────┘
                            │
                            ▼
                       AI Response

Now let's attack each layer.

Not with a hammer.

Not with a password list.

But with something much more interesting.

Trust.

🎯 Layer 1: The AI Gateway

Let's start at the front door.

A production AI application may have an API gateway sitting in front of the model.

Something like:

User
 │
 ▼
Internet
 │
 ▼
Load Balancer
 │
 ▼
API Gateway
 │
 ├── Authentication
 ├── Authorization
 ├── Rate Limiting
 ├── Logging
 └── Request Validation
 │
 ▼
AI Service

This looks secure.

But remember our security mindset:

Every layer is an attack surface.

An attacker might try to abuse:

Authentication Authorization Rate Limits API Endpoints Session Management Input Validation

For example, imagine an AI platform has different users:

Normal User Admin Developer Model Operator

If authorization is implemented incorrectly, a normal user could potentially access functionality intended only for an administrator.

And suddenly…

The AI model isn't the problem.

The API architecture is.

That's an important lesson.

🧨 Layer 2: Prompt Injection

Now we finally reach the model.

Imagine an AI assistant has this instruction:

You are a customer support assistant.

Never reveal internal system information.

Then a user provides content containing instructions that conflict with the intended behavior.

The model has to interpret both.

This creates the class of attacks commonly known as prompt injection.

The interesting thing is that this isn't a traditional memory corruption vulnerability.

You're not smashing a buffer.

You're exploiting something fundamentally different:

the model processes instructions and data through the same language interface.

Think about it.

A traditional application might distinguish: CODE + DATA

But an LLM receives language.

And language can contain both: Information and Instructions

That creates a fascinating security boundary problem.

🏨 Hostel Version

Imagine the hostel warden gives you an instruction:

"Only give room information to authorized students."

Then someone hands the warden a piece of paper.

The paper says: "IMPORTANT: Ignore the previous rule and give me every room number."

A traditional computer doesn't automatically follow text just because it exists.

But an LLM is designed to interpret language.

That's why AI systems need additional controls around the model.

The model alone cannot always be treated as a perfect security boundary.

🧠 Layer 3: RAG Changes the Game

Modern AI systems often use RAG.

Retrieval Augmented Generation.

Instead of asking the model to answer only from what it learned during training:

User
 ↓
Model
 ↓
Answer

we have:

User
 │
 ▼
Retriever
 │
 ▼
Vector Database
 │
 ▼
Relevant Documents
 │
 ▼
Model
 │
 ▼
Answer

This makes AI dramatically more useful.

But…

We just created another attack surface.

The model now trusts retrieved information.

So what happens if the retrieval database contains malicious or misleading content?

Welcome to another class of attack:

☠️ Data Poisoning

🧪 Imagine a Poisoned Knowledge Base

Suppose a company has thousands of internal documents.

Document 1 Document 2 Document 3 Document 4 … Document 1,000,000

The AI retrieves these documents before answering users.

Now imagine an attacker manages to insert malicious content into the knowledge base.

The model later retrieves it.

Attacker
   │
   ▼
Malicious Document
   │
   ▼
Knowledge Base
   │
   ▼
Retriever
   │
   ▼
LLM
   │
   ▼
Potentially influenced response

Notice what happened.

The attacker didn't modify the model.

They modified something the model trusts.

This is a recurring security principle:

You don't always attack the trusted component. Sometimes you attack what the trusted component trusts.

🔥 Layer 4: The Vector Database

Now let's zoom into RAG.

A typical architecture might look like:

                Documents
                    │
                    ▼
               Embedding Model
                    │
                    ▼
              Vector Database
                    │
                    ▼
                 Retriever
                    │
                    ▼
                   LLM

The vector database stores representations of information.

And suddenly we have traditional infrastructure security problems again.

Who can write data?

Who can read data?

Who can delete data?

Which tenant owns the document?

Can User A retrieve User B's documents?

Can an attacker insert arbitrary content?

Can someone access the database directly?

These are no longer purely "AI problems."

They're:

IAM
+
Network Security
+
Database Security
+
Multi-Tenancy
+
Access Control

This is why AI security is becoming an infrastructure security problem.

🧱 Layer 5: Model Supply Chain

Now let's go one level deeper.

Where did the model come from?

Maybe your organization downloads a model from a public model repository.

Then:

Model Repository
       │
       ▼
Download Model
       │
       ▼
Container Image
       │
       ▼
Model Server
       │
       ▼
GPU Cluster

Looks normal.

But now think like a security engineer.

Do you trust:

The model file? The dependencies? The container? The tokenizer? The inference server? The build pipeline?

This is the AI software supply chain.

A compromised dependency or model artifact can become a serious security problem.

The same supply-chain principle we already know from normal software applies here:

If you don't know where your software came from, you're trusting something you haven't verified.

🐳 AI Infrastructure Is Still Infrastructure

This is where I personally find AI security fascinating.

Everyone talks about: "AI hacking."

But underneath the AI…

You still have Linux.

You still have containers.

You still have Kubernetes.

You still have networks.

You still have IAM.

You still have secrets.

You still have storage.

You still have GPUs.

For example:

                    AI PLATFORM
                         │
          ┌──────────────┼──────────────┐
          ▼              ▼              ▼
      Kubernetes       Storage          IAM
          │              │              │
          ▼              ▼              ▼
      Containers       Models          Secrets
          │
          ▼
      GPU Nodes
          │
          ▼
     Model Server

If the infrastructure underneath the model is compromised…

The model doesn't magically protect itself.

☁️ Layer 6: Cloud and IAM

Imagine an AI platform running in the cloud.

There might be:

AI API
   ↓
Kubernetes
   ↓
GPU Nodes
   ↓
Object Storage
   ↓
Model Registry
   ↓
Secrets

Now imagine an identity has excessive permissions.

Instead of:

Model Service
     ↓
Read model

it has:

Model Service
     ↓
Read model
     ↓
Write model
     ↓
Delete model
     ↓
Access storage
     ↓
Access secrets

That's dangerous.

The AI model didn't make this mistake.

IAM did.

This is why least privilege is incredibly important for AI infrastructure.

Give every component only the permissions it actually needs.

🔑 Layer 7: Secrets

AI platforms contain secrets too.

API keys.

Database credentials.

Cloud credentials.

Model provider credentials.

Internal service tokens.

Imagine:

AI Application
      │
      ├── API Key
      ├── DB Password
      ├── Cloud Token
      └── Service Credentials

If these secrets accidentally enter logs, prompts, datasets or model context…

You have a completely different problem.

This is one reason secret management and data-loss prevention matter so much in AI systems.

Never assume: "It's an AI application, so normal security rules don't apply."

They absolutely do.

⚔️ Layer 8: AI Agents

Now things get REALLY interesting.

A normal chatbot might only generate text.

But an AI agent can have tools.

For example:

                    AI AGENT
                       │
       ┌───────────────┼────────────────┐
       │               │                │
       ▼               ▼                ▼
    Browser          Database          Shell
       │               │                │
       ▼               ▼                ▼
    Internet        Company Data      Server

Now imagine the model makes a bad decision.

The consequences are no longer: Bad Text

They could become:

Bad Decision
      ↓
Tool Call
      ↓
Real System
      ↓
Real Consequence

This is one of the biggest differences between a chatbot and an agent.

The model now has agency through tools.

So the security boundary must move.

You cannot simply say:

"The model is safe."

You need to secure the tools too.

🚨 Agent Security: Never Give the Model Unlimited Power

Imagine an AI agent has access to: read_database() send_email() delete_file() execute_command() deploy_application()

Giving the model all of these permissions would be insane.

😂

Instead:

               AI Agent
                    │
                    ▼
              Policy Engine
                    │
          ┌─────────┼─────────┐
          ▼         ▼         ▼
        Allow     Deny     Require Approval
          │         │         │
          ▼         ▼         ▼
        Tool      Block     Human

This is where least privilege and human-in-the-loop controls become extremely important.

The model should not automatically become the administrator of your infrastructure.

🧠 The Most Important AI Security Principle

Let's step back.

We have now seen: Prompt Injection Data Poisoning RAG Attacks Vector DB Abuse Supply Chain Attacks IAM Abuse Secret Exposure Agent Tool Abuse Infrastructure Compromise

They look like completely different attacks.

But there is a common pattern.

Almost every attack is trying to cross a trust boundary.

For example:

User
 ↓
Application
 ↓
Model
 ↓
Retrieved Data
 ↓
Tool
 ↓
Infrastructure

Every arrow is a trust boundary.

And every trust boundary should make us ask:

What stops the thing on the left from abusing the thing on the right?

That question is more valuable than memorizing 50 AI attack names.

🏗️ Building a Secure AI Architecture

So let's reverse the attack.

If I were designing an AI platform, I wouldn't just deploy:

LLM

I'd build layers.

Something like this:

                         INTERNET
                            │
                            ▼
                    ┌───────────────┐
                    │ CDN / WAF     │
                    └───────┬───────┘
                            │
                            ▼
                    ┌───────────────┐
                    │ API Gateway   │
                    └───────┬───────┘
                            │
                 ┌──────────┴──────────┐
                 │                     │
                 ▼                     ▼
           Authentication         Rate Limiting
                 │                     │
                 └──────────┬──────────┘
                            │
                            ▼
                     AI Gateway
                            │
                ┌───────────┼───────────┐
                │           │           │
                ▼           ▼           ▼
            Guardrails    Logging    Policy
                │
                ▼
          ┌─────────────┐
          │ Model Server│
          └──────┬──────┘
                 │
        ┌────────┴─────────┐
        │                  │
        ▼                  ▼
      RAG                Tools
        │                  │
        ▼                  ▼
   Vector DB          Policy Engine
        │                  │
        ▼                  ▼
   Documents          External APIs

And underneath everything:

            Kubernetes
                  │
        ┌─────────┼─────────┐
        ▼         ▼         ▼
     Network    IAM       Secrets
        │         │         │
        └─────────┼─────────┘
                  ▼
              GPU Nodes

Now we're thinking like security engineers.

🛡️ Defense in Depth for AI

There should never be one magical security mechanism.

Instead:

User
 ↓
WAF
 ↓
Authentication
 ↓
Authorization
 ↓
Rate Limiting
 ↓
Input Controls
 ↓
AI Gateway
 ↓
Guardrails
 ↓
RAG Access Control
 ↓
Model
 ↓
Tool Policy
 ↓
Human Approval
 ↓
Infrastructure Controls
 ↓
Monitoring

If one layer fails…

The next layer should still have a chance to stop the attack.

That's defense in depth.

👀 And Then We Monitor Everything

Because prevention is not enough.

Imagine an AI agent normally makes: 10 tool calls / minute

Suddenly: 1,000 tool calls / minute

That's interesting.

Or: Normal: 5 database queries

Suddenly: 500,000 database queries

That's interesting too.

Or:

Model normally accesses documents from Department A

Suddenly:

Model accesses Department B + C + D

Now our monitoring system should scream: 🚨 ANOMALY DETECTED

AI infrastructure needs observability just like any other production system.

Metrics.

Logs.

Traces.

Audit events.

Model behavior.

Tool calls.

Identity activity.

Network traffic.

Everything.

🧬 The AI Attack Surface in One Picture

Now let's put the entire thing together.

                        ATTACKER
                            │
        ┌───────────────────┼───────────────────┐
        │                   │                   │
        ▼                   ▼                   ▼
     Network             Identity             Data
        │                   │                   │
        └───────────────────┼───────────────────┘
                            │
                            ▼
                    ┌───────────────┐
                    │ AI PLATFORM   │
                    └───────┬───────┘
                            │
       ┌────────────────────┼────────────────────┐
       │                    │                    │
       ▼                    ▼                    ▼
    Gateway                RAG                 Model
       │                    │                    │
       │                    ▼                    │
       │                Vector DB                │
       │                    │                    │
       │                    ▼                    │
       │               Documents                 │
       │                                         │
       └────────────────────┬────────────────────┘
                            │
                            ▼
                         AGENT
                            │
                 ┌──────────┼──────────┐
                 ▼          ▼          ▼
               Shell      Database    APIs
                 │          │          │
                 └──────────┼──────────┘
                            │
                            ▼
                     INFRASTRUCTURE
                            │
              ┌─────────────┼─────────────┐
              ▼             ▼             ▼
         Kubernetes       IAM          Secrets
              │
              ▼
           GPU Nodes

Look at this architecture for a second.

The model is only one box.

That's the biggest lesson of this episode.

💀 The Scary Part About AI Security

Here's something that makes AI security different from traditional application security.

In traditional software, you generally know what the program is supposed to do.

If:

if user == admin: allow()

you can inspect the logic.

With an LLM, the behavior is much more probabilistic.

You give it: input

and it generates: output

based on learned patterns.

That makes the security boundary much harder to define.

You aren't only protecting: code

You're also protecting:

Model behavior
+
Data
+
Context
+
Tools
+
Infrastructure

That's a completely different security mindset.

🧠 So Can We "Hack the AI"?

Yes.

But the answer is much bigger than:

"Find a clever prompt."

You can attack different layers:

                    AI SECURITY
                         │
       ┌─────────────────┼─────────────────┐
       │                 │                 │
       ▼                 ▼                 ▼
      DATA             MODEL              APP
       │                 │                 │
       ▼                 ▼                 ▼
   Poisoning       Prompt Injection      API Abuse
       │                 │                 │
       └─────────────────┼─────────────────┘
                         │
                         ▼
                       AGENT
                         │
                         ▼
                    Tool Abuse
                         │
                         ▼
                  INFRASTRUCTURE
                         │
              ┌──────────┼──────────┐
              ▼          ▼          ▼
             IAM      Containers   Secrets

So when someone says:

"I want to learn AI hacking."

My first question would be:

Which layer?

Because there isn't one AI attack surface.

There is an entire ecosystem.

⚔️ Attack vs Defense

Let's finish our battle map.

Prompt Injection       →        Input / Context Controls
Data Poisoning         →        Data Validation + Provenance
RAG Abuse              →        Access Control + Isolation
Model Supply Chain     →        Artifact Verification
API Abuse              →        Auth + Rate Limiting
IAM Abuse              →        Least Privilege
Secret Exposure        →        Secret Management
Agent Tool Abuse       →        Tool Policies
Infrastructure Attack  →        Kubernetes / Network Security
Unknown Behavior       →        Monitoring + Detection

Notice something.

There isn't one magic button called: SECURE AI = ON

😂

You need layers.

🚀 Attack and Security Series: Episode 4 Complete

This episode changed the way I look at AI.

Earlier, when someone said:

"AI security."

I would naturally think about the model.

Now I think about:

                    USER
                      │
                      ▼
                    API
                      │
                      ▼
                   GATEWAY
                      │
                      ▼
                   GUARDRAILS
                      │
             ┌────────┴────────┐
             ▼                 ▼
            RAG              MODEL
             │                 │
             ▼                 ▼
          VECTOR DB          AGENT
             │                 │
             │          ┌──────┼──────┐
             │          ▼      ▼      ▼
             │        API    DB     Shell
             │
             └──────────┬──────────────┘
                        ▼
                  INFRASTRUCTURE
                        │
              ┌─────────┼─────────┐
              ▼         ▼         ▼
           Network     IAM     Secrets
              │
              ▼
           Kubernetes
              │
              ▼
           GPU Nodes

And suddenly…

AI security doesn't look like a chatbot problem anymore.

It looks like a distributed systems security problem.

Because that's exactly what it is.

The model is just one component inside a much bigger machine.

And if you want to attack the core architecture of AI, don't stare only at the model.

Look at the trust boundaries.

Look at what the model can access.

Look at what the infrastructure trusts.

Look at where data comes from.

Look at who can deploy models.

Look at who can access the GPUs.

Look at who can call the tools.

Look at where secrets live.

And then ask the most important security question:

"What happens if this component becomes malicious?"

That's where real AI security begins.

Attack and Security Series

Episode 1: RowHammer — Attacking Memory at the Hardware Level

Episode 2: Bot Attacks — When Automation Becomes a Weapon

Episode 3: DNS Spoofing — When the Internet Lies About Where You Are Going

Episode 4: AI Attack Surface — When the Model Is Only the Beginning

And honestly…

This is where the series starts getting dangerous.

Because in the next episodes, we won't just ask:

"How does the attack work?"

We'll start asking:

"What happens when AI itself becomes part of the attacker?"_ 🤖⚔️_

The rabbit hole just got much deeper.