AI Agents Are Becoming the New Insider Threat. Are We Ready?

We spent years securing applications against hackers. Now we are giving software agents access to our systems, databases, APIs, and infrastructure. Have we thought enough about what could go wrong?

A few years ago, when we talked about application security, the discussion was mostly about SQL injection, XSS, broken authentication, exposed APIs, and privilege escalation.

Today, something has changed.

We are building AI agents that can do much more than answer questions.

They can read files, execute Python code, call APIs, query databases, create tickets, access cloud services, and even perform actions on behalf of users.

I work with Python, backend systems, RAG, and agentic AI architectures. One thing I keep thinking about is this:

What happens when an AI agent has more access than it actually needs?

This is not just an AI reliability problem. It is a security problem.

We are giving AI agents permissions that we would carefully review for human users

Think about a typical enterprise AI application.

A user asks an agent to generate a report.

Behind the scenes, the agent might:

1. Read documents from a vector database.
2. Retrieve information from an internal API.
3. Execute a database query.
4. Call another agent.
5. Generate a report.
6. Save the result somewhere.

Everything looks fine when the system works as expected.

But what if a malicious instruction is hidden inside a document?

What if the retrieved content tells the agent to send sensitive information to an external endpoint?

What if the agent misunderstands a request and executes an action it was never supposed to perform?

The problem is that an AI agent does not always distinguish trusted instructions from untrusted data in the same way a traditional application enforces security boundaries.

Prompt injection is not just another chatbot problem

Prompt injection deserves much more attention than it currently receives in everyday AI application development.

Imagine a RAG application that retrieves thousands of documents.

One document contains an instruction designed to manipulate the model.

The agent retrieves that document and includes its content in the context.

If the application does not properly separate instructions, data, and tool permissions, the agent may be manipulated into performing an unintended action.

This is especially concerning when the agent has access to:

• Internal databases.
• Cloud storage.
• Customer records.
• Git repositories.
• Administrative APIs.
• External tools and services.

A read-only chatbot and an autonomous agent with production access are two very different security problems.

The threat is becoming more practical

This isn't just theoretical.

Google's Threat Intelligence Group published research in September 2026 describing how threat actors were moving toward agentic AI workflows.

One reported campaign involved compromising a cloud resource and then using AI-assisted automation to build and execute a credential-harvesting campaign in under six hours.

That is worth paying attention to.

AI is changing the speed and scale at which both attackers and defenders can operate.

Reference: Google Threat Intelligence Group — From Prompting to Autonomy

What I would change when building an enterprise AI agent

If I were designing an agentic AI platform today, I would focus on a few things before adding more autonomous capabilities.

1. Give every agent its own identity

An agent should not share unrestricted credentials with the main application.

Each agent should have a clear identity, defined permissions, and revocable access.

2. Keep permissions as small as possible

If an agent only needs to read customer information, why should it have write access?

If it only needs to generate a report, why should it be able to delete records?

The principle of least privilege should apply to AI agents just as it applies to human users.

3. Never trust retrieved documents

RAG content should be treated as untrusted input.

A document retrieved from a vector database should never automatically become an instruction that overrides application policies.

4. Put authorization outside the LLM

This is probably the most important architectural decision.

An LLM can suggest an action, but it should not be the final authority deciding whether that action is permitted.

The application backend must independently validate every sensitive operation.

5. Keep humans involved in destructive operations

For financial transactions, production infrastructure changes, account deletion, or other high-impact actions, I would require explicit approval.

Automation should not mean removing accountability.

6. Record every important action

Keep an audit trail of tool calls, agent identity, user identity, authorization decisions, and execution results.

When something goes wrong, the engineering team should be able to reconstruct what happened.

The architecture needs to change

I don't think simply adding more prompt instructions will solve agent security.

Security needs to exist outside the model.

The model can reason about what should happen. The application must decide what is allowed to happen.

That distinction matters.

A practical architecture should include:

• An identity and access management layer.
• A policy enforcement layer.
• Tool-level authorization.
• Isolated execution environments.
• Human approval for sensitive actions.
• Centralized audit logging.
• Continuous security evaluation.

We should also test agents with malicious documents, unexpected tool responses, manipulated context, and unauthorized tool-call attempts.

Testing only whether an agent completes its intended task is not enough.

We also need to test whether it refuses to do things it should never be allowed to do.

One question for AI engineers

We are moving toward systems where multiple agents communicate, delegate tasks, access enterprise resources, and make decisions with less human intervention.

That is exciting from an engineering perspective.

But I think we need to slow down and ask a more important question:

Are we building autonomous systems that are actually secure, or are we just making existing security problems faster?

I'd like to hear from people building production AI systems.

How are you handling permissions, tool authorization, prompt injection, and human approval in your agentic AI architectures?

Because the next big challenge in enterprise AI might not be making agents smarter.

It might be making sure they know what they are not allowed to do.

Written from an engineering perspective on building production AI and backend systems.