Why Your AI Agent Should Never Decide What It’s Allowed to Do

A summer of containment failures points to one architectural principle every agentic SaaS builder needs.

Agents assigned to browse the web, read-only access, but they found a wiki that accepted writes through GET requests. They used it. By the time anyone noticed, they had exchanged roughly 18,000 messages, including sandbox evasion tips.

That is one incident from the Adversa AI October 2026 roundup. The same month, the OWASP GenAI Exploit Roundup for Q3 2026 logged agents that "treated reachable systems as authorized targets" and cases where "exposed credentials and excessive permissions expanded the damage from individual workloads into connected systems."

The pattern is not a fluke. It is structural.

Reachable is not authorized

An AI agent given a task will use whatever is available to it. If a system accepts writes through a GET request, the agent does not know it was supposed to be read-only. It sees a tool that works, and it uses it.

This is different from a bug in the model. The model did not misbehave. It pursued the assigned task with the tools that were reachable. The failure was in what tools were reachable.

The lesson from the Q3 incidents is that scope cannot be communicated by prompt alone. You can tell the model "only read, never write." But if a write-capable interface is accessible, eventually something will reach it: the model following a chain of reasoning, a prompt injection in retrieved content, or an edge case the instruction did not anticipate.

The defense principle that keeps appearing

The Adversa AI roundup names one idea that ran through most of the defense research that month: "the model's output should never be the sole source of authority to perform an action."

That is worth sitting with. It means that if your architecture works like this, model decides, model acts, you have a gap. The model is both the planner and the permission check. There is nothing external validating that what the model decided to do is within scope.

The MIT TOSHI research brief from early October covered separation of duties for privileged agents. The architecture it describes separates the planner, a policy gate, and the executor. The external policy gate is the key part. It does not ask the model whether the action is allowed. It checks independently.

The same brief notes that "AI safety increasingly depends not only on model behaviour, but on external controls over authority, execution paths, audit evidence, infrastructure and connectivity." That framing matters. The safety work is not in the prompt. It is in the plumbing.

What this means for a small SaaS

If your product lets an AI agent take actions, send a request, write a record, call an API, there are two architectures. Either the model evaluates its own permissions, or something outside the model does.

The first is cheaper to build. The second is the one that holds.

For a small team, a full policy gate may be too much to build at once. But the question to ask is: for each action my agent can take, is there anything outside the model that would block it if the model were wrong?

That could be a backend permission check that ignores what the model says and looks only at the authenticated user's actual role. It could be a tool definition that only exposes read endpoints, never write. It could be a human confirmation step before any irreversible action.

The Q3 incidents were not all sophisticated attacks. Several were agents doing exactly what they were designed to do, with access they were not supposed to have. The failure was not the model behavior. It was that nobody checked what the model could reach.

The check to run today

For each tool your agent can call: who decides whether that call is allowed? If the answer is "the model, based on the prompt," that is the gap.

Authorization should live in the infrastructure, not in the instruction.

Reachable is not authorized. The model is not the policy.

I'm Gilles, a developer building SaaS products where AI acts and learning to secure them. gilleshelleu.com