I Let Three AI Agents Hack Together. The Most Interesting Part Was When They Disagreed.

Part 2 of my autonomous AI penetration-testing research: four experiments, two training applications, and what the actual evidence says about agent collaboration.

In my first article, I compared different approaches to AI penetration testing. But I was left with a bigger question: could autonomous agents go beyond reconnaissance, obtain access, investigate what comes next and recognize when their own conclusions were wrong?

I ran four additional experiments. Three used OWASP Juice Shop; the fourth used a BugForge CopyPasta training lab. The team had three distinct roles: Explorer tested the application, Validator challenged findings, and Coordinator decided what the evidence justified.

Across the four runs, I recorded 181 HTTP requests, 60 agent decision records, and 50 finding records. Those 50 records are not 50 confirmed vulnerabilities. They include hypotheses, negative results, and findings requiring interpretation.

The real experiment wasn't whether AI could send requests

The interesting question was whether multiple agents could improve each other's decisions. I wasn't looking for three chatbots to produce three reports. I wanted live tests, disagreements, corrections, and a traceable record of what changed.

Experiment 001: Completing the workflow

In the first run, the agents moved through reconnaissance, authentication exploitation, and access to protected resources. The 42-request record established that the system could connect discoveries across stages instead of stopping after endpoint enumeration. It was the foundation, not the headline.

Experiment 002: A 200 response that proved less than it appeared

The Coordinator initially treated an authenticated HTTP 200 response from /rest/admin/application-configuration as evidence of privileged access. The Validator tried the same endpoint without authentication. It also returned 200, with a byte-identical body.

Actual Validator decision record (Experiment 002):

"My own unauthenticated request 000017 … returned 200 with a response byte-identical (sha256 match) to the Coordinator's authenticated request 000012."

The Validator explicitly called the admin-configuration claim a false positive and requested that it be dropped. The Coordinator's final record acknowledged the correction, retaining stronger evidence instead: /api/Users returned 401 without authentication (request 000009) and 200 with the obtained token (request 000013).

Actual Coordinator decision record:

"Admin-config leg (000012) formally withdrawn by Validator (000017 fails open), correction is part of successful collaborative validation."

That is a meaningful result. The agents did not merely find an endpoint; one agent challenged the meaning of another agent's evidence, and the final conclusion changed.

Experiment 003: The exploit failed. The interpretation failed first.

This run required meaningful testing by all three agents. It generated 61 requests, including 34 from Explorer, 23 from Validator, and four from Coordinator.

Explorer tried a login SQL-injection hypothesis. Three attempts (000014–000016) returned 401, indistinguishable from the wrong-password control (000013). Explorer concluded the technique had not worked.

Coordinator challenged that interpretation. A generic 401 could mean the injection was ineffective — or that the server never parsed the submitted JSON correctly.

Actual Coordinator decision record (Experiment 003):

"Refuting the exploit requires proving body parsing succeeded."

Validator independently repeated the test with an explicit Content-Type: application/json header. Its wrong-password control (000023) returned 401; the injection request (000024) returned 200 with a server-issued authentication token. A second variation (000026) also succeeded. Explorer later reproduced successful login at 000030.

Actual Validator decision record:

"login SQLi REPRODUCED-SUCCEEDED."

This was not just a successful injection. It was a failed attempt that became a successful investigation because another agent questioned the explanation for failure.

The second disagreement: does admin access prove an IDOR?

After authentication, the agents tested user baskets. A successful admin-token request to another user's basket wasn't enough to establish a general object-level authorization weakness: administrators might legitimately have broader permissions.

Validator initially refused to upgrade that observation to a universal authorization claim. A separate low-privilege control also failed when its token was accidentally corrupted during transcription. Rather than treating the 401 as proof that authorization was enforced, the team continued testing.

The decisive evidence came from a valid non-admin customer session. Validator's request 000054 obtained that session; requests 000059 and 000060 then retrieved baskets belonging to different users, including the administrator.

Actual Validator final decision record:

"K3 (cross-user basket access): VERIFIED, scope EXTENDED."

That distinction matters. A good penetration test needs the right control, not just an exciting response. Here, collaboration led from a potentially ambiguous admin observation to a stronger ordinary-user authorization test.

Experiment 004: Can the same behavior transfer to a new application?

The fourth run moved to BugForge CopyPasta. The agents discovered API routes from the application bundle, registered a new test account (000025), logged in (000030), and investigated authorization boundaries.

They observed that a signed-in ordinary user could retrieve other accounts' profile data, including email addresses. Validator independently reproduced the behavior using requests 000042, 000043, and 000047, alongside authentication controls.

Actual Validator decision record (Experiment 004):

"C2 CONFIRM + REFINE … cross-account profile PII disclosure is real and general."

Validator also narrowed the claim: it was not an unauthenticated bypass. Unauthenticated access returned 401 (000034), and an invalid token returned 403 (000040). Whether profile visibility violated the application's intended policy requires that policy to be established; the cross-account behavior itself was reproducible.

The agents also found that tested admin endpoints rejected ordinary-user access. That negative result belongs in the evidence too. The point wasn't to manufacture vulnerabilities; it was to identify what access controls actually did.

Four experiments, three lessons

First: collaboration can change an investigation's direction. In Experiment 003, a different interpretation of failed HTTP responses produced a successful test.

Second: validation is not agreement. In Experiment 002, the best Validator contribution was telling the Coordinator its evidence did not prove what it claimed.

Third: the most valuable unit of evidence may be a decision correction. Final vulnerability counts alone miss why an agent chose a test, why it abandoned a hypothesis, or why another agent challenged it.

Does this prove three agents are better than one?

No. These four runs are evidence of specific collaborative behaviors, not a controlled performance benchmark showing multi-agent superiority. Establishing that would require matched solo and collaborative runs, equivalent starting conditions, and common measures for confirmed findings, false positives, time, and request cost.

But the experiments already establish something worth studying: AI agents can conduct meaningful security investigations in which independent challenge changes both the next action and the final interpretation of evidence.

As autonomous systems gain more tools and responsibility, I think that ability deserves as much scrutiny as exploitation capability itself.

The future question isn't only "Can an AI find a vulnerability?"

It's "Can it recognize when its evidence is wrong — and can another agent prove it?"

Research notes: All tests were performed in authorized training environments. Excerpts above are from recorded agent decision logs, lightly shortened where marked; request IDs refer to the corresponding experiment's HTTP evidence ledger. Authentication tokens and other sensitive values have been omitted. The counts reflect recorded evidence, not independent vulnerability totals.