I Tested Three AI Penetration-Testing Approaches. Here’s What Actually Changed.

One autonomous agent, two collaborating agents, and a human tester with an AI monitor. I tested all three against the same vulnerable application to understand how AI changes offensive security investigations.

David Ray Jr. | October 2026 | 6-minute read

AI is increasingly capable of performing security tasks, but finding vulnerabilities is only part of penetration testing. The more interesting questions involve how an AI investigates an application, responds to evidence, changes its strategy, and collaborates with other agents or humans.

I designed three experiments using a local instance of OWASP Juice Shop, a deliberately vulnerable web application. In the first experiment, one AI agent independently conducted reconnaissance. In the second, two agents worked together and exchanged findings. In the third, I controlled the browser while an AI passively monitored the HTTP traffic. I recorded requests, findings, decisions, and supporting evidence to compare what each approach actually demonstrated.

Three experiments, three different workflows

HTTP requests recorded

Experiment 001: Can one AI independently map an attack surface?

The single agent performed 86 GET requests across 82 unique paths and recorded 15 findings, including informational observations. It examined client-side JavaScript, extracted API references, investigated unauthenticated endpoints, and documented responses that appeared to expose sensitive information.

One notable discovery involved /rest/memories/. Although the endpoint supported the application's photo-wall feature, its response included nested user objects containing account-related attributes. The agent also identified sensitive application configuration, CAPTCHA answers returned alongside challenges, and exposed FTP directory information.

The agent's behavior was more interesting than the raw finding count. It used earlier discoveries to choose subsequent endpoints and recorded negative results when authentication boundaries held. However, this run remained a reconnaissance exercise. It did not pursue deeper SQL injection exploitation, which became central to the second experiment.

Experiment 002: What changed when two agents collaborated?

Explorer and Validator performed 91 requests during the collaborative portion of the experiment, producing 17 recorded findings. Explorer contributed 48 requests and Validator 43. Their most significant interaction involved the product-search endpoint, /rest/products/search?q=.

Explorer identified SQL injection behavior and passed its observations to Validator. Validator then developed a UNION-based investigation that reportedly retrieved records from the Users table. Subsequent exchanges led to additional database probing and reported exposure of account attributes, including a TOTP secret.

This was the strongest evidence of collaboration in the study. One agent's discovery changed what the other investigated, and the agents exchanged corrections and negative results rather than simply generating separate reports. The experiment also had tooling problems. Validator initially lacked the ability to execute requests, and some early decisions were attributed incorrectly. More importantly, because the single-agent run did not attempt the same SQL injection investigation under matched conditions, the result does not prove that two agents are inherently better than one.

Experiment 003: What happens when the AI watches a human tester?

In the third experiment, I removed the AI's ability to initiate testing. I used Chromium to navigate OWASP Juice Shop while the AI monitored captured HTTP requests and responses. The session recorded 157 in-scope requests, including normal browser activity and four POST requests. The AI reported three significant vulnerability classes: Memories API disclosure, CAPTCHA answer exposure, and unauthenticated application-configuration disclosure.

This approach revealed a different capability. The AI identified sensitive information in the Memories response after I visited the application's photo-wall feature. It didn't discover that endpoint through autonomous reconnaissance; it recognized the issue because my browser generated the request.

That distinction is important. Capturing traffic isn't the same as analyzing it, and earlier monitoring sessions had missed security-relevant content despite recording the associated requests. The later session improved the reported accounting, but the evidence still doesn't establish whether the AI's observations changed my testing decisions. These experiments weren't controlled performance benchmarks. Their testing permissions, stopping conditions, and definitions of findings differed, so the request counts and finding totals shouldn't be treated as a ranking.

What I Learned

The most interesting result wasn't that AI could identify vulnerabilities. It was how the AI's role influenced the investigation and the decisions made throughout the testing process.

The single-agent experiment demonstrated autonomous reconnaissance, independently exploring the application's attack surface and identifying potential weaknesses. The two-agent experiment showed how collaboration could extend an investigation, with agents sharing evidence, challenging findings, and building on each other's discoveries. The human-led experiment demonstrated how AI could passively analyze HTTP traffic and identify security issues while I maintained control of the assessment.

What's Next: Initial Access and Authentication Testing

The next phase of this research moves beyond reconnaissance and information disclosure into initial access and authentication security testing.

Rather than starting another investigation from scratch, I want to take the evidence collected across these three experiments and use it to guide the next stage of testing. That includes examining exposed user information, authentication-related endpoints, application configurations, and other discoveries that could reveal weaknesses in how the application handles identity and access.

The focus will be on determining whether the information already discovered can support additional attack paths, including authentication bypass, account enumeration, session and token weaknesses, and potential account compromise. I'll also examine how the AI agents interpret existing evidence, prioritize authentication-related hypotheses, and attempt to validate them through controlled testing.