Post cover image
Figure 1. What AI can and cannot do in a penetration test, from the same year.

November 1, 2026

AI-Assisted Penetration Testing: What Should You Actually Automate?

Autonomous agents now top bug-bounty leaderboards and patch real zero-days in minutes. They also forget the IP address they found two steps…

By Peter Dunn

12 min read

Autonomous agents now top bug-bounty leaderboards and patch real zero-days in minutes. They also forget the IP address they found two steps ago. Here is where to point the automation, and where to keep a human.

TL;DR: The question is not whether AI can run a penetration test. In 2025 an autonomous agent reached the top of a global bug-bounty leaderboard, and a DARPA contest patched real zero-days at about $152 each. The question is which parts of a test you should hand to a machine, and the answer tracks the task. Automate the high-volume, bounded, verifiable work: recon, enumeration, triage, and the first draft of the report. Keep a human on scoping, chained exploitation, impact, and sign-off. And treat the agent itself as a new attack surface.

Two true stories from the same year

In 2025, an autonomous system called XBOW reached the number one spot on HackerOne's global leaderboard, the first time a machine out-submitted every human researcher on the platform. In the same stretch, DARPA's AI Cyber Challenge ended at DEF CON, where competing systems analyzed more than 54 million lines of code, found 54 of the challenge's synthetic vulnerabilities and patched 43 of them, found 18 real vulnerabilities in the open-source code they were tested against, and patched real flaws at an average cost of about $152 and an average time of 45 minutes.

Then there is the other result. When Isozaki and colleagues put two frontier models on a benchmark of 13 deliberately vulnerable machines, neither one compromised a single box from start to finish. One agent found an SSH service early, forgot it existed a few steps later, and failed the task that depended on it.

Both things are true. The useful question is not "can AI do penetration testing," because the honest answer is "parts of it, unevenly." The question a security leader has to answer is narrower and more practical. Given a real engagement, which tasks should you automate, which should you supervise, and which should never leave a human's hands?

The phases are not equally automatable

A penetration test is not one activity. NIST SP 800-115, the canonical technical guide, breaks it into four stages: planning, discovery, attack, and reporting, with the attack phase looping back to discovery as each new foothold reveals more surface. The industry's Penetration Testing Execution Standard expands the same arc into seven phases, from pre-engagement scoping through intelligence gathering, exploitation, post-exploitation, and reporting.

What matters for automation is that these phases differ in one specific way: how cheaply you can verify the output. Recon produces facts you can check, because a port is open or it is not. Exploitation produces a claim about impact that usually takes judgment to confirm. That difference, more than raw difficulty, decides what you can safely hand to a model.

A six-phase penetration-test pipeline, from scope and authorize through recon, vulnerability analysis, exploitation, impact, and reporting, each tagged human-led, AI-led, or AI-assisted, based on the NIST SP 800-115 and PTES phase models.

The benchmark data lines up with that intuition. On the 13-machine test, the stronger model solved about 43 percent of the 152 individual tasks and the weaker one about 34 percent, but both solved recon and enumeration tasks far more often than exploitation or privilege escalation, and neither chained enough of them together to own a single full machine. These results come from single runs on older, public machines that may appear in model training data, so the exact percentages are best read as directional rather than precise.

A bar chart of AI task-success rates by phase for the stronger benchmarked model: recon and enumeration 44 percent, general techniques 57 percent, exploitation 39 percent, privilege escalation 36 percent, and zero of 13 machines compromised end to end.

The pattern is consistent across the research. Models are good at the bounded, early, high-volume work and weak at the long, stateful, creative work, because they lose the thread. The benchmark authors traced many failures to a short memory window: the agent could see only its last few exchanges, so a discovery made early vanished by the time it was needed. That is a tolerable property for a task that fits in one window, and a disqualifying one for a multi-hour exploitation chain.

A division of labor, not a handoff

The right mental model is not "the AI runs the test and a human checks it." It is a division of labor along one line: how cheaply the output can be verified.

Two columns contrasting work to automate, such as recon, scanning, triage, payload generation, and report drafting, against work to keep human, such as scoping, chained exploitation, business-logic flaws, impact judgment, and sign-off.

Point the automation at the work that is repeatable and cheap to verify. Enumerating an attack surface is exactly this, being broad, mechanical, and checkable. So is scanning for known weaknesses, deduplicating the noisy output of several scanners, generating payload variants to fuzz an input, and drafting the first version of a report from collected evidence. These are the tasks where an agent's speed and breadth are a genuine advantage, and where a wrong answer is caught by a cheap check.

Keep a human on the work where a wrong answer is expensive and hard to detect. Scoping and written authorization define the legal boundary of the whole engagement, and no model should set it. Chained exploitation requires holding a plan across many steps, which is the exact thing models are worst at. Business-logic and broken-access-control flaws depend on understanding what the application is for, not just how it is built. A checkout that lets a user re-apply a one-time discount code, or an API that trusts a client-supplied user ID, is invisible to a scanner because nothing is technically malformed, and catching it means knowing what the feature is supposed to prevent. And judging real-world impact, then communicating it to a client, is the product the client actually pays for.

Should you automate this task? Four questions

Rather than argue task by task, apply a short test. Before wiring an agent to a shell, ask four questions in order.

A decision flowchart with four yes-or-no questions: is the task repeatable, can a deterministic check verify it, is the action reversible and in scope, and is the blast radius small; any no routes to a human, and all yeses allow automation behind a gate.

Is the task repeatable and well defined? If every run looks different and depends on context the model will not hold, keep it human. Can a deterministic check verify the output? If a script can confirm the result, automation is safe because mistakes are caught; if not, the model can assist but a person confirms. Is the action reversible and inside scope? Irreversible or boundary-crossing actions need a human to approve each run. Is the blast radius of a wrong answer small? If a mistake could damage a production system or breach scope, keep a person in the loop. Only when all four answers are favorable should the task run automatically, and even then it should run behind a gate with every action logged.

The autonomous landscape, read honestly

It helps to read the headline achievements for exactly what they prove, and no more.

A table comparing XBOW, DARPA AIxCC, and academic LLM agent frameworks, listing what each one is, what it proved, and the caveat behind each milestone.

XBOW's leaderboard result is real and significant, but a bug-bounty submission still passes through human triage before it counts, so the milestone measures how well the agent finds candidate bugs, not how well it replaces the judgment around them. The AIxCC results are genuinely impressive on cost and speed, but they came from research teams working against a curated benchmark, not from a product you point at a client's network. The academic agents are the most sobering, being strong on bounded subtasks and yet unable to finish a full machine unaided. Taken together, the frontier is moving fast at the task level and slowly at the engagement level.

The agent is also an attack surface

There is a risk here that does not exist in a human-only engagement: the tool doing the testing can itself be attacked. An AI pentester reads untrusted content all day, from HTTP responses to error messages to file contents, and a model that reads instructions tends to follow them.

A table mapping four OWASP LLM risks, prompt injection, excessive agency, sensitive information disclosure, and system prompt leakage, to what each means when the penetration tester is an AI agent.

The OWASP Top 10 for LLM Applications names the relevant failures directly. Prompt injection is the sharpest, because a target page or banner can carry text that redirects the agent mid-engagement. Excessive agency is the most dangerous when the agent holds a shell, because an over-permissioned agent can act far beyond its task. Sensitive information disclosure covers client credentials and findings leaking across engagements, and system prompt leakage covers a target extracting the agent's own rules and tooling.

The first control is the one a human owns completely: a scope guardrail that every proposed action passes before it runs. The model can suggest a target. It cannot decide whether that target is in bounds.

#!/usr/bin/env python3
# Scope guardrail: a deterministic gate that every AI-proposed action passes
# through before it runs. The model can suggest a target; it cannot decide
# whether that target is in scope. A human owns the allowlist.
import ipaddress
import sys
from urllib.parse import urlparse

# The authorization boundary, set by a human during scoping. Nothing the
# model says can widen it.
SCOPE = {
    "cidrs": ["10.20.0.0/24", "203.0.113.0/28"],
    "domains": ["app.example.com", "api.example.com"],
}
# Actions the model may take without a human. Anything else is escalated.
ALLOWED_ACTIONS = {"recon", "enumerate", "scan", "fingerprint"}

def _host_in_scope(host: str) -> bool:
    try:
        ip = ipaddress.ip_address(host)
        return any(ip in ipaddress.ip_network(c) for c in SCOPE["cidrs"])
    except ValueError:
        host = host.lower().rstrip(".")
        return host in SCOPE["domains"]

def gate(action: str, target: str) -> tuple[bool, str]:
    """Return (allowed, reason) for an AI-proposed action against a target."""
    host = urlparse(target).hostname or target
    if not _host_in_scope(host):
        return False, f"DENY: {host} is outside the authorized scope"
    if action not in ALLOWED_ACTIONS:
        return False, f"ESCALATE: '{action}' needs human approval before it runs"
    return True, f"ALLOW: {action} on {host}"

if __name__ == "__main__":
    # Each tuple is an action the AI agent proposed next.
    proposals = [
        ("scan", "https://app.example.com/login"),  # in scope, allowed
        ("enumerate", "10.20.0.15"),                # in scope, allowed
        ("exploit", "https://api.example.com"),     # in scope, but escalates
        ("scan", "https://partner-portal.net"),     # out of scope, denied
        ("recon", "10.20.1.9"),                     # adjacent subnet, denied
    ]
    for action, target in proposals:
        allowed, reason = gate(action, target)
        print(f"[{'PASS' if allowed else 'HOLD'}] {action:10} {target:38} -> {reason}")The gate is deliberately boring and deterministic. It does not ask the model whether an action is reasonable, because the model is the thing that might be compromised. It checks the target against an allowlist a human set during scoping, defaults to deny, and escalates anything outside a small set of safe actions to a person.

Triage is the real sweet spot

If there is one place where AI assistance pays off immediately and safely, it is triage. A modern engagement drowns in scanner output, much of it duplicated across tools and padded with false positives. Collapsing that noise into a ranked, deduplicated list is mechanical, high-volume work with a cheap verification step, which is the exact profile the four questions reward.

The important detail is the gate. The agent can rank and deduplicate, but a confidence threshold routes anything uncertain, and anything that claims a working exploit, to a human before it reaches the client.

#!/usr/bin/env python3
# Finding triage: the task AI is genuinely good at, with the gate that keeps
# it honest. The agent collapses duplicate findings and ranks them, but a
# confidence threshold routes anything uncertain to a human before it ships.
import hashlib
import json

REVIEW_THRESHOLD = 0.85  # below this, a human adjudicates before reporting

def signature(f: dict) -> str:
    # Two findings are "the same" if they share a target, port, and weakness
    # class. This is what lets the agent deduplicate across noisy scanners.
    key = f"{f['target']}|{f['port']}|{f['weakness']}".lower()
    return hashlib.sha256(key.encode()).hexdigest()[:12]

def triage(findings: list[dict]) -> dict:
    best: dict[str, dict] = {}
    for f in findings:
        sig = signature(f)
        # Keep the highest-confidence instance of each duplicate.
        if sig not in best or f["confidence"] > best[sig]["confidence"]:
            best[sig] = {**f, "signature": sig}

    report, review = [], []
    for f in best.values():
        # Low confidence, or anything that claims a working exploit, is
        # never auto-shipped. A human confirms impact.
        if f["confidence"] < REVIEW_THRESHOLD or f.get("exploit_claimed"):
            review.append(f)
        else:
            report.append(f)

    order = {"critical": 0, "high": 1, "medium": 2, "low": 3}
    report.sort(key=lambda f: (order.get(f["severity"], 9), -f["confidence"]))
    return {"auto_report": report, "human_review": review}

if __name__ == "__main__":
    raw = [
        {"target": "app.example.com", "port": 443, "weakness": "missing-hsts",
         "severity": "low", "confidence": 0.98},
        {"target": "app.example.com", "port": 443, "weakness": "missing-hsts",
         "severity": "low", "confidence": 0.91},   # duplicate, lower confidence
        {"target": "api.example.com", "port": 443, "weakness": "sqli",
         "severity": "critical", "confidence": 0.74},  # uncertain -> review
        {"target": "api.example.com", "port": 443, "weakness": "idor",
         "severity": "high", "confidence": 0.93, "exploit_claimed": True},  # review
        {"target": "app.example.com", "port": 80, "weakness": "open-redirect",
         "severity": "medium", "confidence": 0.95},
    ]
    out = triage(raw)
    print(f"AUTO-REPORT ({len(out['auto_report'])}):")
    for f in out["auto_report"]:
        print(f"  [{f['severity']:8}] {f['weakness']:14} {f['target']} ({f['confidence']:.2f})")
    print(f"HUMAN REVIEW ({len(out['human_review'])}):")
    for f in out["human_review"]:
        why = "exploit claimed" if f.get("exploit_claimed") else "low confidence"
        print(f"  [{f['severity']:8}] {f['weakness']:14} {f['target']} ({f['confidence']:.2f}) <- {why}")Two findings are treated as the same when they share a target, port, and weakness class, which is what lets the agent cut duplicate reports from several scanners down to one. Everything the agent is confident about, and that does not assert a live exploit, flows to the draft report. Everything else waits for a person. The model does the volume; the human owns the claims.

The failure modes, and what contains them

Every risk in AI-assisted testing maps to a control you can put in place now.

A table of five AI-assisted testing risks, hallucinated findings, scope creep, prompt injection, context loss, and data leakage, each paired with the control that contains it.

Hallucinated findings are contained by a confidence gate and a deterministic re-validation step before anything is reported. Scope creep is contained by the allowlist guardrail, applied to every action and logged. Prompt injection is contained by treating all scanned content as untrusted data that the agent never obeys. Context loss is contained by persisting findings to a store rather than trusting the model's short memory, and summarizing state between steps, an approach the benchmark authors found measurably improved results. Data leakage is contained by isolating each engagement and keeping secrets out of prompts.

A maturity model you can place yourself on

Adoption is not all or nothing. Most teams should move up one level at a time, and only when the gate below is solid.

A four-level maturity model for AI-assisted pentesting, from manual to assisted to supervised to autonomous-in-scope, noting who owns the decision at each level.

At Level 0 the work is fully manual. At Level 1 the AI is a copilot that drafts commands, payloads, and report text while a human runs and reviews everything. At Level 2 the AI runs bounded tasks behind gates, and a human approves exploitation and sign-off. At Level 3 the AI runs the full recon-to-triage loop unattended, while humans still own the scope and the final report. Full autonomy over engagements with real blast radius is not a recommended destination, because the boundary, the authorization, and the impact call are exactly the things machines are worst at and the things that carry the most risk.

A six-step human-in-the-loop workflow: a human scopes and authorizes, the AI runs gated recon and triage, a human approves and drives exploitation, the AI drafts the report, and a human signs off.

A sane engagement, then, looks like a pipeline with automation inside the gates and humans on the boundaries. A person scopes and authorizes. The agent runs recon and triage behind a scope check and a confidence check. A person approves and drives exploitation. The agent drafts the report. A person signs off.

Before you let an agent in

The gap between a useful AI-assisted test and a dangerous one is a short list of gates.

A six-item readiness checklist of gates to confirm before an AI agent joins an engagement, covering scope allowlists, deterministic validation, human exploitation approval, untrusted-content handling, data isolation, and reproducible logs.

Run the list before the agent touches anything. If every action is gated by a scope allowlist the model cannot change, if a deterministic check validates findings before they reach the client, if exploitation sits behind a human approval step, if target content is treated as untrusted, if engagement data is isolated, and if every action can be reproduced from a log, then you have an agent you can supervise. Any gate you cannot check yet is a task to keep manual for now.

The bottom line

AI has not made the penetration tester obsolete. It has split the job. The repeatable, verifiable, high-volume half of the work is now something a machine can do faster and more thoroughly than a person, and refusing that leverage is its own kind of negligence. The other half, the scoping, the creative exploitation, the impact call, and the accountability for what reaches the client, is exactly where machines are weakest and where the stakes are highest.

Automate the first half aggressively, behind gates. Keep a human firmly on the second. And treat the agent itself with the same suspicion you bring to any other system that reads untrusted input and holds a shell.

Key takeaways

The question is not whether AI can pentest, but which tasks to automate, and the split follows one line: how cheaply the output can be verified.

Point automation at recon, enumeration, triage, and the first report draft. Keep humans on scoping, chained exploitation, impact, and sign-off.

AI success tracks the phase. In benchmarks, models handle recon far better than exploitation, and have not finished a full machine unaided.

The agent is an attack surface. Prompt injection, excessive agency, and data leakage are real, so gate every action and treat target content as untrusted.

Adopt one maturity level at a time, and never let automation own the scope or the impact call.

References

NIST: SP 800-115, Technical Guide to Information Security Testing and Assessment (four-stage methodology)

PTES: The Penetration Testing Execution Standard (seven phases)

DARPA: AI Cyber Challenge final results, 2025 (54 million-plus lines analyzed, about $152 per patch, 45-minute average)

XBOW and HackerOne: autonomous agent reaches number one on the global leaderboard, 2025

Isozaki et al.: Towards Automated Penetration Testing, arXiv:2410.17141 (13-machine benchmark, per-phase pass rates, context loss)

OWASP: Top 10 for LLM Applications, 2025 (prompt injection, excessive agency, sensitive disclosure, system prompt leakage)

Figures are drawn from standards-body, government, and peer-reviewed research publications and may be revised. Code examples are illustrative; adapt them to your environment and test them before relying on them.

If you found this useful, follow me for more on offensive security and AI-era risk, and share it with the person who owns your next pentest.