Post cover image
Cover: an AI security agent linking an application's source code to its running web interface, tracing an attack path to a single verified finding.

October 14, 2026

From Source Code to Exploit: How Shannon Rethinks AI Penetration Testing

A technical walkthrough of the open-source, code-aware AI pentester — its agent architecture, its “no exploit, no report” philosophy, how…

By Lakshitha Wijerathne

14 min read

A technical walkthrough of the open-source, code-aware AI pentester — its agent architecture, its "no exploit, no report" philosophy, how to run it safely on a lab app, and where its limits are.

The release is green. Unit tests pass, the static-analysis scan comes back clean, the container deploys. Three weeks later, a customer discovers they can read someone else's invoices by changing a number in a URL. Nothing in the pipeline was wrong, exactly. It just never asked the one question that mattered: can someone actually do this?

That question is the dividing line between most automated tooling and what a human penetration tester does. A scanner tells you a piece of code looks dangerous. A pentester tells you whether an attacker, starting from the outside, can actually reach it and turn it into a breach. The first produces suspicions; the second produces evidence.

Shannon, an open-source project from Keygraph, tries to push an AI system across that line — to go beyond flagging suspicious code and instead attempt to prove a vulnerability against a running application. This is a technical look at how it's built, what it genuinely does, how to experiment with it safely, and where it falls short. It isn't a review or an endorsement: there's no credible independent benchmark of Shannon's effectiveness yet, and I'll be explicit about which claims come from the vendor and which are verifiable from the code.

Everything here was checked against the repository on 13 October 2026, at release v3.4.0. Shannon moves fast, so verify commands and features against the current repository before relying on them.

1. What is Shannon?

Shannon describes itself as "an autonomous, AI pentester for web applications and APIs." Its tagline is blunt: "No exploit, no report." The idea is that it analyzes your application's source code to identify attack paths, then uses browser automation and command-line tools to try to execute real exploits against the running application — and reports only the vulnerabilities it can back with a working proof of concept.

Three things shape everything else:

  • It is genuinely open source. The repository is licensed under AGPL-3.0. You run it yourself, locally or in CI, and you pay only your own model-provider costs. Keygraph also sells commercial editions (Pro and Enterprise) on an enhanced version of the same engine, adding capabilities the open-source tool lacks — black-box testing without source, dependency and secrets scanning, findings management. The open-source project is the focus here.
  • It is white-box by design. Shannon expects your source code. That's the "code-aware" part — it reads the code to decide what to test and where to look, rather than probing blindly from the outside.
  • It is not pitched as a replacement for people. The project is unusually direct: Shannon is "built to work alongside expert pentesters and red teamers, not replace them." Its purpose is to close the gap between how often teams ship (constantly) and how often most apps get a real assessment (rarely), by making exploitation-backed testing cheap enough to run against staging at the cadence software actually ships.

The name nods to Claude Shannon, the founder of information theory; the team frames pentesting as an information problem, where each probe reduces uncertainty about a system's state.

2. Why traditional security testing isn't enough (and why it's still necessary)

"Traditional" testing isn't one thing, and it's a mistake to treat any single approach as universally better. Each sees something the others miss. The useful question is what kind of signal each one produces.

None

Static analysis reads code at scale, but a pattern that looks like SQL injection may sit behind an auth check, a parameterized query, or a code path no user can reach. A scanner exercising the running app has the opposite problem: it can confirm a response but has no idea why the code behaves that way. A skilled human closes the loop — reads the code, forms a hypothesis, tests it live, confirms impact — but humans don't scale to every internal app and API an organization runs.

Shannon's bet is that an AI agent can do a narrow version of that loop: use the code to form specific, credible hypotheses, then try to confirm them against the live application. It doesn't make the other approaches obsolete — dependency scanning, secrets detection, configuration review, and deep manual assessment all cover ground it explicitly does not.

None

3. How Shannon works: from source code to evidence

Under the hood, Shannon is a TypeScript application orchestrated by Temporal (a durable workflow engine) and driven by the Pi agent harness. The language models do the reasoning; Temporal makes the long, multi-stage run durable and resumable. Each scan runs inside an ephemeral Docker container, with your repository mounted read-only and results written to a local workspace.

The pipeline moves through several stages — some sequential, some parallel. Here's the verified shape of it, from the source and the project's own architecture notes:

  1. Preflight and validation. Before any testing, Shannon validates your configuration, checks the chosen model is reachable and cleared for security workloads, and — if you've configured authentication — verifies it can log in. You can run these checks alone with --validate-model and --validate-auth.
  2. Pre-recon (code analysis). The first agent reads the source to build a security-relevant map of the application: architecture, trust boundaries, interfaces, data flows, and the files worth a closer look. It's the only stage with full source access, so its thoroughness sets the ceiling for everything downstream.
  3. Recon (correlation). A second agent correlates live application behavior with the source — building an attack-surface map, not yet hunting bugs. Pre-recon and recon run one after the other; the rest of the pipeline depends on their output.
  4. Vulnerability analysis — in parallel. Shannon runs specialized analysis agents across five classes at once: Injection, Cross-Site Scripting (XSS), Broken Authentication, Broken Authorization, and Server-Side Request Forgery (SSRF). Each produces an analysis plus a structured queue of candidate findings. The class set is fixed; there's no selector to narrow it.
  5. Finding reconciliation. The convergence point. If you've enabled Shannon's optional agentic static analysis (it runs concurrently from the start), its findings are merged here with the vulnerability-analysis candidates. Reconciliation deduplicates overlap and groups what remains into one exploitation queue per class. The project is careful here for a reason worth quoting: "A false merge can hide a real vulnerability. A missed merge leaves a visible duplicate."
  6. Exploitation — in parallel. For each class, an exploitation agent works the reconciled queue and attempts real proof-of-concept attacks against the running application — but only if its class has candidates to test. A sixth "miscellaneous" lane handles findings that don't fit the five named classes.
  7. Validation and reporting. Candidates Shannon can't demonstrate are discarded. As the project puts it: "Only live-validated vulnerabilities become Shannon pentest findings or count toward CI/CD severity gates."

Output comes in several formats at once: an evidence-rich PDF and Markdown report, plus structured JSON and SARIF 2.1.0 (the OASIS-standard format that security dashboards and code-scanning services read).

None

4. Understanding the agent architecture

What stands out isn't one clever agent — it's the decomposition. Instead of a single model holding an entire penetration test in its head, the work is split into specialized agents, each with a narrow job and a defined input and output.

Concretely, the pipeline is a set of named agents wired together by prerequisites: a pre-recon and a recon agent; then a *-vuln analysis agent and a matching *-exploit agent for each of the five classes (injection-vuln → injection-exploit, and so on); a miscellaneous-exploit agent; and a final report agent. Each writes a named deliverable that later stages consume. Temporal tracks each agent's progress, which is what lets an interrupted scan resume without repeating completed work.

Why decompose it? The same reasons you'd decompose any complex system:

  • Bounded context. A model asked only to find authorization flaws, given a clean attack-surface map, has a far more tractable problem than one asked to "test everything."
  • Parallelism. Five independent class pipelines run concurrently, so one slow class doesn't stall the others; each proceeds independently once its own analysis and reconciliation are done.
  • Durability and auditability. Because each stage produces an artifact and reports its own status, a class that fails to complete is recorded as incomplete rather than silently reported as clean — an important distinction when "no findings" could mean "nothing there" or "never checked."

The architecture above is what the code and docs actually describe; I've avoided guessing at internal prompt wording or model configuration beyond that.

5. What makes Shannon interesting

The most interesting idea in Shannon is also its simplest: a suspicion and a proof are not the same thing, and the tool refuses to treat them as if they were.

Most automated tooling hands you candidates and leaves validation to you. Shannon's design inverts that: a candidate that can't be demonstrated against the running application is discarded, not downgraded to "low confidence." That has a real, practical consequence — it attacks the false-positive fatigue that makes so many security reports go unread. If a finding made it into the report, Shannon is asserting it reproduced something.

That choice deserves a careful reading, though. Validating by exploitation reduces one specific kind of error — the false positive, the flagged issue that isn't actually exploitable. It does not make the results complete or correct. Shannon can still miss real vulnerabilities (false negatives), misjudge severity, or produce a report with weakly supported details — the project says so itself. "Proven" here means "the agent reproduced something under test conditions," not "an auditor signed off."

The second property is that it's code-informed. Because Shannon reads the source first, it targets its testing instead of probing blindly, and can tie a runtime observation back to the specific code responsible. That's what a pure black-box scanner can't do, and why "code-aware" best describes the category.

None

6. Getting started with Shannon (safely)

⚠️ Authorization first. Shannon actively executes exploits. The project is explicit: run it "only against applications and environments you own or have explicit written authorization to test. Do not run Shannon against production systems." The commands below assume a disposable, local lab application you control.

Prerequisites: Docker, Node.js 18+, and credentials for an AI provider. On Windows, Shannon runs through WSL2 — native Windows (including Git Bash) is not supported. Anthropic and OpenAI apply real-time safeguards to security workloads that can interrupt a run, so legitimate testers should complete their provider's verification first; --validate-model checks this before you commit to a full scan.

A good first target is a deliberately vulnerable app you run yourself — for example, OWASP Juice Shop, which starts locally from its public Docker image and whose source is on GitHub. Because the scanner runs inside a container, a service on your host machine is reached as host.docker.internal, not localhost.

# 1. Configure provider credentials interactively.
npx @keygraph/shannon@latest setup

# 2. Validate your model is reachable and cleared, before a full run.
npx @keygraph/shannon@latest start \
  -u http://host.docker.internal:3000 \
  -r /path/to/local/app-source \
  --validate-model

# 3. Run a full pentest against your authorized local target.
npx @keygraph/shannon@latest start \
  -u http://host.docker.internal:3000 \
  -r /path/to/local/app-source \
  -w my-first-audit

A few arguments are worth understanding rather than copying. -u/--url is the running target and -r/--repo is its source (mounted read-only). -c/--config points at a YAML file for authenticated testing, scope rules, and report filters. -w/--workspace names the run — re-using the same name resumes an interrupted scan (the URL must match); there's no separate resume command. And --validate-auth runs preflight plus a single real login, then stops, so you can confirm a login flow before a long run.

While it runs, npx @keygraph/shannon status shows phase-by-phase progress and logs tails the live log. When it finishes, the report lands in the workspace (under ~/.shannon/workspaces/ in npx mode) as Security-Assessment-Report.pdf and .md, with run internals under a hidden .shannon/ directory. For authenticated apps, a small config file lets you describe a login flow in plain language (with $username / $password / $totp placeholders), narrow or emphasize parts of the app with avoid / focus rules, and set rules_of_engagement such as request throttling. Full details are in the configuration guide.

7. A practical example (hypothetical)

This scenario is illustrative, not a real scan. It describes conceptually how a finding would move through the pipeline. No real output is shown, and there are no payloads here.

Imagine a lab e-commerce app with an endpoint that returns an order by its numeric ID. During pre-recon, the code-analysis stage notes the handler looks up the order by ID but never checks that it belongs to the requesting user. That's a classic Broken Access Control pattern — the top category in the OWASP Top 10:2025 — but at this point it's only a hypothesis. Plenty of code that looks like this is actually fine: the ownership check lives in middleware, the ID is unguessable, or the route is unreachable.

The authorization analysis agent turns that observation into a specific candidate: this endpoint may let one user read another user's orders. Reconciliation deduplicates it against anything the optional static-analysis pass found and places it on the authorization exploitation queue.

Now the distinction that defines Shannon kicks in. The exploitation stage doesn't take the candidate on faith. Using two separate test accounts in the lab environment, it attempts to reach one account's order while authenticated as the other. If the app returns the data, Shannon has reproduced the issue and records the evidence a developer needs to see it for themselves — which account, which request, which response. If the app correctly blocks the attempt, the candidate is discarded.

That evidence is what makes the finding useful. A developer doesn't relitigate whether it's real; they reproduce it, add the missing ownership check, and re-run the scan to confirm the fix. The gap between "this code looks risky" and "here is a reproduction of user A reading user B's data" is the gap between a ticket that gets argued about and one that gets fixed.

8. Integrating AI penetration testing into CI/CD

Shannon ships official integrations for GitHub Actions and GitLab CI/CD, which is what makes the "shift-left" framing more than marketing. Both analyze the checked-out source while attacking a running target, preserve the PDF/Markdown/SARIF reports as artifacts, and can optionally fail a build when a vulnerability is exploited at or above a chosen severity.

None

Wiring this into a pipeline, a few practices matter:

  • Staging only, never production. Point it at a disposable, isolated environment with throwaway data — exactly as the safety guidance requires.
  • Gate on proven findings. Shannon's severity gate "counts only findings Shannon actually exploited." A code-analysis hypothesis does not fail the pipeline. This keeps the gate from firing on unproven noise — but note that the GitHub Action ships with fail-on-severity defaulting to none, so you must set a threshold deliberately.
  • Mind the clock and the cost. The docs describe a typical run as roughly one to 1.5 hours, and the Action defaults to a six-hour job timeout; the vendor's own benchmark runs ranged from about 2.5 to 5.5 hours. A full exploitation scan is not a pre-commit hook — scheduled or release-gated runs fit far better than blocking every push.
  • Treat secrets and permissions carefully. Provider keys belong in CI secrets. The Action refuses to run on public repositories (reports can contain sensitive findings) and needs security-events: write to publish SARIF to code scanning.
  • Keep a human in the loop. Publish SARIF so findings land in your code-scanning UI with locations, and route them to someone who triages before anything is treated as ground truth.

9. Limitations and risks

An honest article has to spend real time here, because autonomous exploitation is not a free lunch.

  • Bounded scope. The open-source tool targets five vulnerability classes. Whole categories — vulnerable dependencies, misconfiguration, secrets, business-logic flaws, broad policy issues — are explicitly out of scope. "No findings" means "none of these kinds, that could be proven."
  • False negatives are real. Proof-by-exploitation reduces false positives; it does nothing to guarantee coverage. A missed vulnerability produces no finding and no warning.
  • Model variability. Results depend heavily on the model you choose. A weaker model that doesn't follow instructions or tool constraints reliably produces weaker, less stable runs. Provider-side safety safeguards can also cause a model to refuse mid-run.
  • Human review remains essential. The project states plainly that "LLM-generated reports can contain weakly supported or incorrect details." Evidence lowers the review burden; it doesn't remove it.
  • Exploitation has side effects. By design, the agents "can create users, submit forms, mutate application state, trigger outbound requests, and otherwise affect the target system." That's exactly why production is off-limits and a disposable VM is recommended.
  • Prompt injection from untrusted code. Because Shannon reads source, a hostile repository could try to manipulate it — a documented risk aligned with OWASP's guidance on prompt injection. The advice is direct: "Do not scan untrusted or adversarial codebases."
  • Effectiveness isn't independently established. The comparison numbers Keygraph publishes (against apps in an Aikido-sponsored Doyensec study) are vendor-run and vendor-verified, not independent. Treat them as a starting point, and run your own trials on apps you understand.
  • AGPL and cost. The AGPL-3.0 license carries real obligations for anyone building a service on top of it, and you pay your own model costs for every run.

10. Who should explore Shannon?

Shannon is most compelling for teams that ship faster than they can get security coverage: developers and small teams who want exploitation-backed testing between annual assessments; DevSecOps practitioners wiring recurring checks into staging pipelines; and AI-agent researchers who want a substantial, real-world example of multi-agent orchestration with durable workflows and validated outputs.

It's a complement, not a substitute. When you need breadth — dependency and configuration coverage, secrets, business-logic testing — a conventional scanner or a purpose-built platform still earns its place. When stakes are high or the application is novel, a skilled human assessment remains irreplaceable for the business context and creative attack-chaining current models can't match. The right mental model is another layer in defense-in-depth, run far more often than a human can, on the long tail of apps that otherwise get no testing at all.

11. Final thoughts

Shannon is interesting less because of any single capability and more because of a discipline it builds into its design: the refusal to treat a suspicion as a finding. In a field drowning in scanner output nobody has time to read, "show me the proof, or drop it" is a genuinely useful stance — as long as you remember what proof does and doesn't mean. It cuts false positives; it doesn't promise coverage, correctness, or a replacement for human judgment.

The broader signal is that AI-assisted security engineering is shifting from pattern-matching toward investigation — systems that read code, form hypotheses, test them against a live target, and show their work. Whether that reliably holds up across messy real-world applications is exactly the kind of question the community should be testing in the open, with authorized targets and a healthy skepticism.

If you want to see it for yourself, the project lives at github.com/KeygraphHQ/shannon. Stand up a deliberately vulnerable app you control, read the safety guidance first, and run it against something you're allowed to break.

If you've used Shannon or another autonomous security tool, what did it actually catch — and, just as important, what did it miss? That second answer is the one the field needs more of.

References

Written October 2026. Verified against Shannon v3.4.0; the project changes quickly, so check the repository for current commands and features before relying on them.