How I built a multi-agent AI recon tool that refuses to guess

After some break we are back again with some exiting topics. This time the topic is to how to use AI efficient in cybersecurity. Do not worry this is not one of the blog posts written with a sugar coating AI language :) .

Like me you have seen many post where pen testing is supposedly all done by AI etc. Some of the are too good to be true and are "too fantastic". In this blog post I will share with you a realistic solution.

Every lab pen-test starts the same way. Run nmap. Wait. Run nikto. Wait. Run whatweb, gobuster, ffuf, wpscan, sslscan. Seven tools, seven blocks of terminal output — then an hour of reading through all of it and writing it up into something with a risk table and remediation notes. It is all the same again and again and especially in the age of automation it only makes sense to use what we are offered.

None of that is hard. It's just mechanical. So I automated it.

Not with a chatbot that guesses what your network probably looks like. With a pipeline that actually runs the tools, and only lets AI reason over what they actually printed.

What it solves, and what you actually get out of it

The problem isn't "recon takes tools." It's that recon takes tools and then a person has to read seven different output formats, cross-reference them, and turn that into a coherent writeup — and that part doesn't compress, no matter how fast the scans run.

What this gets you:

• The grunt work done for you. One command plans the recon, runs it for real, and hands you a finished report — not a to-do list.
• A report grounded in evidence, not vibes. Every claim traces back to an actual line of tool output. If you ask "where did this finding come from," the answer is always "this exact command printed this exact thing," never "the model thought it was likely."
• Honesty about uncertainty. CVEs get labeled Confirmed, Candidate, or Needs-lookup depending on how sure the evidence actually makes it — instead of either overclaiming or silently dropping anything it's not 100% sure about.
• A safety net that doesn't rely on you remembering it. It physically cannot run a tool outside a small allowlist, or point that tool at a target you didn't authorize. That's enforced in code, not just written in a warning banner.
• A pattern you can steal. If you're building anything that hands real command output to an LLM and expects structured data back, the guardrail and evidence-parsing approach here is the actual reusable idea — more on that below.

How it works, without the deep dive

I am also one of the type getting bored when diving in too much so let us keep it simple. Not forget to mention that I was inspired by a You Tube video in developing this solution but the You Tube one has some flaws.

Three stages, each with one job:

1. Plan. Given a target, it produces a recon plan — which tools, what order. But the tool doesn't trust the model's exact command syntax. It swaps in a fixed, hand-written template for the real target, so the model decides that recon happens, not the exact shell command that touches the network.
2. Run, with a gate in front. Before any command executes, it's checked against two rules: is the binary on a small fixed allowlist (nmap, nikto, whatweb, gobuster, ffuf, wpscan, sslscan — nothing else, ever), and is the target private/lab/explicitly authorized. Fail either check and the whole run stops before anything touches the network.
3. Analyze, then report. The real stdout/stderr from every tool gets handed to the model along with a pre-parsed summary (open ports, versions, TLS findings, discovered paths). It comes back as structured findings with confidence-labeled CVEs, then a second pass turns that into the actual report: executive summary, risk table, detailed findings, remediation roadmap.

That's it conceptually. The plan doesn't touch the network. The execution is locked down. The write-up only ever describes what actually happened.

One thing worth being upfront about: two different AI systems are involved, doing two different jobs. I built the tool itself with Claude (Anthropic's Claude Code) as my pair programmer — the architecture, the guardrails, the tests. What the tool calls at runtime, reading your scan output and writing your report, is OpenAI's GPT-4o. One helped me write the code. The other runs when you actually use it. You could make it work in any way, it is all about the concept.

How to deploy it

So where is this solution? Check out https://github.com/gbutun/multi_agent_ai_cyber_code

pip install ankacore-attack-chain
export OPENAI_API_KEY="sk-..."
attack-chain --target dvwa.local

pip install ankacore-attack-chain
export OPENAI_API_KEY="sk-..."
attack-chain --target dvwa.local

That's the whole thing. It generates or reuses a recon plan, validates every command against the safety rules, runs the tools, and writes four files: the plan, the raw scan output, the AI's structured analysis, and the final report — plain files, nothing hidden behind a dashboard.

If you don't already have the seven underlying tools installed, there's a setup_req.sh you can run first — it checks for each one and installs whatever's missing through your system's package manager (apt-get, brew, dnf, or pacman, whichever it finds).

Two flags worth knowing before your first real run:

• --dry-run walks the entire plan-and-validate pipeline without touching the network. Use this to sanity-check a target before you actually scan it.
• --report-only reuses scan data you already have and just re-runs the AI side. Useful if you're iterating and don't want to re-scan the same target every time.

Requirements

• Python 3.9+
• Your own OpenAI API key — the agents call the OpenAI API directly; nothing is proxied or metered for you.
• The seven tools themselves, on your PATH — nmap, nikto, whatweb, gobuster, ffuf, wpscan, sslscan. Standard packages on Kali/Parrot, available via apt/brew elsewhere. Run setup_req.sh and it'll install whichever ones you're missing; the chain itself just checks that a command's binary is allowlisted, it doesn't install anything mid-run.
• A target you're actually authorized to test. The guardrails block everything else, but that's a backstop, not a substitute for having permission in the first place.

One thing that broke, worth knowing

ffuf prints its ASCII-art banner to stderr even on success. My first pass at summarizing tool output defaulted to reading stderr when it wasn't empty — reasonable assumption, wrong for this one tool. That banner's backslashes ended up in what I handed the model, the model copied them into JSON without escaping properly, and the entire structured response silently fell back to raw text — every finding gone, no error, just quietly empty.

The lesson: if your pipeline hands raw tool output to an LLM and expects structured data back, test the parsing before you trust it. The failure mode is silent. You get a technically successful response missing everything you actually needed.

That's the real shape of this project — not "AI writes your pentest report," but a real tool-execution pipeline with a hard safety allowlist, wrapped in just enough structure that an LLM can turn messy terminal output into something you can act on, and tells you when it isn't sure instead of making something up.

Conclusion

Let us wrap this up, we just generate a plan which is running the usual suspects and have it evaluated by AI. The generation of the parameters and feeding the tool is not the magic , the magic is when to have those outputs evaluated by AI and in our case it is Claude. At the end we have a report with risks, vulnerabilities and CVEs. It is not perfect but soves a lot of things.

I hope you will find it useful, stay safe.