Fighting AI with AI
Let me be blunt: the only way we're going to defend against AI-driven cyber threats is by using AI ourselves.
That's not hyperbole. Adversaries are already using generative AI to write polymorphic malware, craft highly personalized spear-phishing, and automate reconnaissance. These aren't proofs of concept — they're operational realities.
The Federal Bureau of Investigation has issued public warnings about criminals leveraging artificial intelligence for scams, impersonation, and fraud.¹ Meanwhile, research from IBM X-Force and the World Economic Forum highlights the rapid escalation of AI-assisted attack capability across global threat ecosystems.² ³
This is the new asymmetry.
Attackers are operating at machine speed. Defenders are still operating at human speed. And if that doesn't change, we lose.
Which brings me to agentic AI.
Today, true goal-driven autonomy does not yet exist. Large language models are wrapped in orchestration frameworks that make them appear autonomous. They can chain steps, call tools, and adapt superficially. But they do not possess long-term intent, strategic reasoning, or moral accountability.
Still, the trajectory is clear. We are moving toward systems that simulate intent convincingly enough to reshape cyber conflict. And when that day arrives, the most valuable application in cybersecurity will be automated penetration testing.
Why Pentesting Must Lead
I've been in cybersecurity long enough to understand the rhythm of penetration testing. Human pentesters are artists. They improvise. They pivot. They chain weaknesses creatively.
But they are constrained:
- Strict scope boundaries
- Contractual time limits
- Fatigue
- Episodic reporting cycles
Attackers face none of those constraints.
As models from OpenAI and Anthropic grow more capable, offensive actors are embedding them into reconnaissance and exploitation workflows.
We are already seeing concrete signals.
A Chinese AI-assisted penetration testing framework known as Villager, reportedly powered by DeepSeek models, has been widely downloaded and analyzed in security reporting.⁴
The OWASP community has formally documented prompt injection and LLM exploitation techniques as an emerging class of security risk.⁵
The NIST AI Risk Management Framework explicitly calls out adversarial misuse, autonomy risk, and governance failures as systemic concerns.⁶
Researchers have also demonstrated early proof-of-concept AI-assisted ransomware variants, showing how generative models can be folded into offensive toolchains.⁷
Attackers are experimenting with autonomy.
If defenders don't, we're outmatched.
The Advantages of Autonomous Pentesting
Agentic AI aligns almost perfectly with the structural weaknesses of modern defensive security.
Persistence An AI agent doesn't sleep. It doesn't lose focus. It doesn't run out of hours.
Scale Enterprise infrastructure now spans cloud workloads, SaaS platforms, APIs, IoT devices, containers, and hybrid identity systems.
Adaptability When blocked, a human attacker pivots. An advanced agentic system could test alternative exploit paths instantly.
Proactivity Incident response is reactive. Continuous autonomous pentesting applies adversarial pressure before a breach occurs.
For the first time, defenders could maintain a permanent red team — continuous, not quarterly.
Analytics and Incident Response: Important but Secondary
Security Operations Centers are already integrating AI into alert triage and enrichment workflows.
Research from MITRE and the SANS Institute emphasizes automation's role in reducing dwell time and improving detection fidelity.⁸ ⁹
These are critical advances.
But they remain reactive.
Pentesting is adversarial and proactive. That makes it the ideal proving ground for agentic systems.
The Dangers of Overconfidence
The biggest risk isn't the technology.
It's us.
We already over-trust AI. Confident output is mistaken for accurate output. In cybersecurity, that mistake can be catastrophic.
The NIST AI Risk Management Framework stresses accountability, oversight, and traceability for precisely this reason.⁶
Every deployment of autonomous pentesting must follow three principles:
Humans in the Loop No critical exploit execution without validation.
Transparency Every recommendation must provide explainable reasoning and traceable decision chains.
Fail-Safes Escalation triggers, stop conditions, scope enforcement, and kill switches are mandatory.
Machines do not own the consequences.
We do.
Governance and Legal Exposure
Autonomous pentesting introduces legal and compliance risk:
- Potential exposure under computer misuse statutes if scope controls fail
- Data handling violations if live data is exfiltrated
- Regulatory scrutiny under emerging AI governance standards
Alignment with frameworks such as the NIST AI RMF and guidance from OWASP is foundational.
Final Thoughts: Why Humans Still Matter
Automated penetration testing will likely be the first true proving ground for agentic AI because autonomy's strengths — persistence, scale, adaptability — align perfectly with adversarial simulation.
This does not eliminate human pentesters.
It amplifies them.
Machines provide scale and speed. Humans provide judgment, ethics, creativity, and accountability.
The stronger the machine becomes, the more essential the human becomes.
Agentic AI may illuminate every flaw in our systems.
But only humans can decide what to do with that knowledge.
And that is why pentesting must lead.
About the Author
Len Noe is a Divergent Research Engineer for CW-PenSec, a transhuman, Podcaster, international cybersecurity speaker, author, technical evangelist, and biohacker with 11 implanted microchips. A former blackhat with more than 30 years in technology, he has presented in over 70 countries and is featured in the documentary I Am Machine, which premiered at DEF CON 2025. https://www.i-am-machine.com
References
- Federal Bureau of Investigation (FBI). Criminals Use Artificial Intelligence for Fraud and Scams. https://www.ic3.gov/Media/Y2023/PSA230321
- IBM Security. X-Force Threat Intelligence Index 2024. https://www.ibm.com/reports/threat-intelligence
- World Economic Forum. Global Cybersecurity Outlook 2024. https://www.weforum.org/publications/global-cybersecurity-outlook-2024/
- TechRadar. AI-powered pentesting tool downloaded 10,000+ times. https://www.techradar.com/pro/security/a-mysterious-chinese-ai-pentesting-tool-has-appeared-online-with-over-10-000-downloads-so-far
- ITPro. DeepSeek-powered pen-testing tool raises concerns. https://www.itpro.com/technology/artificial-intelligence/this-deepseek-powered-pen-testing-tool-could-be-a-cobalt-strike-successor-and-hackers-have-downloaded-it-10-000-times-since-july
- OWASP. Top 10 for Large Language Model Applications. https://owasp.org/www-project-top-10-for-large-language-model-applications/
- NIST. AI Risk Management Framework (AI RMF 1.0). https://www.nist.gov/itl/ai-risk-management-framework
- Infosecurity Magazine. First AI-powered ransomware identified. https://www.infosecurity-magazine.com/news/first-ai-powered-ransomware/
- TechRadar. AI-powered ransomware spotted. https://www.techradar.com/pro/security/the-first-ai-powered-ransomware-has-been-spotted-and-heres-why-we-should-all-be-worried
- MITRE. Adversarial Threat Landscape for Artificial Intelligence Systems. https://attack.mitre.org/resources/ai/
- SANS Institute. The Role of Automation in Modern SOCs. https://www.sans.org/white-papers/automation-modern-soc/