Can AI Find Vulnerabilities Better Than Humans?

What Real-World Security Research Has Actually Shown

For years, vulnerability research has been a deeply human process.

A researcher reads unfamiliar code, builds a mental model of how the software works, notices something unusual, forms a hypothesis, writes a test, and investigates the result.

Now AI agents are entering that workflow.

So the interesting question is no longer:

Can AI theoretically find vulnerabilities?

We already have evidence that it can.

The better question is:

How capable is AI at finding real vulnerabilities in real software?

Instead of relying on predictions, let's look at documented cases.

The SQLite Case

One of the clearest examples comes from Google Project Zero and Google DeepMind.

Their AI security research system, Big Sleep, was used to investigate SQLite, one of the world's most widely deployed database engines.

In October 2024, Big Sleep identified a previously unknown, exploitable memory-safety vulnerability in SQLite. Google reported the vulnerability to the SQLite developers, who fixed it the same day. The issue was found before it appeared in an official SQLite release.

That is already significant.

But the interesting part is how the system reached the finding.

Big Sleep wasn't simply prompted with:

"Find a vulnerability."

The system was designed around a vulnerability-research workflow involving code analysis, hypothesis generation, testing, and investigation.

During the SQLite investigation, it analyzed a change involving SQLite's handling of WHERE terms and eventually produced a crashing SQL input involving generate_series and ROWID.

Project Zero's researchers investigated the result and determined that it represented an exploitable stack buffer underflow.

The AI had produced something that could be investigated and validated as a real vulnerability.

What Happened When Traditional Fuzzing Was Used?

Project Zero also investigated whether conventional fuzzing would have found the same issue.

Using AFL against SQLite's CLI configuration, the researchers reported that the vulnerability remained undiscovered after 150 CPU-hours of fuzzing.

But there is an important caveat.

Project Zero explicitly noted that AFL was not necessarily the ideal tool for this particular SQL-oriented problem. A target-specific fuzzer could potentially have performed better.

So this result should not be interpreted as:

AI defeated fuzzing.

The narrower conclusion is more useful:

Big Sleep found a real vulnerability through a reasoning-driven workflow that the particular fuzzing setup did not uncover during the reported test.

Was SQLite Just a One-Off?

That is the obvious next question.

Larger evaluations provide additional evidence.

CyberGym is a benchmark built around real-world vulnerabilities from open-source projects.

Its dataset contains 1,507 vulnerability-analysis tasks across 188 projects, with vulnerabilities sourced from OSS-Fuzz.

The benchmark goes beyond asking an AI to describe a potential vulnerability.

For vulnerability reproduction, the agent must produce a working proof of concept.

The PoC is executed against the vulnerable version and checked against the patched version.

That changes the standard of evidence.

Instead of:

"The AI thinks this is vulnerable."

the question becomes:

"Did the generated proof of concept actually trigger the vulnerability?"

AI Agents Finding Previously Unknown Vulnerabilities

CyberGym also evaluated open-ended vulnerability discovery.

In this setting, agents receive current codebases without being told where the vulnerabilities are.

According to CyberGym's published results, OpenHands was evaluated with GPT-4.1 and GPT-5 across 431 OSS-Fuzz projects containing 1,748 executables.

The reported results were:

ModelCrashes TriggeredConfirmed Zero-DaysGPT-4.1167GPT-55622

Four of the GPT-5 findings overlapped with findings from GPT-4.1.

These are not hypothetical vulnerabilities generated from synthetic code.

They were reported as previously unknown vulnerabilities discovered in real open-source projects.

Finding a Bug Is Only the Beginning

A vulnerability research workflow does not end when a crash occurs.

An agent may need to:

1. Identify suspicious behavior
2. Produce a reproducible proof of concept
3. Determine the root cause
4. Develop a fix
5. Verify that the fix works
6. Ensure the fix doesn't break the software

This is why newer benchmarks are moving beyond vulnerability discovery.

From Discovery to Remediation

CyberGym-E2E evaluates a broader vulnerability lifecycle.

The benchmark currently contains 920 real-world vulnerabilities across 139 open-source projects.

It evaluates four stages:

S1 — Proof of Concept

Can the agent create a working PoC that triggers the vulnerability?

S2 — Patch Validation

Does the proposed patch stop the vulnerability?

S3 — Application Correctness

Does the software continue functioning after the patch?

S4 — Vulnerability Resolution

Did the patch actually address the underlying vulnerability rather than simply suppressing one symptom?

According to the published leaderboard, GPT-5.4/Codex achieved 67.9% at S1 and 22.2% at S4 under the listed evaluation configuration.

That gap is revealing.

Generating a working PoC is considerably easier than completing the entire vulnerability lifecycle correctly.

Microsoft Is Doing This Internally Too

The idea is also appearing in enterprise security research.

In May 2026, Microsoft described MDASH, a multi-model agentic scanning system designed for vulnerability discovery.

Microsoft reported that MDASH helped identify 16 new vulnerabilities across Windows networking and authentication components, including four critical remote-code-execution vulnerabilities.

Microsoft also reported that the system found all 21 planted vulnerabilities in one private test driver with zero false positives and achieved an 88.45% CyberGym score under its reported configuration.

These figures are Microsoft's own reported results, so they should be treated as vendor-reported evidence rather than independently reproduced measurements.

Still, the direction is clear: AI-assisted vulnerability research is moving beyond laboratory demonstrations and into internal security workflows at major technology companies.

So, Is AI Better Than Humans?

The evidence does not establish that AI has surpassed expert human vulnerability researchers.

The studies above generally do not provide a controlled, apples-to-apples competition between an AI agent and an expert researcher using identical targets, tools, time limits, and conditions.

What they do establish is narrower — and arguably more interesting:

AI systems are capable of performing meaningful vulnerability-research tasks on real software.

Big Sleep found a real SQLite vulnerability.

CyberGym reported previously unknown vulnerabilities from open-ended AI-agent testing.

Microsoft reported new vulnerabilities discovered through its agentic security system.

Those are substantially different from asking a chatbot to review a small piece of code.

Where Humans Still Matter

Vulnerability research involves more than finding a technical flaw.

Researchers also need to understand:

• Application context
• Business logic
• Exploitability
• Security impact
• Attack chains
• Responsible disclosure
• Remediation trade-offs

AI can assist with parts of this process, but the documented results also show why verification matters.

A generated crash is not automatically a security vulnerability.

A potential vulnerability is not automatically exploitable.

And a patch that stops one test case is not necessarily a correct security fix.

The further the task moves from finding something suspicious toward understanding and fixing the underlying problem, the more demanding the task becomes.

What the Evidence Actually Tells Us

The most defensible conclusion from these cases is not that AI has replaced human security researchers.

It is that the capabilities of AI vulnerability research have moved from theoretical possibility to demonstrated results.

We now have documented examples of AI systems:

• Finding previously unknown vulnerabilities
• Generating reproducible proof-of-concepts
• Discovering bugs in real open-source projects
• Analyzing large codebases
• Assisting with vulnerability remediation

The remaining challenge is reliability.

Can an AI agent consistently distinguish a real vulnerability from a false positive?

Can it understand complex business logic?

Can it determine whether a vulnerability is actually exploitable?

Can it produce a correct patch without introducing another problem?

And eventually:

Can an AI agent complete the entire vulnerability lifecycle reliably enough to operate with minimal human intervention?

The research is moving toward that question.

And the answer, based on the evidence available today, is still being worked out.

Sources

• Google Project Zero — From Naptime to Big Sleep: documented Big Sleep's discovery of the SQLite vulnerability and comparison with conventional fuzzing.
• CyberGym: benchmark covering real-world vulnerability-analysis tasks and open-ended vulnerability discovery.
• CyberGym-E2E: evaluates vulnerability discovery, PoC generation, patching, and validation.
• Microsoft Security — MDASH: Microsoft's report on its multi-model agentic vulnerability-discovery system.

Research note

This article is an analysis of publicly documented security research. It does not claim that the author independently reproduced the experiments or discovered the vulnerabilities discussed above.

All security testing discussed in the cited research was conducted within authorized research environments.