July 25, 2026

I Built an AI-Powered Security Code Review Tool

By Sanaullah Aman Korai

4 min read

I Built an AI-Powered Security Code Review Tool. Here's What I Learned.

I review a lot of code. As an application security engineer, I spend hours reading other people's pull requests looking for SQL injection, broken auth, path traversal — the usual suspects. After doing this for three years across 70+ engagements, I started wondering: can I offload the first pass to an LLM?

Not replace the human review. Just triage. Let AI flag the obvious stuff so I can focus on the tricky business logic bugs that actually need a human brain.

So I built something. It's called ai-sec-review, and it does two things: security code review and STRIDE threat modeling, both powered by LLM APIs. It's open source, it took about a weekend to get working, and here's how it came together.

The Core Idea

The problem with asking ChatGPT or Claude "is this code secure?" is that you get a wall of text back. Sometimes it hallucinates bugs that don't exist. Sometimes it misses the obvious SQL injection sitting right there. And the output format is never consistent — one time you get bullet points, next time a paragraph, next time a table.

What I needed was structured output. Same JSON schema every time. Validated. Machine-readable.

Anthropic's API has a feature called tool use where you define a schema and the model is forced to return data matching it. No regex parsing. No "please format your response as JSON." The model calls your tool with validated JSON, or it fails.

That was the foundation.

How It Works

The architecture is dead simple:

  1. Scanner — walks your directory, finds code files, detects the language by extension, reads the content (with smart truncation for files over 600 lines so you don't burn tokens)
  2. Reviewer — takes that code, wraps it in a security-focused system prompt, and sends it to the LLM with a tool schema defining exactly what a vulnerability finding looks like (title, severity, CWE ID, location, description, impact, remediation, code snippet)
  3. Threat Modeler — same pattern but for architecture descriptions. You describe your system ("React frontend, Express API, PostgreSQL, JWT auth, on AWS") and it returns a full STRIDE model with trust boundaries, categorized threats, and mitigations
  4. Reporter — formats the structured output as colored terminal output, JSON (for CI/CD), or Markdown (for reports)

The whole thing is about 800 lines of Python. No external dependencies beyond the standard library — I used urllib to call the API directly instead of the Anthropic SDK, which keeps it portable.

The Tool Schema Is Everything

This was the hardest part to get right. The tool schema defines your output contract. Too loose and the model rambles. Too strict and it finds nothing.

Here's what I landed on for each finding:

{
  "title": "SQL Injection in Login Endpoint",
  "severity": "CRITICAL",
  "cwe_id": "CWE-89",
  "vuln_class": "SQLi",
  "location": "app.py, line 9",
  "description": "User input interpolated directly into SQL query...",
  "impact": "Attacker can bypass auth and exfiltrate database...",
  "remediation": "Use parameterized queries: cursor.execute(\"...\", (user, pass))",
  "code_snippet": "query = f\"SELECT * FROM users WHERE pass='{pw}'\""
}

The key design decision: required fields. I made title, severity, description, and remediation required. cwe_id, location, impact, and code_snippet are optional. This means the model always gives you something actionable, but it can skip the optional fields if it's not confident — which reduces hallucination.

Testing It on Real Code

I tested it on a deliberately vulnerable Flask app. Four endpoints, three vulnerability classes:

  • SQL injection in the login route (user input → f-string → raw SQL)
  • Broken access control (plaintext cookie for role check)
  • Debug mode enabled in production (Werkzeug console → potential RCE)
  • Plaintext password storage
  • GET parameters for credentials
  • Raw database rows exposed in responses

The tool found all six. More importantly, it didn't fabricate anything. Each finding had a specific line reference, a working exploit description, and a code-level fix.

When I pointed it at a clean codebase, it returned zero findings. No hallucination. That's the behavior you want in CI/CD — you can't have false positives blocking builds.

The Threat Model Surprised Me

I added threat modeling as an afterthought. You pass in a system description and it generates a STRIDE model. I didn't expect it to be that useful.

Then I tested it with: "Simple web app: React frontend, Express.js API backend, PostgreSQL database. JWT auth. AWS EC2 behind ALB."

It returned 23 threats across all six STRIDE categories, identified 6 trust boundaries, and prioritized everything by likelihood and impact. It caught things I'd expect a senior architect to flag — JWT algorithm confusion, ALB host header spoofing, EC2 metadata service exposure — all from a two-sentence description.

Is it a replacement for a proper threat modeling session with the engineering team? No. But as a starting point before that meeting? It's surprisingly good.

What I'd Do Differently

  1. Token costs — Reviewing a directory of 20 files can burn $2–5 in API credits depending on the model. For personal use that's fine. For CI/CD on every PR, you'd want to cache results, skip unchanged files, and use a cheaper model for the first pass.
  2. Language coverage — It detects 20+ languages by extension, but the review quality varies. The model understands Python, JavaScript, and Java really well. Rust and Solidity? Less so. Language-specific fine-tuning would help.
  3. False negatives — It misses things a SAST tool catches. Race conditions, crypto weaknesses, and subtle business logic flaws still need a human. This tool augments manual review — it doesn't replace it.
  4. Thinking mode — DeepSeek's API has a "thinking" mode that burns extra tokens before calling the tool. The output is better but slower. I added a higher token limit to accommodate it, but there's a tradeoff between depth and cost.

Try It Yourself

The repo is at github.com/NagiSheshero7/ai-sec-review. Installation is:

git clone https://github.com/NagiSheshero7/ai-sec-review.git
cd ai-sec-review
pip install -e .

Set your API key and run:

# Review a single file
ai-sec-review scan app.py

# Review a whole project
ai-sec-review scan ./src --output markdown --output-file report.md

# Generate a threat model
ai-sec-review threat-model "Describe your system here" --output json

Works with Anthropic, DeepSeek, or any Anthropic-compatible endpoint.

If you're doing AppSec work and spending hours on repetitive code review, give it a shot. It won't replace your judgment, but it'll save you from staring at yet another unsanitized f-string at 11pm.