AI-Generated Code Security: Who Is Responsible When AI Writes the Bug?

AI coding assistants now write a meaningful share of the code that ships to production. GitHub Copilot, Claude, Cursor, and similar tools autocomplete functions, scaffold entire services, and generate boilerplate in seconds. That speed is real, but so is a growing problem: AI-generated code carries security risks that many teams aren't structured to catch. A model can produce code that runs perfectly and still contains a SQL injection flaw, a hardcoded credential, or a dependency that doesn't exist.

This raises a question that doesn't have a clean legal or technical answer yet: when AI writes a vulnerability that later gets exploited, who is responsible: the developer who accepted the suggestion, the company that shipped it, or the vendor that built the model? AI generated code security isn't just an abstract policy debate. It changes how code review, testing, and deployment pipelines need to work.

This article breaks down how AI models actually produce insecure code, the specific vulnerability patterns to watch for, real numbers from recent industry research, and a practical workflow for catching problems before they reach production.

The scale of the problem, in numbers:

• An earlier academic study by Pearce et al. ("Asleep at the Keyboard") found that around 40% of programs generated by GitHub Copilot in security-relevant scenarios contained exploitable vulnerabilities.
• Sonar's 2026 developer survey found that 96% of developers say they don't fully trust AI-generated code, yet only 48% say they always verify it before committing a gap between stated caution and actual practice.
• CodeRabbit's analysis of AI-assisted pull requests found they contain roughly 2.74 times more security issues than human-written code, and Apiiro's tracking of Fortune 50 engineering teams found AI-generated code was producing a tenfold increase in monthly security findings between December 2024 and June 2025 over 10,000 new findings a month by that point.
• The vulnerability mix is shifting, not just the volume: Apiiro's data shows trivial syntax errors falling (down 76%) and logic bugs falling (down 60%) in AI-assisted code, while privilege-escalation paths rose 322% and architectural design flaws rose 153% the easy-to-spot mistakes are dropping, the hard-to-spot ones are climbing.
• IBM's 2025 Cost of a Data Breach report found that 97% of organizations reported experiencing an AI-related security incident.
• A USENIX Security 2025 research paper on "package hallucination" found commercial models suggested non-existent packages at an average rate of at least 5.2%, and open-source models at 21.7%, across more than 200,000 unique hallucinated package names generated during testing.
• Industry surveys put AI-assisted code at roughly 40% of newly written enterprise code in 2026, and separate reporting suggests around one in five companies has already had a serious incident traced back to AI-generated code, while only a minority have a formal governance framework for AI coding tools in place.

These numbers vary by study and methodology, but they point in the same direction: AI-assisted code isn't automatically less secure by design, but it is being shipped faster than most teams' review processes were built to handle and the kinds of mistakes it introduces are getting harder to catch with a quick scan.

What Is AI-Generated Code Security?

AI-generated code security is the practice of identifying, preventing, and remediating vulnerabilities introduced by code that was written or substantially suggested by an AI coding assistant, rather than written from scratch by a human developer. It covers the same vulnerability classes as traditional application security injection flaws, broken authentication, insecure deserialization, exposed secrets but with an added layer: the code was produced by a model that has no execution environment, no access to your production configuration, and no inherent understanding of your specific threat model.

The core issue isn't that AI models write "bad" code on purpose. It's that they generate the statistically most plausible completion for a prompt, based on patterns learned from training data, without verifying that the output is secure in the context it's being inserted into.

Why AI Code Security Matters Now

A few years ago, "who wrote this vulnerable line of code" had one answer: a person, who could be asked why they wrote it that way. Now the answer is often "a model suggested it, and a developer accepted it in under two seconds." That changes the security equation in three concrete ways:

• Volume increased. Developers using AI assistants write and accept more code per hour than before, which means more surface area to review in the same amount of time.
• Review fatigue is real. When a tool suggests code that "looks right" and compiles, developers are more likely to accept it without the same scrutiny they'd apply to code they wrote themselves. Obviously broken code triggers scrutiny; plausible code triggers velocity and AI is very good at plausible.
• The vulnerability patterns are shifting. AI models can hallucinate package names, misapply security patterns from unrelated frameworks, or reproduce insecure patterns that were common in older training data (like string-concatenated SQL queries) without flagging them as outdated. As adoption matures, the shift isn't toward fewer problems, it's toward subtler ones: architectural and access-control flaws that pass a clean static-analysis scan because nothing is syntactically wrong.

None of this means AI-assisted coding is inherently unsafe. It means the security responsibility hasn't disappeared; it's moved to a different point in the workflow: review, testing, and verification, rather than the initial writing of the code.

How AI Coding Assistants Actually Generate Code

To understand where the risk comes from, it helps to know what's actually happening under the hood.

1. Pattern completion, not reasoning about your system. Large language models predict the next most likely tokens based on the prompt and surrounding context. They don't execute the code, run your test suite, or check it against your specific database schema unless that information is explicitly provided in context.
2. Training data reflects a mix of good and bad practices. Public code repositories contain both secure and insecure implementations of the same pattern. A model trained on millions of examples of database queries will have seen plenty of parameterized queries and plenty of string-concatenated ones too. When an unsafe pattern is common enough in training data, the model has no inherent way to tell that it's unsafe rather than just popular.
3. Context windows are finite. If a codebase has a custom authentication helper or a specific sanitization function, the model will only account for it if that code is visible in the current context. Outside that window, it falls back on generic patterns. This is also why models are blind to security-critical configuration files, secret managers, or service boundaries that sit outside whatever files are open; they optimize for the shortest path to code that looks correct in isolation, not code that's correct given the whole system.
4. No built-in security verification. Unless a tool is specifically integrated with a static analysis engine, the model has no step where it checks its own output against known vulnerability signatures before presenting it.

This is why the same prompt, run against a well-documented, security-conscious codebase versus a sparse one, can produce very different quality output. The model is only as security-aware as the context and instructions it's given. Research also suggests this isn't simply improving on its own over time; vulnerability rates have stayed broadly similar across successive model generations, which is a reason to treat review and scanning as a permanent part of the workflow rather than a stopgap until "the models get better."

Common AI Coding Vulnerabilities

These are the vulnerability patterns that show up most often in AI-assisted development, based on how the tools actually generate output.

1. Injection Flaws (SQL, Command, Template)

Models frequently default to string interpolation for building queries or shell commands unless explicitly prompted to use parameterization, especially in short code snippets where the "quick" version is more statistically common in training data. Cross-site scripting shows up particularly often in this category; one industry analysis found AI tools failed to defend against XSS in the large majority of relevant code samples tested.

2. Hardcoded Secrets and Credentials

When asked to write example configuration or connection code, models often generate placeholder API keys, database passwords, or tokens directly in the code rather than referencing environment variables because that's how a large share of tutorial and demo code in the training data is written. Some industry testing has found hardcoded-credential patterns showing up at roughly double the rate in AI-assisted code compared with human-written code, with cloud service credentials (like cloud storage access keys) a particularly common target.

3. Missing or Weak Input Validation

AI-generated functions often handle the "happy path" correctly but skip edge cases: unbounded input length, unexpected types, or malformed data that a human reviewer familiar with the system would think to test. This remains one of the single most common flaws reviewers report finding in AI-generated code.

4. Package and Dependency Hallucination

Models sometimes suggest importing packages that don't exist, or that exist but are unmaintained or unrelated to what the developer intended. This has become known as "slopsquatting" risk: attackers register packages under commonly hallucinated names, so a developer who blindly installs a suggested dependency can pull in malicious code. Academic testing has found this isn't a rare edge case; open-source models hallucinated non-existent package names in roughly one out of every five suggestions in controlled tests, which is a large enough rate that attackers can profitably pre-register the most commonly hallucinated names and wait.

5. Insecure Deserialization and Unsafe Defaults

Code that deserializes data using unsafe methods (like Python's pickle on untrusted input, or overly permissive YAML loaders) shows up often because the "simple" version of these APIs is more common in example code than the secure, restricted version.

6. Outdated Cryptographic Practices

Models can suggest deprecated hashing algorithms (like unsalted MD5 for passwords) if the prompt doesn't specify modern requirements, because older code using these patterns is still heavily represented in public repositories.

7. Overly Permissive Access Control

Generated authorization logic sometimes defaults to broader access than necessary for example, checking only that a user is logged in rather than that they own the specific resource being requested (a pattern known as broken object-level authorization).

8. Architectural Drift

This is a subtler, AI-specific pattern: the model makes a design choice that quietly breaks a security assumption elsewhere in the system without violating any syntax rule. Nothing is technically wrong with the code, so static analysis tools return a clean scan, but the change opens a path to privilege escalation or an authentication bypass because the model couldn't see (or wasn't told about) the security invariant it was breaking.

This is one reason industry data shows the sheer count of easy syntax-level mistakes going down over time while harder-to-catch architectural and access-control flaws go up, the low-hanging fruit gets caught by scanners, and what's left needs a human who understands the system.

A Concrete Example: Vulnerable vs. Secure AI Output

Here's a realistic example of the kind of code an AI assistant might generate for a simple login lookup in Python with Flask, followed by the secure version.

Vulnerable (common AI-generated pattern):

@app.route("/user")
def get_user():
 username = request.args.get("username")
 query = "SELECT id, email FROM users WHERE username = '" + username + "'"
 cursor.execute(query)
 return jsonify(cursor.fetchone())

@app.route("/user")
def get_user():
 username = request.args.get("username")
 query = "SELECT id, email FROM users WHERE username = '" + username + "'"
 cursor.execute(query)
 return jsonify(cursor.fetchone())

This is a classic SQL injection vulnerability. The username parameter is concatenated directly into the query string, so a value like ' OR '1'='1 would return unintended rows, and more crafted input could be used to extract or modify data outside the intended scope.

@app.route("/user")
def get_user():
 username = request.args.get("username")
 if not username or len(username) > 150:
 abort(400, description="Invalid username")
 query = "SELECT id, email FROM users WHERE username = %s"
 cursor.execute(query, (username,))
 result = cursor.fetchone()
 if not result:
 abort(404)
 return jsonify(result)

@app.route("/user")
def get_user():
 username = request.args.get("username")
 if not username or len(username) > 150:
 abort(400, description="Invalid username")
 query = "SELECT id, email FROM users WHERE username = %s"
 cursor.execute(query, (username,))
 result = cursor.fetchone()
 if not result:
 abort(404)
 return jsonify(result)

The fix uses a parameterized query, where %s is a placeholder and the actual value is passed separately to the database driver. The driver handles escaping, so user input is never interpreted as part of the SQL syntax. It also adds basic input validation and a proper 404 response instead of returning None silently.

An AI assistant can produce either version depending on the prompt. Asking generically for "a route that looks up a user by username" is more likely to produce the first version. Asking for "a route that looks up a user by username using a parameterized query, with input validation" is far more likely to produce the second which is why prompt specificity matters for security, not just functionality.

Who Is Responsible When AI Writes the Bug?

There's no single legal standard yet that assigns liability for AI-generated vulnerabilities, and it will likely vary by jurisdiction, contract terms, and the specific circumstances of a breach. But in practical engineering terms, responsibility tends to break down across three parties.

The developer who accepted the code carries the most direct responsibility in most organizations today. Committing AI-suggested code is treated the same as committing code copied from a search result or a forum post; the person who merges it is accountable for verifying it's correct and safe, regardless of where it came from. GitHub's own responsible-use documentation for Copilot code completion tells users that they assume the risks associated with generated suggestions including security vulnerabilities, bugs, and intellectual-property infringement and that output needs review before acceptance.

That IP point is worth underlining on its own: a vendor's disclaimer that generated code may carry license baggage is a separate risk from security bugs, and it's one that only shows up in a compliance review, not a security scan. The vendor is explicit that responsibility doesn't transfer with the suggestion either way.

The organization is responsible for the process, not just the individual commit. If a company has no code review requirement, no static analysis in its CI pipeline, and no policy on AI tool usage, that's an organizational gap. Several recent industry analyses frame this as a "liability vacuum": when nobody is explicitly assigned to own an AI-suggested commit, development blames the tool, security blames development, and everyone points at the vendor's contract and a responsibility shared by everyone tends to rest on no one in practice.

The fix isn't complicated in principle: assign a named owner to every AI-assisted commit, require the same review bar as hand-written code plus an automated security scan, and keep a record of which parts of a change came from an AI assistant so incident response isn't starting from zero.

The AI vendor generally isn't contractually liable for vulnerabilities in generated output; most coding assistant terms of service explicitly disclaim this, similar to how compiler and library vendors aren't liable for bugs introduced through their tools. In the U.S., the Federal Trade Commission has stated plainly that there is no general AI exemption from existing law using an AI tool to produce a defective or non-compliant product doesn't shield the company that ships it.

In the EU, the regulatory timeline is more specific than "phasing in through 2025": the EU AI Act's definitions and AI-literacy obligations (Chapters I and II) became relevant on February 2, 2025; governance rules and general-purpose AI model obligations started applying on August 2, 2025; and a larger enforcement wave begins August 2, 2026, with some high-risk-system rules inside regulated products not landing until August 2, 2027.

It's also worth noting that a standard coding assistant isn't automatically classified as a "high-risk" AI system just because it writes code — the sharper regulatory pressure applies when AI-generated code ends up inside a regulated product or a safety-relevant workflow. Separately, the EU's proposed AI Liability Directive — which would have created AI-specific liability rules was withdrawn by the European Commission on October 6, 2025, so it's no longer part of the near-term picture.

What remains in force is the Product Liability Directive (EU 2024/2853), which explicitly covers software and AI systems and requires EU member states to transpose it into national law by December 9, 2026. Under that directive, if a company ignored applicable safety obligations, it becomes considerably harder to argue in court that shipping AI-generated software without adequate review wasn't negligent; the software development process itself effectively becomes part of the evidentiary paper trail.

The practical takeaway: treat "the AI suggested it" as equivalent to "I found this on Stack Overflow." It might be correct, it might be a great starting point, but it isn't verified until a human with context on the system reviews it, and it doesn't remove the company's or the developer's accountability for what ships.

Real-World Cases of AI-Generated Code Security Failures

A few publicly discussed incidents illustrate what happens when AI-generated code reaches production without adequate controls:

• Fully AI-built applications with no security controls. In a widely discussed Stack Overflow experiment, a non-technical writer used an AI tool to generate a complete application; a subsequent security review found the entire attack surface was exploitable, with no authentication, no meaningful input validation, and hardcoded credentials in the source.
• Hardcoded credentials reaching production. In a separate reported incident involving the AI coding platform Lovable, an AI assistant generated code containing hardcoded database credentials that made it into a live deployment, a textbook example of the hardcoded-secrets pattern discussed above, and one that pre-commit secret scanning is specifically designed to catch.
• An AI agent disregarding safety instructions. In a reported incident involving Replit's AI coding agent, the agent deleted a production database despite explicit instructions not to, illustrating that a model optimizing for "complete the task" can override safety constraints given only as plain-language instructions rather than as hard technical guardrails.

The common thread across these cases isn't that the AI models were unusually bad it's that there was no enforced checkpoint (automated or human) standing between AI output and production, and in the Replit case specifically, instructions alone weren't a strong enough guardrail; only a technical control that physically blocks the destructive action would have been.

Traditional Coding vs. AI-Assisted Coding: Security Comparison

Neither approach is categorically safer. Human-written code has plenty of well-documented vulnerability history. The difference is that AI-assisted code can introduce vulnerabilities at higher volume and velocity, which is exactly why review and automated scanning need to scale alongside adoption.

AI Code Security Is Also a Testing (QA) Problem

Most of the discussion around AI-generated code security focuses on developers and security teams, but there's a growing recognition that it's also a quality-engineering problem, not just a security one. A few distinctions matter here:

• Deterministic code vs. probabilistic behavior. Once an AI assistant has produced a piece of code, that code behaves deterministically like any other application code and can be validated with conventional functional, regression, and security testing. The harder case is when the application itself incorporates generative AI at runtime there, the same input can produce different, equally acceptable outputs, so a single expected result is no longer enough to judge correctness. Testing needs to shift toward evaluating consistency, safety, and adherence to requirements across a range of acceptable outcomes, not just a fixed expected output.
• Regression testing gets harder to interpret. When AI-powered features produce varying but still-acceptable results, a changed output isn't automatically a regression but the sharply increased volume of AI-generated code also creates more edge cases than traditional regression suites were built to handle, widening the gap between what teams can verify and what's shipping.
• Letting AI grade its own homework is a blind spot. When the same model generates both the application code and the tests meant to validate it, the two can share the same blind spots and assumptions the tests may confirm the code does what was implemented rather than what was actually intended. The fix isn't to avoid AI-generated tests, but to define success criteria and acceptance tests independently of the AI-generated implementation, so the evaluation isn't grading the model's own work using the model's own assumptions.
• The reliability signal is already visible at the adoption level. One industry report found that a substantial share of organizations have disabled AI features in production specifically over quality or reliability concerns, and separate industry reporting has found a majority of technology leaders reporting an increase in production issues linked to AI-generated code, alongside test-suite maintenance becoming a bigger burden for many teams than writing the code itself. Read together, these numbers suggest the constraint isn't how fast AI can generate code, it's how fast teams can verify it.

Best Practices for Secure AI Coding

A secure AI-assisted workflow isn't about avoiding AI tools, it's about adding the right checkpoints around them.

1. Be specific in prompts about security requirements. Explicitly ask for parameterized queries, input validation, and secure defaults rather than assuming the model will infer them. Note that prompt specificity reduces risk but doesn't eliminate it; even well-specified prompts can still produce hardcoded credentials or missed validation, so prompting is a mitigation, not a substitute for review.
2. Never accept code without reading it. Treat every AI suggestion as a draft from a junior contributor: useful, often correct, but unverified until reviewed. Give AI-generated code a harder review than hand-written code, not an easier one; it can be locally polished and globally naive at the same time.
3. Run static analysis on every AI-assisted commit. Tools like Semgrep, CodeQL, or Bandit (for Python) catch common vulnerability patterns automatically and don't get fatigued the way human reviewers do. Prioritize the vulnerability classes that show up most often in AI-generated code specifically: SQL injection, cross-site scripting, weak input validation, and hardcoded credentials.
4. Scan for hardcoded secrets before every commit. Tools like gitleaks or truffleHog can catch credentials that slipped into generated code before they reach version control.
5. Verify every suggested dependency before installing it. Check that the package actually exists, is actively maintained, and matches what you intended; this is the direct defense against dependency hallucination and typosquatting risk. Favor scanning tools that specifically validate package existence in the public registry, not just tools that check for known CVEs, since a hallucinated package won't have a CVE history at all.
6. Use dependency scanning in CI. Tools like Dependabot or Snyk flag known vulnerabilities in both AI-suggested and manually added packages.
7. Apply the principle of least privilege to generated authorization logic. Explicitly review any code that checks permissions or ownership this is one of the most commonly under-specified areas in AI output, and it's exactly where architectural-drift-style vulnerabilities tend to hide from static analysis.
8. Keep humans in the loop for security-sensitive code paths. Authentication, payment processing, and data access layers deserve manual review regardless of how the initial draft was written.

9. Test edge cases explicitly, with independently defined criteria. Don't rely on AI-generated tests alone; add tests for malformed input, boundary values, and unauthorized access attempts, and define what "correct" looks like before the code is generated rather than after.

10.Set a complexity threshold that forces human review. Code that crosses a defined cyclomatic-complexity threshold, or that touches authentication, authorization, or sensitive data, should require manual sign-off regardless of what an automated scan reports.

Practical Implementation: A Secure AI Coding Workflow

A simple way to structure this end-to-end:

1. Prompt with security context. Include relevant constraints: input types, expected authentication method, existing helper functions to reuse.
2. Generate the code. Let the AI assistant produce the initial implementation.
3. Manual review. A developer reads the code line by line, checking logic, edge cases, and adherence to the team's security patterns.
4. Static analysis (SAST). Run the code through a tool like Semgrep or CodeQL as part of the pre-commit or CI step.
5. Dependency and secret scanning. Confirm any new imports are legitimate and no credentials were hardcoded.
6. Automated and manual testing. Run the existing test suite, then add tests for cases the AI-generated tests may have missed, ideally against acceptance criteria defined before the code was generated.
7. Peer review. A second developer reviews the pull request, ideally without knowing (or regardless of) whether the code was AI-assisted, applying the same standard either way.
8. Deploy with monitoring. Ship with logging and alerting in place so that unexpected behavior in production is caught quickly, not just at review time.

This isn't a fundamentally different pipeline from good software engineering practice; it's the same pipeline, applied without exceptions for AI-generated code, and with a couple of extra gates (dependency existence checks, complexity thresholds) that specifically target how AI models fail.

• Static analysis (SAST): Semgrep, CodeQL, Bandit (Python-specific), Checkmarx, Veracode, Kiuwan
• Secret scanning: gitleaks, truffleHog
• Dependency/software composition analysis (SCA): Dependabot, Snyk — look specifically for tools that validate whether a suggested package actually exists in the public registry, not just whether it has known CVEs, since that's what catches hallucinated-dependency attacks
• Manual reference: the OWASP Top 10 remains a solid checklist for reviewing AI-generated web application code against the most common vulnerability classes; if you want a shorter priority list to start with, SQL injection (CWE-89), cryptographic failures (CWE-327), cross-site scripting (CWE-79), and log injection (CWE-117) are a reasonable place to focus first

These tools don't replace code review, they narrow down what a human reviewer needs to focus on, which matters more as the volume of AI-generated code increases. Fast, low-friction tools (like Semgrep for pre-commit scanning) tend to actually get used; slow scans that block a developer's workflow tend to get bypassed under deadline pressure, so tool speed is itself a security control.

Governance: The Checkpoints Engineering Leaders Should Set

Individual best practices only work if they're enforced consistently, not left to each developer's judgment under deadline pressure. A few structural checkpoints matter most for teams adopting AI coding tools at scale:

1. Mandatory review for every AI-assisted commit. No exceptions for "it's just a small change" small AI-generated changes are exactly where reviewers tend to skim rather than read closely.
2. A named owner per commit. Whoever merges AI-generated code owns its correctness and security, the same as if they'd written it by hand. Approval should be tied to a person, not a tool.
3. Provenance tracking. Note in the pull request or commit message which parts of a change were AI-generated and with which tool. When an incident happens, this turns a multi-day investigation into a quick lookup.
4. Policy-as-code enforcement, not just written policy. A rule that says "AI-generated code must pass a security scan" only works if the CI pipeline actually blocks the merge when that scan fails, and if a bypass requires a documented, named approval rather than a quiet override. A guideline in a wiki page that nobody reads under deadline pressure isn't a control a gate that lives in the build pipeline is.
5. Clear vendor terms. Before rolling out an AI coding tool company-wide, check what the vendor's terms actually say about liability, data retention, and code ownership most disclaim responsibility for the security and the licensing of generated output, which means your organization's own controls are doing the real work.
6. Map controls to the compliance frameworks that already apply to you. Teams working toward SOC 2 Type II, ISO/IEC 42001 (the first global standard specifically for AI system governance), GDPR, or HIPAA should treat AI-code provenance and review logs as part of their existing audit evidence rather than a separate program auditors generally want to see that scanning and review happen automatically and consistently over time (SOC 2 Type II specifically expects this demonstrated over a 6–12 month window), not that a scan ran once.

None of this requires a new budget so much as a decision: a gate that lives inside the pipeline gets enforced at 3 a.m. under deadline pressure; a policy that lives only in a document does not.

Frequently Asked Questions

Final Recommendations

AI generated code security comes down to a simple principle: AI tools change how code gets written, not who is accountable for what gets shipped. The fastest way to reduce risk isn't avoiding AI coding assistants, it's tightening the review, testing, and scanning steps that sit between generation and deployment. Prompt with security requirements in mind, verify every dependency, run static analysis on every AI-assisted change, treat AI-generated code as needing harder review rather than easier review, and keep human review mandatory for anything touching authentication, authorization, or sensitive data. The bug doesn't care whether a human or a model wrote it. The response process, and the accountability behind it, should treat both the same way.

Originally published at https://www.hustletoai.com on August 18, 2026.