AI Cybersecurity for Developers: A Practical Guide to Building Systems That Don’t Get You Breached

Somewhere in the last two years, "add an LLM to it" became the default answer to almost every product question. Fewer teams stopped to ask what that LLM can now do to the rest of the system. That gap is showing up in the numbers: according to IBM's 2026 Cost of a Data Breach Report, one in four malicious breaches last year was AI-enabled, a 56% jump over the previous year, and those breaches cost organizations about $6 million on average, roughly $1 million more than the overall average breach. More than one in five organizations reported a breach that specifically targeted their AI models or applications, most often through compromised APIs or misconfigured cloud infrastructure, not exotic model attacks.

That last detail matters. Most AI security failures in production aren't sophisticated adversarial-ML research problems. They're ordinary application security mistakes wearing a new costume: unvalidated input, over-privileged service accounts, secrets in the wrong place, and trust boundaries nobody drew on a diagram. This piece is a practical map of where those mistakes happen in AI systems specifically, and what to actually do about them, written for the engineer who has to ship something this quarter, not the researcher publishing the next jailbreak paper.

The threat model has two halves

AI changes security in two directions at once, and conflating them leads to bad prioritization.

First, AI systems are a new attack surface. An LLM-powered feature accepts natural language as input and often has tool access, database access, or the ability to take actions on a user's behalf. That's a much larger and fuzzier trust boundary than a traditional REST endpoint with a fixed schema.

Second, AI is a force multiplier for attackers targeting everything else. Phishing emails are more convincing, malware is easier to generate and obfuscate, and reconnaissance against your infrastructure can be automated at a scale that used to require a team. This guide focuses mostly on the first half, because it's the part developers directly control, but keep the second in mind when you're prioritizing detection and response, not just prevention.

Start from OWASP's Top 10 for LLM Applications

The OWASP GenAI Security Project maintains a Top 10 specifically for LLM applications, and it's the closest thing the industry has to a shared vocabulary for these risks. Here's the practical version of each one.

Prompt injection (LLM01). This is the big one, and it's structurally hard to fully solve because most LLM architectures don't cleanly separate "instructions" from "data" the way a SQL engine separates queries from parameters. An attacker who can get text in front of your model, through a user message, a retrieved document, a webpage your agent reads, an email your assistant summarizes, can potentially get the model to follow their instructions instead of yours. Direct injection is a user typing "ignore previous instructions." Indirect injection is worse and more common in production: a malicious instruction sitting inside a document, a customer support ticket, or a webpage that your AI agent later ingests as "data" and treats as an instruction.

You cannot fully prevent this with a better system prompt. Prompt engineering is a mitigation, not a control. Treat any content your model reads, not just what the user typed, as untrusted input, and design the surrounding system so that even a fully successful injection is contained.

Sensitive information disclosure (LLM02): Models memorize fragments of training and fine-tuning data, and they'll happily repeat back whatever is sitting in their context window, including your system prompt, internal tool schemas, other users' data pulled into a shared context, or API keys someone pasted into a prompt during debugging and forgot about. Never put secrets, credentials, or one user's private data into a context that another user's request might touch.

Supply chain risks (LLM03): You're not just trusting your own code anymore; you're trusting whichever pretrained model, fine-tuning dataset, embedding model, LoRA adapter, or third-party plugin you pulled in. A compromised or subtly poisoned model on a public hub is functionally similar to a compromised npm package, except the "code" is a few billion opaque weights instead of readable JavaScript, which makes it much harder to audit.

Data and model poisoning (LLM04): If you fine-tune on user-submitted data, feedback loops, or scraped content, an attacker can seed that data to bias the model or plant a backdoor that only activates on a specific trigger phrase. This is a real risk for any team doing continuous fine-tuning from production feedback without a review step in between.

Improper output handling (LLM05): The inverse of injection: if you take the model's output and pipe it into a shell, a database query, a browser render, or another tool call without validation, you've reintroduced classic injection vulnerabilities, XSS, SQLi, command injection, with an LLM as the new untrusted input source. Treat every token that comes out of the model with the same suspicion you'd apply to raw user input, because functionally, it often is.

Excessive agency (LLM06): This is the one that turns a contained prompt injection into a real incident. An agent with a send-email tool, a delete-file tool, and a run-SQL tool, all reachable from the same conversation that also processes untrusted text, is a chain waiting to be exploited. Scope what each agent can actually do to the minimum required for its task, and put a human or a hard-coded check in front of anything irreversible.

System prompt leakage (LLM07): If your system prompt contains actual security logic, like never reveal X, or only allow Y if role equals admin, it's not a control, it's a suggestion the model can be talked out of, and it's usually extractable anyway. Enforce real constraints in code, outside the model, and treat the system prompt as a behavioral nudge, not a security boundary.

Vector and embedding weaknesses (LLM08): RAG pipelines introduce their own attack surface: a poisoned document in your vector store can get retrieved and treated as trusted context, and access control on the retrieval layer is easy to forget, especially if you built a quick semantic search before you built proper row-level permissions. If different users or tenants shouldn't see each other's documents, that boundary has to be enforced at retrieval time, not hoped for at generation time.

Misinformation (LLM09): Confident, fluent, wrong. If your product surfaces model output as fact, especially in domains like health, finance, or legal, without a verification step or clear AI-generated framing, you're shipping a misinformation risk, not a security vulnerability in the traditional sense, but a real liability.

Unbounded consumption (LLM10): Without caps, a single crafted request, or a bot doing it repeatedly, can trigger enormous compute and API costs, sometimes called denial of wallet, or be used to slowly extract a proprietary model's behavior through systematic querying. Rate limit and cost-cap everything, per user and in aggregate.

Agentic AI and MCP: the newer, sharper edge

If your system uses agents that call tools, especially through the Model Context Protocol (MCP), which has become the de facto standard for connecting LLMs to external systems, there's a newer set of risks worth knowing specifically, drawn from MCP's own published security guidance and recent independent research on agentic infrastructure.

The confused deputy problem. An MCP proxy server that uses a single static client ID to talk to a third-party API, combined with per-user dynamic registration on the client side, can be tricked into handing an attacker an authorization code meant for a legitimate user. If you're building an MCP server that proxies to another API, implement per-client consent that runs before you ever touch the third-party auth flow, not after.

Token passthrough. Never accept a token from an MCP client and forward it unmodified to a downstream API without validating it was actually issued for you. If your server doesn't check the audience claim, it becomes a laundering point for tokens stolen elsewhere, and it breaks every downstream rate limit, audit log, and access control that assumed the token's origin.

SSRF through OAuth metadata discovery. A malicious MCP server can hand your client URLs pointing at internal IPs or cloud metadata endpoints during OAuth discovery. Block private and link-local IP ranges at the network layer for any client that fetches URLs supplied by a server you don't fully control.

Local server compromise. A locally installed MCP server runs with your full user privileges. A one-click install flow that doesn't show the exact command being executed, or that doesn't sandbox the process, is an arbitrary-code-execution vector with extra steps. If you're building a client that supports local servers, show the real command, flag dangerous patterns, and sandbox by default.

Scope inflation. It's tempting to request one broad scope up front to avoid a clunky consent flow. Don't. A stolen token with full admin access does far more damage than one scoped to the single read operation it was needed for. Request narrowly, elevate incrementally, and let users see exactly what they're approving.

The common thread across all of these: agentic systems collapse authentication, authorization, and business logic into the same conversational loop, and every seam between those three layers is a place where an attacker can jump from text the model read to action the system took.

A practical defense checklist

None of this requires solving alignment or inventing new cryptography. It requires applying security engineering you probably already know, aimed at the right places.

Separate instructions from data wherever you can, structurally, not just with prompt wording, for example by using distinct message roles, delimiters the model is fine-tuned to respect, and separate context windows for untrusted retrieved content versus your own system instructions.

Apply least privilege to every tool and API key an agent can touch. Scope credentials per-tool and per-task rather than issuing one omnibus service account key to your entire agent runtime.

Validate and sanitize model output before it touches a shell, a database, a browser, or another system, exactly as you would validate any other untrusted input.

Put a human, or at minimum a deterministic policy check that the model cannot talk its way around, in front of any irreversible or high-impact action: sending money, deleting data, sending an email to someone outside your organization, changing account settings.

Never place secrets, API keys, or one user's data into a context window shared with another user's request, and audit your prompt templates the same way you'd audit for SQL string concatenation.

Rate-limit and cost-cap at the user, session, and account level, and alert on anomalous usage the same way you'd alert on anomalous API traffic.

Treat pretrained models, fine-tuning datasets, and third-party plugins as supply chain dependencies. Pin versions, verify checksums or signatures where available, and know where each model artifact in your stack actually came from, the AI equivalent of maintaining a software bill of materials.

Log everything an agent does at the tool-call level, not just the final output, so an incident is investigable after the fact rather than a mystery.

Red-team your own system before someone else does. Adversarial testing for prompt injection and excessive agency is now a standard part of a pre-launch checklist, not an optional extra.

The frameworks worth actually knowing

You don't need to memorize all of these, but knowing they exist saves you from reinventing a taxonomy badly. OWASP's Top 10 for LLM Applications is the best starting vocabulary for the risks above. MITRE ATLAS catalogs real-world adversarial tactics and techniques against AI systems, structured the same way MITRE ATT&CK does for traditional infrastructure, and is useful when you're threat-modeling a specific deployment. NIST's AI Risk Management Framework is the more governance-oriented counterpart, useful if you need to justify your security posture to a compliance team or a customer's security questionnaire.

The uncomfortable bottom line

Prompt injection, in particular, doesn't have a clean fix the way SQL injection eventually got parameterized queries. The mitigation available today is architectural: assume it will sometimes succeed, and make sure that when it does, the blast radius is small. That's not a satisfying answer, but it's an honest one, and it's the same lesson application security has been teaching for twenty years, just with a more persuasive attacker sitting in the input field. Build for containment, not just prevention, and the rest of this list gets a lot less scary.

Sources: