OWASP Top 10 for LLM Applications — Revised Write-up

Introduction

Large Language Models (LLMs) are used in chatbots, customer-support systems, AI search, and AI agents. These applications can create security risks when they access private data, call APIs, use tools, or process content supplied by users.

The OWASP Top 10 for LLM Applications describes important security risks in applications that use LLMs. This write-up explains the categories in simple English and includes four practical PortSwigger labs.

Scope note:_ The category names below follow the OWASP Top 10 for LLM Applications 2025. OWASP may publish newer versions, so check the official page before using this as a current-version checklist._

Official reference: https://genai.owasp.org/llm-top-10/

AI-powered scanner vulnerabilities Application security teams often deploy AI-powered scanners that use Large Language Models (LLMs) to scan web…

2. The Main Risks

LLM01 — — Prompt Injection

An attacker writes instructions that manipulate the LLM into ignoring its intended task or performing an unsafe action.

• Example: A chatbot reads a product review containing hidden instructions and follows them.
• Impact: Unauthorized actions, data exposure, or misuse of connected tools.
• Mitigation: Treat external content as untrusted, limit tool permissions, and enforce security checks outside the model.

LLM02 — — Sensitive Information Disclosure

The application may reveal private data through its answers, connected data sources, logs, or training data.

• Example: A chatbot reveals another user's private information.
• Mitigation: Minimize sensitive data sent to the model, enforce access controls in the backend, and avoid using secrets in prompts.

LLM03 — — Supply Chain

Models, datasets, plugins, libraries, and integrations can introduce security or trust problems.

• Example: A downloaded model or third-party component contains unsafe behavior.
• Mitigation: Use trusted sources, verify versions and licenses, and review dependencies and model provenance.

LLM04 — — Data and Model Poisoning

An attacker manipulates training or fine-tuning data to influence the model's behavior.

• Example: Poisoned training examples cause the model to produce biased or unsafe results.
• Mitigation: Validate data sources, clean datasets, control training access, and test models before deployment.

LLM05 — — Improper Output Handling

The application uses LLM output without correctly validating, encoding, or sanitizing it.

• Example: Model-generated HTML is rendered in a browser and triggers cross-site scripting (XSS).
• Mitigation: Treat model output as untrusted input. Apply context-appropriate output encoding and safe rendering.

LLM06 — — Excessive Agency

An LLM or AI agent has more permissions or tools than it needs.

• Example: A chatbot can delete accounts or execute database commands without proper authorization.
• Mitigation: Apply least privilege, restrict available tools, validate every action server-side, and require confirmation for sensitive operations.

LLM07 — — System Prompt Leakage

An attacker may persuade the model to reveal its hidden system instructions.

• Example: The model exposes internal instructions or configuration details.
• Mitigation: Never store credentials or secrets in system prompts. Keep sensitive configuration on the server and enforce permissions independently.

LLM08 — — Vector and Embedding Weaknesses

Applications using vector databases may retrieve or expose information incorrectly because of weak isolation, access controls, or data handling.

• Example: A retrieval system returns documents belonging to another user.
• Mitigation: Enforce authorization before retrieval, isolate tenants, and test retrieval permissions.

LLM09 — — Misinformation

An LLM may produce false, misleading, or fabricated information that users trust.

• Example: An assistant gives an incorrect answer about a product or policy.
• Mitigation: Verify important outputs, show sources where possible, and use human review for high-impact decisions.

LLM10 — — Unbounded Consumption

An attacker or careless user can cause excessive model usage, cost, or resource consumption.

• Example: Repeated large prompts cause unexpected API costs.
• Mitigation: Set input/output limits, rate limits, budgets, timeouts, and usage monitoring.

3. Practical Testing Methodology

When testing an LLM-enabled web application, follow this workflow:

1. Map the inputs: Find chat prompts, product reviews, uploaded files, retrieved pages, and other content the model reads.
2. Map the tools: Identify APIs, functions, plugins, databases, and files the LLM can access.
3. Check permissions: Verify that the backend checks the current user's authorization for every action.
4. Test direct prompt injection: Try to influence the model through the chat interface.
5. Test indirect prompt injection: Check whether content from reviews, emails, or web pages changes the model's behavior.
6. Test connected APIs: Check whether the APIs have traditional web vulnerabilities, such as command injection or unsafe database queries.
7. Inspect output handling: Check whether model-generated HTML or other output is safely rendered.
8. Record evidence: Save relevant requests, responses, logs, and screenshots.
9. Retest fixes: Confirm that the issue is fixed without breaking normal functionality.

Only test systems you own or have explicit permission to assess. The steps below are for the designated PortSwigger Web Security Academy labs.

4. PortSwigger Lab Walkthroughs

Lab 1 — — Exploiting LLM APIs with Excessive Agency

Category: Related to excessive agency Difficulty: Apprentice Official lab: https://portswigger.net/web-security/llm-attacks/lab-exploiting-llm-apis-with-excessive-agency

Lab: Exploiting LLM APIs with excessive agency To solve the lab, use the LLM to delete the user carlos. Required knowledge To solve this lab, you'll need to know: How…

Goal

Use the lab's LLM integration to delete the test user carlos.

Steps

1. Open the lab and select Live chat.
2. Ask the LLM which APIs or functions it can access.
3. Identify the Debug SQL API and ask what arguments it accepts.
4. Ask it to run SELECT * FROM users and observe the returned user records.
5. Ask it to run DELETE FROM users WHERE username='carlos'.
6. Confirm that the lab reports as solved.

Why it works

The LLM can call an API that accepts raw SQL. The application gives the model too much authority, allowing a chat interaction to trigger a destructive database operation.

Mitigation

• Do not expose unrestricted SQL execution to an LLM.
• Enforce authorization and query restrictions in the backend.
• Require confirmation for destructive actions.
• Give the model only the minimum tools and permissions it needs.

Screenshots

Lab 2 — — Exploiting Vulnerabilities in LLM APIs

Category: LLM integration security; related to excessive agency and traditional command injection Difficulty: Practitioner Official lab: https://portswigger.net/web-security/llm-attacks/lab-exploiting-vulnerabilities-in-llm-apis

Lab: Exploiting vulnerabilities in LLM APIs This lab contains an OS command injection vulnerability that can be exploited via its APIs. You can call these APIs via…

Goal

Delete the morale.txt file from Carlos's home directory in the lab.

Steps

1. Open Live chat and ask which APIs the LLM can access.
2. Identify the Newsletter Subscription API and ask what argument it accepts.
3. Submit a normal email address that uses your lab's Exploit Server domain.
4. Open Email client and confirm that the subscription email arrives.
5. In the lab, test whether shell command substitution is interpreted by the vulnerable API using the documented $(whoami) probe in the email field.
6. Check the resulting email in the lab's Email client. If it shows the command output in the recipient address, this indicates command execution.
7. Use the lab's documented command-injection technique to remove /home/carlos/morale.txt.
8. Confirm that the lab is solved.

Important: Replace YOUR-EXPLOIT-SERVER-ID with the ID shown in your own lab. The live LLM may need the request rephrased.

Why it works

The API handles user-controlled input unsafely. The LLM provides a route to the API, but the underlying command-injection vulnerability is in the application integration.

Mitigation

• Never build shell commands by concatenating user-controlled values.
• Use safe APIs and argument arrays rather than shell interpretation.
• Validate email addresses on the server.
• Restrict the LLM's access to sensitive functions.

Screenshots

Lab 3 — — Indirect Prompt Injection

Category: Prompt injection Difficulty: Practitioner Official lab: https://portswigger.net/web-security/llm-attacks/lab-indirect-prompt-injection

Lab: Indirect prompt injection This lab is vulnerable to indirect prompt injection. The user carlos frequently uses the live chat to ask about the…

Goal

Use attacker-controlled product-review content to influence the LLM and cause it to delete the test user carlos.

Steps

1. Open Live chat and ask which APIs or functions the LLM can use.
2. Ask the LLM about a product other than the leather jacket, such as the umbrella.
3. Add a harmless test review that changes how the LLM describes the product. For example, say that the product is out of stock.
4. Ask the LLM about the product again and check whether it repeats the review's claim.
5. This confirms that product reviews influence the model's response.
6. Replace your test review with the lab's indirect-prompt-injection payload that instructs the LLM to call its account-deletion function.
7. Ask the LLM about the product again to trigger the vulnerable workflow.
8. Create a fresh account if needed, then place the corresponding payload on the leather-jacket product page.
9. Wait for Carlos's automated chat interaction and confirm that the lab is solved.

Use the exact payload and workflow from the official lab page if the live model behaves differently.

Why it works

The LLM treats untrusted product-review content as instructions rather than data. When it has access to account-management functions, this can lead to an unauthorized action.

Mitigation

• Separate trusted instructions from retrieved content.
• Treat reviews, documents, and web pages as untrusted data.
• Check authorization on every function call.
• Do not rely on a system prompt alone to prevent attacks.

Screenshots

• [Screenshot 1 — — Product review before testing]
• [Screenshot 2 — — LLM response influenced by the review]
• [Screenshot 3 — — Lab solved confirmation]

Lab 4 — — Exploiting Insecure Output Handling in LLMs

Category: Improper output handling Difficulty: Practitioner Official lab: https://portswigger.net/web-security/llm-attacks/lab-exploiting-insecure-output-handling-in-llms

Lab: Exploiting insecure output handling in LLMs This lab handles LLM output insecurely, leaving it vulnerable to XSS. The user carlos frequently uses the live chat to…

Goal

Use indirect prompt injection and XSS to delete Carlos's account in the lab.

Steps

1. Register an account using the email address associated with your lab instance. Confirm registration through the Email client and log in.
2. Open Live chat and test whether model output is rendered as HTML by submitting this harmless XSS probe:

<img src=1 onerror=alert(1)>

<img src=1 onerror=alert(1)>

3. Confirm whether an alert appears. This indicates that chat output may be rendered unsafely.

4. Open a product page other than the leather jacket and add the lab's XSS test payload as a review.

5. Ask the LLM about that product and observe how the review is included in its response.

6. If the model detects the bare payload, use the official lab's approach of embedding the payload inside an ordinary-looking review sentence.

7. Confirm that the payload is reflected through the LLM's response.

8. Place the corresponding payload on the leather-jacket product page and wait for Carlos's automated chat interaction.

9. Confirm that the lab is solved.

Use only the lab's documented account-deletion payload. Do not test this technique on real users or websites.

Why it works

The application trusts and renders model output without appropriate sanitization. An attacker-controlled review influences the LLM, which then returns content that executes in a user's browser.

Mitigation

• Apply context-aware output encoding.
• Avoid rendering model output as raw HTML.
• Sanitize untrusted content with a well-maintained HTML sanitizer.
• Use browser security controls such as Content Security Policy as an additional layer.

Screenshots

• [Screenshot 1 — — Harmless XSS probe]
• [Screenshot 2 — — Review influencing the LLM response]
• [Screenshot 3 — — Lab solved confirmation]

6. General Security Recommendations

1. Least privilege: Give the LLM only the tools and permissions required for its task.
2. Server-side authorization: Check user permissions for every action; never trust the model to enforce access control.
3. Safe output handling: Treat model output as untrusted data.
4. Data protection: Do not provide the model with secrets or data the current user is not allowed to access.
5. Human approval: Require explicit approval for sensitive or destructive actions.
6. Monitoring and limits: Log tool calls, set usage limits, and alert on unusual activity.
7. Testing: Re-test the integration whenever prompts, models, tools, or permissions change.

7. Conclusion

LLM security is not only about making a model refuse harmful prompts. The application around the model matters just as much: APIs, permissions, retrieved content, output rendering, and data access can all create vulnerabilities.

The most important lesson from these labs is:

Never trust the LLM to be the security boundary. Enforce security in the application and its backend.

References

• OWASP Top 10 for LLM Applications: https://genai.owasp.org/llm-top-10/
• PortSwigger Web LLM attacks: https://portswigger.net/web-security/llm-attacks
• PortSwigger Web LLM attacks learning path: https://portswigger.net/web-security/learning-paths/llm-attacks
• PortSwigger AI-powered scanner vulnerabilities: https://portswigger.net/web-security/llm-attacks/ai-powered-scanner-vulnerabilities