Reviewed by the editorial team · Last reviewed 2026-11 · Report an error
LLM app threat model
Tick what your AI app or agent does. You get the threats that apply, mapped to the OWASP LLM Top 10, with the controls to put in place and the tool that tests each one.
This design has the lethal trifecta: access to private data, exposure to untrusted content, and a way to send data out. Break at least one leg for each task.
8 threats
critical
Indirect prompt injection from content it reads
LLM01
Hidden instructions inside pages, files, emails or documents get treated as commands.
Scan documents before ingestion
Mark retrieved content as untrusted data
Strip hidden text, comments and invisible Unicode
Limit what the AI can do after reading untrusted content
Confident wrong answers get acted on, especially in legal, financial or safety contexts.
Show sources for answers
Human review for high-impact outputs
Grounding checks for RAG
Threat model (Markdown)
# Threat model: Customer support assistant
Components: Users chat with it directly; Answers from internal documents (RAG); Reads files users upload; Shows output as Markdown or HTML
## [CRITICAL] Indirect prompt injection from content it reads (LLM01)
Hidden instructions inside pages, files, emails or documents get treated as commands.
Controls:
- [ ] Scan documents before ingestion
- [ ] Mark retrieved content as untrusted data
- [ ] Strip hidden text, comments and invisible Unicode
- [ ] Limit what the AI can do after reading untrusted content
## [CRITICAL] Lethal trifecta: private data + untrusted content + a way out (LLM01 / LLM02)
An injected instruction can make the AI read private data and send it out through an action, a link or an image.
Controls:
- [ ] Remove at least one of the three legs for any single task
- [ ] Allow-list destinations for links, images and outbound requests
- [ ] Require human approval for anything that sends data out
## [HIGH] Direct prompt injection and jailbreaks (LLM01)
Users can type instructions that override your system prompt or safety rules.
Controls:
- [ ] Never rely on the system prompt alone for security decisions
- [ ] Enforce permissions in code, outside the model
- [ ] Red-team before release and after every model change
## [HIGH] Zero-click exfiltration through rendered images and links (LLM05)
A Markdown image pointing at an attacker URL leaks data the moment the answer renders.
Controls:
- [ ] Render output with a strict Markdown allow-list
- [ ] Block images from domains you do not control (CSP img-src)
- [ ] Show the real URL behind link text
## [HIGH] Knowledge base poisoning (LLM04 / LLM08)
Anyone who can edit a source document can change what the AI tells everyone.
Controls:
- [ ] Restrict who can add content to indexed sources
- [ ] Keep provenance for every chunk
- [ ] Re-scan sources on change
## [MEDIUM] System prompt leakage (LLM07)
Assume users will extract your system prompt. Secrets or internal logic in it will leak.
Controls:
- [ ] No secrets, keys or internal URLs in prompts
- [ ] Keep business rules in code
## [MEDIUM] Denial of wallet and unbounded consumption (LLM10)
Long prompts, loops or abuse can run up model costs fast.
Controls:
- [ ] Per-user rate and spend limits
- [ ] Cap output length and agent steps
- [ ] Alert on cost spikes
## [MEDIUM] Misinformation and overreliance (LLM09)
Confident wrong answers get acted on, especially in legal, financial or safety contexts.
Controls:
- [ ] Show sources for answers
- [ ] Human review for high-impact outputs
- [ ] Grounding checks for RAG