Prompt Injection: What It Is and How to Defend Against It

OWASP describes two forms (OWASP LLM01):

• Direct injection comes from the user's own prompt. It may be deliberate, from a malicious user, or accidental, from a user who pastes in text that happens to contain instructions.
• Indirect injection arrives through external sources the model reads, such as websites, files, emails or search results.

Indirect injection is the bigger risk for most products because the attacker never touches your interface. The Alan Turing Institute's Centre for Emerging Technology and Security describes it as hiding instructions "in the data it accesses, such as incoming emails or saved documents," and notes that it requires no direct access to the AI system (CETaS). The same report points out that an assistant does not read data the way a person does, so instructions can be invisible on screen and still sit in the model's context.

The OWASP cheat sheet lists where indirect payloads hide: code comments and documentation read by coding assistants, commit messages and merge request descriptions, issue descriptions, user reviews, document metadata, images processed by multimodal models, and documents planted in a retrieval-augmented generation (RAG) index (OWASP Cheat Sheet). OWASP's own scenarios include a user asking a model to summarize a web page that contains hidden instructions, causing the model to insert an image whose URL carries the private conversation to the attacker (OWASP LLM01).

The consequences depend on what the application can do. For a chatbot with no tools, the worst case is usually a leaked system prompt or a misleading answer. For an agent that can read mail, query a database and call APIs, the worst case is data theft or unauthorized actions taken in the user's name.

The natural first response is to filter bad input or tell the model to ignore instructions in data. Both help a little, and neither is a boundary.

OWASP states plainly that "it is unclear if there are fool-proof methods of prevention for prompt injection," given how models work (OWASP LLM01). Its cheat sheet catalogs the ways attackers get past keyword filters: Base64 and hex encoding, invisible text rendered through LaTeX, scrambled spellings that a model still reads correctly, and simply generating many variations until one slips through (OWASP Cheat Sheet). The cheat sheet also warns, about its own example of separating instructions from data with labels, that text labels and prompt wording should not be treated "as an enforcement boundary," and that permissions must be enforced at the tool.

Willison is sharper about commercial guardrail products that claim to catch most attacks: "in web application security 95% is very much a failing grade" (Simon Willison). An attacker only needs one phrasing that works, and the space of phrasings is unlimited.

The Frontier Model Forum's brief on agent security draws the practical conclusion. Because prompt injection research is still maturing, "robust agent security should not rely on detection alone." Deterministic controls outside the agent's reasoning loop, such as limits on tool access and permitted actions that context manipulation cannot override, remain the dependable layer (Frontier Model Forum).

Willison offers a simple test for when injection turns into data theft. An agent is exposed when it combines three capabilities (Simon Willison):

1. Access to private data, which is often the reason the tools exist.
2. Exposure to untrusted content, meaning any path by which attacker-controlled text or images reach the model.
3. The ability to communicate externally, which can be as small as making an HTTP request, loading an image or showing a link the user might click.

With all three, an attacker can plant instructions in content the agent reads, have it collect private data, and send that data out. Willison's example is an email assistant: anyone can email it, so anyone can give it instructions.

The test is useful in design reviews because it turns a vague risk into a question with a yes or no answer for each tool. Willison notes that connecting tools from different sources, as tool protocols such as MCP encourage, makes it easy to assemble all three without noticing. A single tool can even combine them, as in Willison's example of a code-hosting integration that could read public issues written by anyone, read private repositories, and open pull requests.

The fix is to break the triangle for any session that touches untrusted content. Remove external communication, or remove access to private data, or keep untrusted content out.

A group of researchers published a set of design patterns built on one principle: "once an LLM agent has ingested untrusted input, it must be constrained so that it is impossible for that input to trigger any consequential actions" (Design Patterns paper). Each pattern trades some flexibility for a guarantee.

The patterns share an idea that applies even if you adopt none of them by name. Decide what the agent may do before it reads anything an attacker could have written, and make sure what it reads afterwards can only fill in narrow, checkable values. A support agent that reads a customer email and may only choose among "refund under a set amount", "escalate" and "reply with a template" is far safer than one that may run any tool with any argument.

Architecture does most of the work, and layered controls catch what remains. Microsoft's guidance on indirect prompt injection recommends a defense-in-depth stack that mixes probabilistic and deterministic measures (Microsoft Learn):

• Input screening to detect obvious injection attempts.
• Spotlighting, which marks external content so the model can distinguish it from instructions.
• Plan drift detection, which watches multi-step tasks for steps that do not fit the request.
• Tool chain analysis, which blocks risky sequences such as reading a secret and then making an outbound request.
• Least privilege and short-lived privileges, granting tools only when needed and removing them after use.
• Human in the loop, which Microsoft calls "the last line of defense," for risky actions.

OWASP's mitigation list for LLM01 points the same way. Give the application its own API tokens and handle privileged functions in code instead of handing them to the model, restrict the model to the minimum access it needs, require human approval for privileged operations, mark untrusted content clearly, and validate output formats with deterministic code (OWASP LLM01).

Two output controls are easy to miss and close common exfiltration paths:

• Do not render images or links from model output automatically. OWASP's cheat sheet lists hidden image tags as a data exfiltration technique (OWASP Cheat Sheet). Allow only known domains, or show URLs as plain text.
• Validate every tool call against an allowlist of arguments in code, not in the prompt. A tool that sends email should check the recipient against the user's contacts or the current thread.

CETaS adds the organizational side: good data hygiene over what the assistant can read, evaluation before deployment, user training, and continuous monitoring for suspicious behavior (CETaS).

OWASP recommends regular penetration testing and breach simulations that treat the model as an untrusted user, to test whether trust boundaries and access controls hold (OWASP LLM01). A small team can start with a short routine:

1. List every input path the model reads: user messages, uploaded files, retrieved documents, tool results, web pages, emails and memory.
2. Plant a canary instruction in each path, for example a hidden line that asks the model to include a unique word or call a harmless tool. If the word appears, that path can steer the model.
3. Try exfiltration through each output channel: images, links, tool arguments, emails and file writes.
4. Check the trifecta for every tool combination a single session can reach.
5. Repeat on every change to the system prompt, tool list or model. Behavior changes across versions, and a test that passed last month proves little today.

Record what each test showed and which control stopped it. When the answer is "the model refused," treat that as luck, and look for a deterministic control that would have stopped it anyway.