Prompt Injection Defenses: A Defense-in-Depth Playbook

Prompt injection is not a prompt problem — it is a trust boundary problem. The attack works because a model treats instructions embedded in retrieved documents, emails, or web pages with the same authority as your system prompt. No amount of 'ignore previous instructions' prose will close that gap; the fix is architectural.

The playbook below layers five controls so that a single injection — even a successful one — cannot turn into a damaging action. Defense in depth means an attacker must defeat every layer, not just one.

Model every input as untrusted

Start by classifying each input the model sees: system prompt (you), user message (authenticated user), and context data (documents, web pages, emails — everything else). Treat context data like a hostile file upload. It can influence the model's text, but it must never directly influence tool calls, permissions, or output that gets executed.

Least privilege on every tool

• Give each tool the narrowest scope it needs: read-only database credentials, a dedicated API key with restricted roles, never the app's main token.
• Separate high-risk tools (send email, write files, transfer money) from low-risk ones (lookup, summarize, search) behind explicit per-tool authorization.
• Require confirmation for dangerous actions triggered from context-derived arguments — or reject them outright unless the human user typed them.

Sandbox where the model can act

Even with scoped credentials, run model-driven actions in an isolated environment. A one-off container for generated code, a restricted service account for file writes, and a separate network zone for any tool the model can call keep a compromise contained. Think blast radius: if the model is fully compromised, what is the worst an attacker can do through this channel?

Validate and constrain the output

• Force structured output (JSON schema or function-calling) so the response shape is predictable and typed.
• Validate tool-call arguments against allowlists — verify a file path is inside the sandbox, an email address is in the customer's domain, an amount is within policy.
• Run a rules-based or small-model filter on generated content that gets rendered or executed, catching injected links, script tags, and dangerous markdown.

Red-team like an attacker

1. Maintain a library of injection payloads: direct 'ignore instructions', indirect text hidden in documents, and multi-turn social-engineering chains.
2. Automate a weekly adversarial sweep against your pipeline and track detection rates over time.
3. Treat every real-world incident as a test case for the library, then fix the underlying control rather than the prompt.

FAQ