You Can't Filter Your Way Out of Prompt Injection Attacks

Why input filtering fails against indirect prompt injection, and what actually reduces the blast radius.

That answer was defensible three years ago. It is now the most common way enterprises give themselves a false sense of security about their agents. 

On Thursday, August 27th, Straiker held a webinar with the Founder of OWASP AI Exchange, Rob van der Veer. There we discussed in depth what security for modern day agents looks like and what actions aren’t enough to secure enterprise AI agents. When we put the question of how effective the method of scanning malicious instructions and blocking red flags is on Rob, he did not soften it. 

"I think it's one of the least effective approaches, and I may offend some people." — Rob van der Veer

The reason is not that the filters are badly built. The reason is that filtering was designed for a system that no longer exists.

Filtering bad prompts made sense in November 2022

It helps to remember where the habit came from. Though the practice is, technically, just a few years old, it is outdated for what agents and attackers are able to do today. Straiker’s Co-Founder and CTO, Sreenath Kurupati, walked through the history during the webinar. 

"When ChatGPT was released 2022 November, at that time it was primarily a chat-based interface, and the kind of harms you were looking for was, trick the bot into giving a system prompt, trick the bot into saying something silly or harmful. At that point a legitimate case could have been made that you could just filter based on the content." — Sreenath

He is right that it was a reasonable call at the time. One user, one text box, one response. The entire attack surface was the string the user typed. If you could grade that string, you had covered the system.

Then the architecture changed underneath the control.

"Sometimes you start with a notion or a habit, and habits are hard to break." — Sreenath

Filtering is a 2022 habit applied to a 2026 architecture. Most of the tooling built around it inherited assumptions that stopped being true somewhere around the point where agents started autonomously accessing tools.

The user prompt is now the smallest part of the input

In an agentic application, the user's message is one contributor among many to what the model actually reasons over. A support agent pulls a customer record. A coding agent reads an issue thread, a README, and the contents of a dependency. A research agent fetches three web pages. Each of those becomes part of the context window, and the model does not maintain a hard boundary between the text that came from your user and the text that came from a retrieved document.

"Right now we're talking about agentic systems, with tools running behind the scenes. These tools have access to data. And then the user prompts, which come in as the initial input, then get overridden, often by documents and other sources of data, which become indirect prompts. And then tools run. And what happens is, the surface area is so large." — Sreenath Kurupati

This is the structural problem with input filtering. It inspects the one part of the input the attacker is least likely to use. The interesting instructions arrive through the tool call, the retrieved document, the MCP server response, or another agent's output. By the time they land, they are no longer distinguishable from anything else in the context.

Amy Heng framed the consequence precisely during the session: what matters is what action the agent takes on the prompt, what tools it reaches for, what context it pulled, and what it plans to do next. A control that only sees the opening message sees almost none of that.

Our threat research traces this across real agent categories, including coding agents, productivity agents, and the MCP supply chain, where untrusted content reaches the model through a channel nobody thought of as user input.

Malicious instructions look like the product working correctly

Even if you inspected every channel, you would hit a harder wall. Rob's example from client work states it cleanly.

"Let's say that you're GitLab, and you have an issue system where people can report bugs. That data goes into the agents. And that data contains instructions, because people say, I want a feature to do blah blah blah, and make it less red. So it seems like it's injecting instructions, but it's perfectly benign. Human text is full of instructions. How are you going to identify what is malicious?" — Rob

A bug tracker exists to carry human instructions into a development process. Feeding those instructions to an agent is the feature. No lexical property separates "add a dark mode toggle" from an instruction crafted to make the agent open a merge request it shouldn't.

Sreenath reached the same conclusion from the false-positive side. Any rule aggressive enough to catch injected instructions also catches legitimate ones, "and you're going to have too many false positives." A control that fires on normal product usage gets tuned down or switched off within a week.

Maliciousness is not a property of the text. It is a property of the text plus who sent it, what the agent was asked to do, and what the resulting tool call would touch.

Direct and indirect injection are different problems

Treating prompt injection as one category causes a lot of confusion. It behaves like two.

‍Direct injection is what the user types: jailbreaks, system prompt extraction, attempts to make the assistant say something it shouldn't. Rob considers these "less difficult than the indirect prompt injection," and the damage is mostly reputational. It's testable, measurable, and improvable.

‍Indirect injection is the agentic threat. The instruction arrives through data the agent consumed and results in an action: a tool call, an API write, an outbound message. Nobody has to read the model's response for damage to occur. In Rob's words, "for agentic AI, the indirect prompt injection is the biggest issue."

Teams reporting strong prompt injection coverage are usually measuring the first problem and reporting it as coverage of the second.

Why the industry keeps shipping shallow controls

Rob named the reason without diplomacy: catching this properly requires an LLM, "but in practice, companies decide not to use an LLM for this type of detection because it's simply too expensive. So that makes this unreliable."

Everyone in this space has run that arithmetic. Judging every prompt, tool output, and retrieved document with a frontier model costs more than the application it protects and adds latency users notice. So teams fall back on regex and stale jailbreak classifiers, then call it prompt injection detection.

Sreenath's version goes to mechanism rather than budget: "You really cannot do this with simple, old-style security rules. We are interplaying tools with natural language, and the only way to do an effective job is by having AI itself look at that."

Detection here means reading natural language, understanding what a tool call would do, and knowing whether that action makes sense for this user. Rules engines were never built for any of that. The fix is not abandoning AI-based detection. It's using domain-tuned small models instead of general-purpose frontier models for a job they're expensive at.

What detection actually requires: the whole trace

If maliciousness lives in the relationship between inputs and actions, the unit of analysis has to be the whole interaction.

"You take the user prompt, you take the agent's reasoning, you take the tool being run, you take the output that's coming in. We internally call it the entire LLM trace. You've got to look at that in its entirety. And that will tell you what the user's intent was, what the business intent was, did the agent get confused, did a malicious tool execution happen." — Sreenath

Read at that level, the GitLab case stops being ambiguous. A bug report requesting a feature is normal. That same report followed by agent reasoning pivoting toward credential access, followed by a tool call reaching an unrelated repository, is not.

Every layer alone is either noisy or blind. The prompt looks clean. The document looks like a support request. The tool call could be legitimate. The pattern only appears when they're evaluated together.

That's the design principle behind Defend AI, which inspects the full runtime trace including MCP and agent-to-agent traffic, using small language models tuned for security. That choice is what makes trace-level evaluation affordable at production volume.

Assume the injection already landed

"In agentic AI, the controls that are the most powerful are for when the prompt injection has already taken effect and trying to minimize the consequences." — Rob

Design for the assumption that some injection gets through, because some will. Rob's guidance follows directly: "you need to think about that every agent at every moment can have been compromised."

For how these attacks show up across real agent deployments, see Straiker Threat Research, Volume 1, including the STAR framework and guidance for defenders.

Three controls do most of the work:

Just-in-time authorization. Static RBAC assumes fixed intent, and agentic intent isn't fixed. One moment your agent is planning an application modernization, the next it's summarizing email. Rob has built mechanisms where the moment untrusted data enters the flow, hardening tightens automatically and certain command executions get disabled.

‍Blast radius limitation. Strip the agent's identity down, scope tokens tightly, expire them fast. Rob's framing is worth borrowing when you brief engineering: the agent should have to struggle a little, and "permission denied" during development should be routine rather than a bug. The common failure is teams running agents with every permission on because relaxing them later is painful.

‍Sound authorization plumbing. Sreenath's Instagram example is the cautionary tale. A support chatbot could be driven into resetting another user's password, meaning auth boundaries were being decided by the agent rather than enforced by the application. No guardrail compensates for that.

The agent kill switch belongs in your stack

The webinar opened on regulatory kill switches for frontier labs, and both speakers called that mandate impractical in a world where anyone can run an open-weight model. Sreenath: "The controls have to be distributed across corporations, customers, vendors." Rob: "The place where the application is being built, that's the place where you can really specify what is malicious and what is benign."

Both arguments land in the same place. A kill switch fails as national policy for the same reason it works as an enterprise capability. You are the only party who knows what your agents are supposed to do.

Rob also noted that vendors have been slow here, defaulting to incident response when the harm is already done: "Ideally you can stop the inference, stop the system if there's a certain detection that has enough certainty."

Defend AI now includes an agent kill switch for exactly this. When a trace shows an agent is 

compromised, you terminate it immediately rather than waiting for a playbook to run. It matters most in the case that's becoming common: several agents sharing tools and context, where one compromise propagates. Killing the affected set in one action contains the incident while your team investigates.

Where to start actually securing your agents

Find the surface. You can't reason about injection paths into agents you don't know exist. Discover AI inventories every agent, MCP server, model, and custom app in your environment and builds an agentic xBOM showing what talks to what. The unregistered MCP server is usually where untrusted data enters.

Prove which injections land. Ascend AI red-teams deployed agents continuously across 30+ exploit categories, including indirect prompt injection and agentic tool testing for dangerous command execution. Every attack it finds becomes a Defend AI policy recommendation.

‍Defend the trace and contain the damage. Defend AI enforces policy at runtime, alerts your SOC with investigation context, and gives you the kill switch when containment beats analysis.

Sreenath closed with the framing we keep returning to: "The AI genie is out in the open. I don't think we can put it back in. What we're going to try to do is hold on to the genie's legs and hands and limit what they can do."

Filtering was an attempt to keep the genie in the bottle. The work now is holding onto its hands.

Which security controls are necessary for agents today and which ones are just for show? 

For the full breakdown of which security controls just aren’t enough anymore and which ones your team actually needs, Straiker’s webinar—Debunking Agentic AI: The Truth About Identity, Agency, & Agentic Security—is available for you to watch at any time. 

Frequently Asked Questions (FAQs)

What is indirect prompt injection, and how is it different from direct prompt injection?

Direct injection is an instruction the user types, usually to jailbreak the model or extract its system prompt. Indirect injection arrives through data the agent consumed instead: a retrieved document, an email body, an issue thread, or an MCP server response. Because agents act on what they read, indirect injection produces real actions rather than just bad output.

Can indirect prompt injection be detected reliably?

Not at the string level, where most tooling looks. Injected instructions are lexically indistinguishable from legitimate ones, since many applications exist specifically to carry human instructions into a process. Detection becomes tractable only when you evaluate the full trace and treat it as one layer alongside authorization and containment.

Why aren't my model provider's guardrails enough?

They cover the model's own inputs and outputs, which is a small part of an agentic application. They have no visibility into which tools your agent can reach, what those tools return, or what an action means in your business.

What is an LLM trace, and why does detection need it?

An LLM trace is the full record of one agent interaction: user prompt, system prompt, reasoning, every tool invoked, arguments passed, and outputs returned. Prompt injection only shows up as a relationship between what was asked and what the agent then tried to do, so you need the sequence to see it.

What controls actually reduce indirect prompt injection risk?

Just-in-time authorization, where permissions tighten the moment untrusted data enters the flow, gives you the most return. Follow it with blast radius limits (minimum permissions, tightly scoped tokens, short expiry), sound authorization plumbing in the application itself, and input segregation. Then add trace-level runtime detection and a way to stop an agent mid-execution.

What is an agent kill switch, and when would I use it?

It terminates a running agent immediately rather than routing the event through a SOC playbook first. An agent with tool access has usually already exfiltrated data or written to a system by the time an analyst opens the ticket. It matters most where agents share tools and context, since one compromise can propagate to others.

‍Part one of a three-part series from our webinar with Rob van der Veer (Founder of the OWASP AI Exchange) and Sreenath Kurupati (CTO and Co-founder, Straiker), hosted by Amy Heng (Head of Marketing, Straiker).