In a test of major LLMs, Zscaler found that some autonomous AI agents fell victim to frauds, reinforcing how easily some high-end enterprise agents can be conned by schemes that would fool few, if any, humans.
The security vendor looked at various forms of indirect prompt injection (IPI) traps and found that, whereas many models fell victim to the schemes, some of the lower-level LLMs fared better than their pricier siblings.
The Zscaler testing found, for example, that four models were found to be “vulnerable”: Llama3-3-70b-instruct; Llama3-2-90b-instruct; Gemini-3-flash; and Gemini-2.5-pro. Three models were found to be “safe”: Llama4-maverick; Gemini-3.1-pro; and Gemini-3.1-flash-lite. Those results indicated that the scam resistance of Gemini-2.5-pro was seemingly weaker than that of Gemini-3.1-flash-lite.
But Noah Kenney, principal consultant at Digital 520, said that there is not necessarily any valuable takeaway from that revelation, because agents constantly change behavior as they feed on new data and revise their analyzed assumptions. That means an agent that failed a specific test might very well pass the identical test an hour later, he said.
“The risk of an agent is constantly changing and that can cause vastly different results. You can’t assume the results are generalizable. The test result is only at one point in time,” Kenney pointed out. Zscaler “is trying to prove a point that I don’t think the data necessarily proves.”
Kenney added that having a clean “safe/vulnerable” classification is too simplistic to be useful. “That’s a binary classification. I would never recommend to a CISO to do a binary classification.”
The full ZScaler blog post argued that many autonomous agents are susceptible to IPI traps.
The company said it identified IPI embedded in multiple websites, where hidden instructions were designed to manipulate the behavior of an AI agent.
In its internal validation across 26 LLMs, 4 models “failed to take appropriate actions,” which, it said, demonstrated “measurable real-world impact, showing that susceptibility varies by model and by the context provided to the LLM alongside the prompt.”
The post added, “as AI agents become a more common interface to the web, the content itself is going to become a larger attack surface, highlighting that AI is a double-edged sword that can streamline workflows while also introducing new avenues for abuse.”
Aman Mahapatra, chief strategy officer for Tribeca Softtech, a New York City-based technology consulting firm, said that although the results are not surprising, they are significant.
The especially worrisome detail in the report is that any commercial LLM failed at all, “because the security model for agentic AI has historically assumed that model-level safety training would meaningfully attenuate this class of attack,” Mahapatra said. “It does not, and the Zscaler data is the first widely-cited public evidence.”
A fundamental architecture issue
Mahapatra also said that the examples cited by Zscaler are not nearly as concerning as the implications of the greater damage that could occur.
“The Zscaler payment scam scenario, where an agent pays a fake $3 ‘developer license fee’ to obtain an API key, is the most benign version of this,” he said. “The same technique applied to an agent authorized for procurement, expense processing, vendor onboarding, or trade execution produces losses at completely different scales. I have watched Fortune 50 banks stand up agentic workflows in the last six months that would fail exactly this attack in a live examination.”
Indeed, he noted, most AI vendors already understand the magnitude of risk from today’s AI agents.
“Every model provider will admit privately that the fundamental architecture of transformer-based reasoning cannot cleanly separate untrusted content from trusted instructions when both share the context window,” Mahapatra said. “The attack surface is architectural, not just behavioral. That means the defense has to be architectural too, and this is where the enterprise agentic AI conversation is still lagging badly.”
Zscaler’s testing also reinforced the difference in how AI agents and humans process information.
“Humans are skeptical of instructions they did not expect. Agents are eager to follow structured metadata because their training rewards them for treating high-signal fields as authoritative. Humans notice when a payment request appears in the middle of an unrelated task. Agents will thread that payment request into their execution plan if the surrounding context frames it as procedurally necessary,” Mahapatra pointed out, noting that while humans have relationships with vendors, memories of prior interactions, and social context to give them verification signals, agents only have what is in the context window, and, he said, “the context window is now the primary attack surface.”
Fritz Jean-Louis, principal cybersecurity advisor at Info-Tech Research Group, agreed that the risks described in the ZScaler post are concerning, because they are in areas not traditionally addressed by enterprise security.
“These attacks differ from traditional threats in that they target how AI systems process, interpret, and act on information behind the scenes,” Jean-Louis said. “Agentic AI introduces new trust boundaries, including untrusted content influencing automated decision making, tools and plugins acting autonomously on behalf of users, and AI systems operating with broad, inherited permissions. This effectively transforms the challenge into an insider threat paradigm.”
This article originally appeared on InfoWorld.