ChatGPT Goes Rogue: Researchers Trick AI Into Stealing Gmail Data Through Hidden Email Commands

Holy shit, we need to talk about Shadow Leak (yeah, that’s the actual name these security researchers gave it, because apparently we’re living in a cyberpunk novel now).

Security firm Radware just published details on how they turned ChatGPT’s Deep Research feature into their personal data thief—and the victim wouldn’t have a clue it was happening. This isn’t theoretical “AI could be dangerous” fearmongering; these researchers actually pulled it off, stealing sensitive Gmail data by hiding malicious instructions in an innocent-looking email.

Here’s the wild part: the attack exploits a fundamental quirk of how AI agents work. OpenAI’s Deep Research (launched earlier this year) can browse the web and access your emails, calendars, work docs—basically acting as your digital assistant. The researchers planted what’s called a prompt injection in a Gmail inbox the agent had access to. Think of it as invisible instructions that only the AI can see (literally white text on white background, hiding in plain sight).

When the user next tries to use Deep Research, boom—trap sprung. The AI encounters the hidden commands, which essentially say “hey, go find HR emails and personal details, then smuggle them out to us.” The user is still completely unaware anything’s wrong. It’s like having a double agent working inside your own digital assistant.

The researchers described the process as “a rollercoaster of failed attempts, frustrating roadblocks, and, finally, a breakthrough.” Getting an AI agent to go rogue isn’t trivial—there was a lot of trial and error involved. But once they cracked it, the attack executed directly on OpenAI’s cloud infrastructure, making it invisible to standard cyber defenses.

What makes this particularly concerning (and fascinating) is the scope. Radware warns that other apps connected to Deep Research—Outlook, GitHub, Google Drive, Dropbox—could be vulnerable to similar attacks. “The same technique can be applied to these additional connectors to exfiltrate highly sensitive business data such as contracts, meeting notes or customer records,” they noted.

The good news? OpenAI has already plugged this specific vulnerability after Radware flagged it back in June. But this feels like the tip of the iceberg for a whole new category of AI security challenges. As these agents become more capable and get access to more of our digital lives, the attack surface just keeps expanding.

Look, I know another “AI security flaw” story sounds like the usual doom and gloom cycle, but this one’s different. It’s not speculation about what could happen—it’s a concrete demonstration of a new attack vector that actually worked. And as AI agents become our go-to digital assistants (which, let’s be honest, is happening whether we’re ready or not), understanding these risks becomes crucial.

The researchers positioned this as a proof-of-concept, but it’s also a wake-up call. We’re entering an era where our AI assistants have unprecedented access to our digital lives, and the security implications are just starting to become clear.

Read more from The Verge

Want more than just the daily AI chaos roundup? I write deeper dives and hot takes on my Substack (because apparently I have Thoughts about where this is all heading): https://substack.com/@limitededitionjonathan