The Adversarial Testing Podcast

Damian
  • 0.0 (0)
  • Technology

Data-driven deep dives into AI adversarial testing, safety, and evaluation.

Episodes

  1. Jun 25

    Anthropic says Alibaba illicitly extracted Claude AI model capabilities

    Anthropic has accused Alibaba of illicitly extracting its Claude AI model capabilities through a large-scale "distillation" campaign, described in a letter to U.S. senators as the biggest known attack of its kind on the company. The piece details the scale of the alleged effort — tens of millions of exchanges across thousands of fraudulent accounts — and places it alongside earlier claims against DeepSeek, Moonshot AI, and MiniMax. It also covers the wider U.S.–China backdrop, including new Commerce Department restrictions on Anthropic's own models.

  2. Jun 23

    Agentic Security Benchmarks are Executable Threat Models: Holistic Evaluation of AI Defences

    This paper argues that security defences for AI agents should be judged not only by the threat they mitigate but by the security delta they introduce across the surrounding system. Using CaMeL — a leading defence against indirect prompt injection — the authors show that while it drives prompt-injection attack success to near zero, it offers no protection against denial-of-service attacks and introduces large latency, token, and retry-driven resource-exhaustion costs invisible under AgentDojo's single-metric evaluation. They call for benchmark environments grounded in threat taxonomies that report security deltas across adjacent attack classes, utility, cost, and operational viability.

  3. Jun 22

    ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?

    A verbatim reading of ExploitGym, a large-scale benchmark of eight hundred and ninety-eight real-world vulnerabilities across userspace programs, Google's V8 JavaScript engine, and the Linux kernel, designed to measure whether AI agents can turn a proof-of-vulnerability into a working exploit that achieves unauthorized code execution. The authors find that frontier models can already exploit a non-trivial fraction of these targets, with the strongest configurations producing working exploits for over a hundred and fifty instances, and that standard mitigations reduce but do not eliminate agent success. The paper establishes exploitation as an under-evaluated, dual-use capability and argues that autonomous exploit development by AI agents is no longer hypothetical.

  4. Jun 18

    Predicting Model Behavior Before Release by Simulating Deployment

    OpenAI describes Deployment Simulation, a method for previewing how a candidate model will behave in the real world before release by replaying recent, de-identified production conversations with the new model. This episode reads OpenAI's write-up in full and closes with a deeper, paper-based look at the technical methodology: the five-step resampling pipeline, how forecast error is decomposed, and the tool-simulator affordances that make agentic simulation realistic.

  5. Jun 18

    Viral Prompt Shows ChatGPT's Content Filters Don't Work

    Mindgard red-team research shows that ChatGPT's image generator can be manipulated into producing violent and sexually explicit content that users never directly requested, by abusing a fun viral "restore the photo" prompt circulating on social media. This episode walks through how nondescript prompts slip past input and output filters, why prompt repetition makes things worse, and OpenAI's inadequate response. Content warning: discussion of death, sexual violence and murder. The exact jailbreak prompts are deliberately not included.

  6. Jun 18

    AI Act: EP approves simplification measures and "nudifier" app ban

    The European Parliament has given final approval to changes to the EU AI Act under the digital omnibus package. The reforms postpone several high-risk and watermarking deadlines to give companies legal certainty, reduce overlapping rules for machinery products, and extend SME-style exemptions to small mid-cap enterprises. Alongside the simplification measures, Parliament secured an outright ban on "nudifier" apps and AI systems that generate child sexual abuse material or non-consensual intimate imagery.

  • 6 Episodes

About

Data-driven deep dives into AI adversarial testing, safety, and evaluation.

Information