AI Safety Newsletter Centre for AI Safety

Voluntary Commitments are Insufficient
AI companies agree to RSPs in Seoul. Following the second AI Global Summit held in Seoul, the UK and Republic of Korea governments announced that 16 major technology organizations, including Amazon, Google, Meta, Microsoft, OpenAI, and xAI have agreed to a new set of Frontier AI Safety Commitments.
Some commitments from the agreement include:
Assessing risks posed by AI models and systems throughout the AI lifecycle.
Setting thresholds for severe risks, defining when a model or system would pose intolerable risk if not adequately mitigated.
Keeping risks within defined thresholds, such as by modifying system behaviors and implementing robust security controls.
Potentially halting development or deployment if risks cannot be sufficiently mitigated.
These commitments [...]
---
Outline:
(00:03) Voluntary Commitments are Insufficient
(02:45) Senate AI Policy Roadmap
(05:18) Chapter 1: Overview of Catastrophic Risks
(07:56) Links
---

First published:

May 30th, 2024


Source:

https://newsletter.safe.ai/p/ai-safety-newsletter-35-voluntary

---
Want more? Check out our ML Safety Newsletter for technical safety research.


Narrated by TYPE III AUDIO.

OpenAI and Google Announce New Multimodal Models
In the current paradigm of AI development, there are long delays between the release of successive models. Progress is largely driven by increases in computing power, and training models with more computing power requires building large new data centers.
More than a year after the release of GPT-4, OpenAI has yet to release GPT-4.5 or GPT-5, which would presumably be trained on 10x or 100x more compute than GPT-4, respectively. These models might be released over the next year or two, and could represent large spikes in AI capabilities.
But OpenAI did announce a new model last week, called GPT-4o. The “o” stands for “omni,” referring to the fact that the model can use text, images, videos [...]
---
Outline:
(00:03) OpenAI and Google Announce New Multimodal Models
(02:36) The Surge in AI Lobbying
(05:29) How Should Copyright Law Apply to AI Training Data?
(10:10) Links
---

First published:

May 16th, 2024


Source:

https://newsletter.safe.ai/p/ai-safety-newsletter-35-lobbying

---
Want more? Check out our ML Safety Newsletter for technical safety research.


Narrated by TYPE III AUDIO.

AI Labs Fail to Uphold Safety Commitments to UK AI Safety Institute
In November, leading AI labs committed to sharing their models before deployment to be tested by the UK AI Safety Institute. But reporting from Politico shows that these commitments have fallen through.
OpenAI, Anthropic, and Meta have all failed to share their models with the UK AISI before deployment. Only Google DeepMind, headquartered in London, has given pre-deployment access to UK AISI.
Anthropic released the most powerful publicly available language model, Claude 3, without any window for pre-release testing by the UK AISI. When asked for comment, Anthropic co-founder Jack Clark said, “Pre-deployment testing is a nice idea but very difficult to implement.”
When asked about their concerns with pre-deployment testing [...]
---
Outline:
(00:03) AI Labs Fail to Uphold Safety Commitments to UK AI Safety Institute
(02:17) New Bipartisan AI Policy Proposals in the US Senate
(06:35) Military AI in Israel and the US
(11:44) New Online Course on AI Safety from CAIS
(12:38) Links
---

First published:

May 1st, 2024


Source:

https://newsletter.safe.ai/p/ai-safety-newsletter-34-new-military

---
Want more? Check out our ML Safety Newsletter for technical safety research.


Narrated by TYPE III AUDIO.

This week, we cover:
Consolidation in the corporate AI landscape, as smaller startups join forces with larger funders.
Several countries have announced new investments in AI, including Singapore, Canada, and Saudi Arabia.
Congress's budget for 2024 provides some but not all of the requested funding for AI policy. The White House's 2025 proposal makes more ambitious requests for AI funding.
How will AI affect biological weapons risk? We reexamine this question in light of new experiments from RAND, OpenAI, and others.
AI Startups Seek Support From Large Financial Backers
As AI development demands ever-increasing compute resources, only well-resourced developers can compete at the frontier. In practice, this means that AI startups must either partner with the world's [...]
---
Outline:
(00:45) AI Startups Seek Support From Large Financial Backers
(03:47) National AI Investments
(05:16) Federal Spending on AI
(08:35) An Updated Assessment of AI and Biorisk
(15:35) $250K in Prizes: SafeBench Competition Announcement
(16:08) Links
---

First published:

April 11th, 2024


Source:

https://newsletter.safe.ai/p/ai-safety-newsletter-33-reassessing

---
Want more? Check out our ML Safety Newsletter for technical safety research.


Narrated by TYPE III AUDIO.

Measuring and Reducing Hazardous Knowledge
The recent White House Executive Order on Artificial Intelligence highlights risks of LLMs in facilitating the development of bioweapons, chemical weapons, and cyberweapons.
To help measure these dangerous capabilities, CAIS has partnered with Scale AI to create WMDP: the Weapons of Mass Destruction Proxy, an open source benchmark with more than 4,000 multiple choice questions that serve as proxies for hazardous knowledge across biology, chemistry, and cyber.
This benchmark not only helps the world understand the relative dual-use capabilities of different LLMs, but it also creates a path forward for model builders to remove harmful information from their models through machine unlearning techniques.
Measuring hazardous knowledge in bio, chem, and cyber. Current evaluations of dangerous AI capabilities have [...]
---
Outline:
(00:03) Measuring and Reducing Hazardous Knowledge
(04:35) Language models are getting better at forecasting
(07:51) Proposals for Private Regulatory Markets
(14:25) Links
---

First published:

March 7th, 2024


Source:

https://newsletter.safe.ai/p/ai-safety-newsletter-32-measuring

---
Want more? Check out our ML Safety Newsletter for technical safety research.


Narrated by TYPE III AUDIO.

This week, we’ll discuss:
A new proposed AI bill in California which requires frontier AI developers to adopt safety and security protocols, and clarifies that developers bear legal liability if their AI systems cause unreasonable risks or critical harms to public safety.
Precedents for AI governance from healthcare and biosecurity.
The EU AI Act and job opportunities at their enforcement agency, the AI Office.
A New Bill on AI Policy in California
Several leading AI companies have public plans for how they’ll invest in safety and security as they develop more dangerous AI systems. A new bill in California's state legislature would codify this practice as a legal requirement, and clarify the legal liability faced by developers [...]
---
Outline:
(00:33) A New Bill on AI Policy in California
(04:38) Precedents for AI Policy: Healthcare and Biosecurity
(07:56) Enforcing the EU AI Act
(08:55) Links
---

First published:

February 21st, 2024


Source:

https://newsletter.safe.ai/p/aisn-31-a-new-ai-policy-bill-in-california

---
Want more? Check out our ML Safety Newsletter for technical safety research.


Narrated by TYPE III AUDIO.