7 min
“Why I’m doing PauseAI” by Joseph Miller EA Forum Podcast (Curated & popular)
-
- Philosophy
GPT-5 training is probably starting around now. It seems very unlikely that GPT-5 will cause the end of the world. But it's hard to be sure. I would guess that GPT-5 is more likely to kill me than an asteroid, a supervolcano, a plane crash or a brain tumor. We can predict fairly well what the cross-entropy loss will be, but pretty much nothing else.
Maybe we will suddenly discover that the difference between GPT-4 and superhuman level is actually quite small. Maybe GPT-5 will be extremely good at interpretability, such that it can recursively self improve by rewriting its own weights.
Hopefully model evaluations can catch catastrophic risks before wide deployment, but again, it's hard to be sure. GPT-5 could plausibly be devious enough so circumvent all of our black-box testing. Or it may be that it's too late as soon as the model has been trained. These [...]
---
Outline:
(01:10) How do we do better for GPT-6?
(02:02) Plan B: Mass protests against AI
(03:06) No innovation required
(04:36) The discomfort of doing something weird
(05:53) Preparing for the moment
---
First published:
April 30th, 2024
Source:
https://forum.effectivealtruism.org/posts/J8sw7o5mWbGFaBW4o/why-i-m-doing-pauseai
---
Narrated by TYPE III AUDIO.
GPT-5 training is probably starting around now. It seems very unlikely that GPT-5 will cause the end of the world. But it's hard to be sure. I would guess that GPT-5 is more likely to kill me than an asteroid, a supervolcano, a plane crash or a brain tumor. We can predict fairly well what the cross-entropy loss will be, but pretty much nothing else.
Maybe we will suddenly discover that the difference between GPT-4 and superhuman level is actually quite small. Maybe GPT-5 will be extremely good at interpretability, such that it can recursively self improve by rewriting its own weights.
Hopefully model evaluations can catch catastrophic risks before wide deployment, but again, it's hard to be sure. GPT-5 could plausibly be devious enough so circumvent all of our black-box testing. Or it may be that it's too late as soon as the model has been trained. These [...]
---
Outline:
(01:10) How do we do better for GPT-6?
(02:02) Plan B: Mass protests against AI
(03:06) No innovation required
(04:36) The discomfort of doing something weird
(05:53) Preparing for the moment
---
First published:
April 30th, 2024
Source:
https://forum.effectivealtruism.org/posts/J8sw7o5mWbGFaBW4o/why-i-m-doing-pauseai
---
Narrated by TYPE III AUDIO.
7 min