Transcript: [00:10–00:19] all right all right welcome back to the super cycle today our guest is theo jaffe who [00:19–00:24] previously had his own podcast called the theo jaffe podcast and then went to work on the new [00:24–00:33] media team for for a16z and now is the co-host of mts or monitoring the situation which is one of [00:33–00:40] of the best tech live streams. It’s probably one of the only tech live streams, but I watch it all [00:40–00:47] the time. And it’s a part of the main places where I get my AI news, not only from their stream, [00:47–00:51] but from their newsletter as well, which I definitely recommend subscribing to. And there, [00:51–00:58] you can find them at mts.now. And so really excited to be able to get to chat with Theo today [00:58–01:06] about new media and his opinions on AI takeoff and forecasting, et cetera. [01:06–01:07] So welcome. [01:07–01:09] Yeah, Eli, thanks for having me on. [01:09–01:10] You know, I’m 22 years old. [01:10–01:13] I’m the youngest person in the room in SF most of the time. [01:13–01:15] Eli is one of the few people that makes me feel old. [01:16–01:17] Yeah. [01:17–01:18] Oh, yeah. [01:18–01:23] And I forgot to mention we’re here at the Overhang Conference in Washington, D.C., [01:23–01:28] which is run by, it’s a project of Sparrow Infrastructure, [01:29–01:32] which is also kind of light cone in which it’s like the manifest crowd, [01:33–01:35] but sponsored by MNX. [01:36–01:41] And so really excited to be here and start talking about this. [01:41–01:44] And so for those who don’t know, can you explain A16Z, [01:45–01:48] because they’re the sponsor of MTS, what’s their thesis about new media [01:48–01:51] and why is it important to have new media? [01:51–02:05] Well, A16Z is an investor in MTS. We were sort of incubated, spun out of the A16Z new media team. A lot of the original MTS team members, including me, were previously on the A16Z new media team. [02:05–02:23] I think the theory behind new media at A16Z is like for the longest time, trad media, especially trad tech media, stuff like, well, to say nothing of the New York Times and the Wall Street Journal and whatnot, which are generally pretty bad on tech coverage, especially the New York Times. [02:23–02:33] But even, you know, tech media like The Verge and TechCrunch and Wired have become like extremely negative to, hostile towards technology. [02:34–02:41] Most of what you read in the Verge Wired Tech Crunch and so on is written by people who kind of hate technology. [02:41–02:43] And they don’t really hide that. [02:43–02:52] You know, to them, journalism is about, quote, speaking truth to power, which they interpret as just like being really critical and negative of everything. [02:52–03:06] And so, you know, E16Z is an investment firm, Andreessen Horowitz, that invests in all kinds of amazing disruptive technologies from, you know, AI to space to biotech to software to like everything you can imagine, really. [03:07–03:15] And so the idea behind new media is just to create media that is unapologetically positive towards technology. [03:15–03:26] And not that this is dishonest or anything, like it is just true that technology is like one of the greatest levers that humanity has ever had for improving its own material living conditions. [03:27–03:35] And so like to have media that is unapologetically pro-tech and also, you know, novel and interesting in other ways is very important. [03:35–03:59] So while I agree with you that in general, in almost all cases, new innovation and new technologies are a net positive, but I’m wondering if you might feel that like traditional media has swung to bias negative on technology gradually. [03:59–04:04] gradually and now they hold this bias, it’s something that they’re kind of like, they have [04:04–04:10] to conform to. So like, even if there’s a single pro-tech writer at New York Times, they have to, [04:10–04:14] you know, kind of agree with the PAC and say, you know, data centers are bad for the environment, [04:14–04:18] which is obviously not true. Do you think the same thing could happen kind of in the other [04:18–04:24] direction where you become so committed to the pro-tech premise that if it looked like, for [04:24–04:29] example, even if you don’t think it’s the case now, if it looks like there’s a lot more AI [04:29–04:34] safety incidents in the near future and you might like not want it to cover them honestly do you [04:34–04:38] think it’s possible that it could become too biased in the other direction yes and like you [04:38–04:44] do see this with some of the pro tech people on twitter who basically deny that ai could ever [04:44–04:49] pose a risk of any kind especially an existential risk which is a very different position than ai [04:49–04:55] as a technology will be immensely net positive i believe ai as technology in expectation will be [04:55–04:59] immensely that positive but that doesn’t mean that there are absolutely no risks associated [04:59–05:05] with ai it doesn’t mean that existential risk is complete sci-fi nonsense like it’s definitely a [05:05–05:12] possibility at the minimum and and like there are people on x who kind of just deny this altogether [05:12–05:18] they’re they’re just like they’re too accelerationist it it becomes ideological [05:18–05:22] and it you know messes with their epistemics it messes with their model of the world [05:22–05:28] like the the correct take here is like technology is good almost all the time [05:28–05:34] all like almost all the time meaning there are very very very few edge cases where technology [05:34–05:38] might not be good can you give an example of one of this yeah existential risk from ai like i think [05:38–05:47] that if ai labs with the current level of if you want to call it safety that’s one thing but [05:47–05:53] techniques to ensure that AI systems are controllable, steerable, reliable, and interpretable. [05:53–05:58] If they were to vastly rapidly increase capabilities through recursive self-improvement [05:58–06:05] without a corresponding increase in our capabilities to control, steer, and interpret [06:05–06:10] these systems, then you could end up in a very, very bad situation where you have like [06:10–06:14] immensely powerful technology that we’re not able to adequately steer. [06:14–06:20] do you do you believe that that that kind of takeoff scenario is the path that we’re on [06:20–06:26] currently it’s hard to say i think like a lot of ai forecasting sort of assumes that [06:26–06:31] we’ll just like speed through a takeoff by default i don’t think that’s really the case i think that [06:31–06:37] there are pretty strong incentives for companies to if they’re about to undergo like ai like full [06:37–06:43] pretty hard takeoff for personal self-improvement and they have not yet [06:43–06:51] received enough assurance that the resulting technology will go well for them like I think [06:51–06:55] the incentive is just to not do that right like OpenAI doesn’t want to blow up the world [06:55–07:01] even in the most like selfish narrow sense you can’t IPO if you’re dead [07:01–07:31] Yeah, that’s true. [07:31–07:37] be like 70% that we like achieve like a $30 trillion market cap or something based on the [07:37–07:43] expected value for him that might be that might kind of contradict that no I think not I mean I [07:43–07:48] think in like first of all if that happened like the researchers a lot of the researchers would [07:48–07:55] just rebel like they would not be okay with kicking off full RSI if they thought there was a 10% [07:55–08:03] chance of human extinction secondly i think that sam all that wouldn’t do that like i i [08:03–08:04] You’re saying St. Albany wouldn’t do that? [08:05–08:06] I think St. Albany wouldn’t do that. [08:06–08:09] I think that like what opening I did after the hugging face incident, [08:09–08:14] where they decided to pause their frontier RL training and implement a bunch of [08:14–08:19] new safeguards and monitoring measures and whatnot was the correct and [08:19–08:21] responsible thing to do. [08:21–08:23] The hugging face incident itself, by the way, [08:23–08:25] it was not a big deal at all. [08:25–08:26] Like it was a big deal in, [08:26–08:29] in its implications for the alignment of future systems, [08:29–08:32] but in terms of what it actually was like opening, [08:33–08:33] I hacked hugging face. [08:33–08:36] It was a felony, but you know, no one was hurt. [08:36–08:38] There was not a lot of monetary damage. [08:39–08:48] If Sam Allman were like a full maximum accelerationist, he would have said like, oh, well, this, you know, safety incident clearly was just like a little bit of a monitoring failure. [08:48–08:50] We’re not going to pause our frontier RL training. [08:50–08:52] We’re not going to invest more resources in alignment. [08:53–08:53] We’re going to go full speed ahead. [08:54–08:54] And he didn’t do that. [08:54–08:57] So I like I reject the premise sort of. [08:57–09:10] Okay, and do you think, how effective currently do you think we can actually have safeguards in place? What’s your personal opinions on how well these are actually helping? [09:10–09:17] I think they’re helping pretty substantially. I think that our level of... [09:18–09:22] of alignment techniques are not sufficient to fully trust [09:22–09:25] that if we went through a hard takeoff, [09:25–09:27] things on the other end would be all right. [09:27–09:32] But I think that like in the vast majority of cases, [09:33–09:34] models do behave in aligned ways [09:34–09:37] and in the vast majority of deployments, right? [09:37–09:39]
