Last Week in AI

Funny story, my wife (a CPA) and I just listened to your podcast for 22 hours (broken up into 2 days) while driving out and back to Indiana for Thanksgiving (past episodes I missed). I was actually a bit saddened when the drive was over when we got back home. You two really are perfectly matched and make the show entertaining as well as informative. Keep up the great work!

Excellent podcast on the latest news. I’m in the industry side and have used ML for a long time in the non-IT sector (industrial and manufacturing). I’ve actually read papers that I first heard from here in this podcast. Doomerism is not my perspective but still really good knowing what are the new developments there. Maybe a quick mention on how erring on the side of caution can lead to negative outcomes? Kind of analogous to greenpeace and anti gmo groups opposing golden rice and that may have caused an additional 250,000 blind kids from vitamin A deficiency. Should we make the doomers pay into a liability fund just in case they prevented faster solving of existing problems? Obviously also thinking of the opposite possible doomsday case (if you think that’s a greater than 1% possibility).

I’m a tech CEO at an AI startup. The AI landscape is evolving so quickly that staying updated is essential - your approach can become obsolete, and roadblocks can vanish overnight with better tools. Your show has been indispensable for keeping up, and you don’t miss much at all. 1 request and 1 helpful note on AI image generators. REQUEST: Please keep us updated on a running basis on how Google’s and Meta’s LLMs stack up against OpenAI and Anthropic. LLMs are the backbone of the economic value in applications built on generative AI. You correctly point out how unreliable the benchmark tests are vs practice, and you do a great job focusing on the latter. You haven’t compared Gemini to GPT-4o, o1-preview, or Claude 3.5 in a while, so I assume it’s not competitive yet. I keep half expecting Google to take over, confused why Gemini currently is still so poor (in our testing). I’m counting on you to flag when they and Meta become meaningfully comparable to OpenAI and Anthropic. HELPFUL NOTE ON AI IMAGE GENERATORS: I want to leave a helpful note because you’re having trouble comparing AI image generator models. Jeremy admits not knowing (dude, you’re an epistemic rockstar!) if the tools are even distinguishable at this point, suspecting they’re already commoditized. Practically there’s huge differentiation. Here are the differentiators you’re missing, and would be helpful to hear you report on: 1. PROMPT ADHERENCE: Most models struggle to generate exactly what the text describes. 2. CHARACTER CONSISTENCY: Maintaining the same face/details (e.g., scars, tattoos) across images is nearly unworkably poor. 3. MULTIPLE CHARACTERS: Generating distinct characters in the same image is very difficult with only two; near impossible with 3 or more. 4. GENERATIVE EDITORS / AKA ERROR CORRECTION TOOLS: Ultimately these errors are irreducible, so generative editing tools become key for practical application at scale. Editing tools exist (e.g., Stability, Adobe, Flux Fill Pro) but remain clunky with all the same #1-3 issues but worse. Most of these tools right now are like working with a mercurial savant - produces breathtaking art, but almost incapable of following instructions let alone detailed ones. Hope this helps you guys distinguish when you report on AI image generation tools.

Big thanks to Jeremie and Andrey for all their work putting on this show. I really appreciate breakdown of all the news of tools, models, scholarship, commercial releases, policy, and more. The conversations help put in context the news which ads so much more value than the news itself. Thank you! I agree with other listeners that a discord community would be really helpful.