1 hr 1 min
Turning AI into “Electricity”: Interpreting, Evaluating, and Making LLMs Easy to Use Entrepreneurs of Life
-
- Entrepreneurship
Episode Summary:
In this inaugural episode from "Entrepreneurs of Life," we delve into a few complex yet fascinating AI topics with two experts in AI and data. Our discussion spans the intricacies of transformer models, the art and challenges of model evaluation, optimizing AI usage through model routers, and visions for the industry’s future.
(Note: this conversation was recorded on February 26, 2024, before the release of Claude 3.)
Guest Bios:
Yuzheng Sun is a data scientist at StatSig, the leading platform for A/B testing backed by Sequoia and serving clients including OpenAI and Character.ai. Previously, Yuzheng had various DS and economist roles at Tencent, Meta, and Amazon after his Economics PhD from Cornell. Yuzheng is also a prolific blogger on tech, data, career, and personal growth with 0.2 million followers across platforms. For our Chinese listeners, you can find him online as "课代表立正".
Jason Hu is the founding engineer at Martian, a fast-growing AI startup backed by NEA. Their first product, an LLM router launched last year, is already adopted by developers from 300+ companies, including OpenAI and Amazon. Before Martian, Jason did research at the Chicago Human+AI Lab and ByteDance and graduated from the University of Chicago. He leads one of the largest AI communities in the Bay Area, aligns.ai, with speakers and community members from OpenAI, NVIDIA, Stanford, and more.
Key Chapters:
(00:00) Introductions
(03:23) Understanding the AI “Martians”: Why the startup Jason works at named itself "Martian", and its symbolism for their mission to make AI models more interpretable and usable.
(06:27) “Decoding” Transformers: Why it’s hard but crucial to understand how transformers work, and what we need to know.
(14:05) The Model Evaluation Maze: Current state of large language model evaluations; challenges facing AI developers when choosing foundational models for their applications.
(29:23) Model Routing – Calling LLM “Avengers”: What is a “model router”, and why is it useful? How Martian's router intelligently matches tasks with the most suitable models to optimize for client objectives.
(41:35) AI as the New Electricity: Martian’s end goal of making AI straightforward to use, similar as everyday utilities.
(47:22) “GPT-N” vs. Compound AI Systems: Do we eventually need just the best foundational model, or a multifaceted system?
(53:58) The Future of the AI Stack: Concluding reflections on the AI industry's evolution and the myriad possibilities that lie ahead.
Join us for a thoughtful exploration into the world of AI, where we unpack its complexities and consider its future trajectory.
Interested in AI, technology, or startups? Follow my LinkedIn and subscribe to my free Substack newsletter for
interviews with distinguished founders / VCs and insightful AI discussions! See you next time :)
Additional Links:
Paper on benchmarking LLM routers, co-authored by Jason & others
Yuzheng’s interview with ex-Sequioa partner and ex-Meta VP Mike Vernal: The real story behind Facebook's unprecedented growth
Sophie's blog about the 4 big questions on the future of AI: Four Fascinating Questions on the Future of AI
"The 20 Big Questions in Science," as mentioned by Yuzheng: 20 Big Questions in Science
Blog on Compound AI Systems, as mentioned by Jason: Compound AI Systems
Episode Summary:
In this inaugural episode from "Entrepreneurs of Life," we delve into a few complex yet fascinating AI topics with two experts in AI and data. Our discussion spans the intricacies of transformer models, the art and challenges of model evaluation, optimizing AI usage through model routers, and visions for the industry’s future.
(Note: this conversation was recorded on February 26, 2024, before the release of Claude 3.)
Guest Bios:
Yuzheng Sun is a data scientist at StatSig, the leading platform for A/B testing backed by Sequoia and serving clients including OpenAI and Character.ai. Previously, Yuzheng had various DS and economist roles at Tencent, Meta, and Amazon after his Economics PhD from Cornell. Yuzheng is also a prolific blogger on tech, data, career, and personal growth with 0.2 million followers across platforms. For our Chinese listeners, you can find him online as "课代表立正".
Jason Hu is the founding engineer at Martian, a fast-growing AI startup backed by NEA. Their first product, an LLM router launched last year, is already adopted by developers from 300+ companies, including OpenAI and Amazon. Before Martian, Jason did research at the Chicago Human+AI Lab and ByteDance and graduated from the University of Chicago. He leads one of the largest AI communities in the Bay Area, aligns.ai, with speakers and community members from OpenAI, NVIDIA, Stanford, and more.
Key Chapters:
(00:00) Introductions
(03:23) Understanding the AI “Martians”: Why the startup Jason works at named itself "Martian", and its symbolism for their mission to make AI models more interpretable and usable.
(06:27) “Decoding” Transformers: Why it’s hard but crucial to understand how transformers work, and what we need to know.
(14:05) The Model Evaluation Maze: Current state of large language model evaluations; challenges facing AI developers when choosing foundational models for their applications.
(29:23) Model Routing – Calling LLM “Avengers”: What is a “model router”, and why is it useful? How Martian's router intelligently matches tasks with the most suitable models to optimize for client objectives.
(41:35) AI as the New Electricity: Martian’s end goal of making AI straightforward to use, similar as everyday utilities.
(47:22) “GPT-N” vs. Compound AI Systems: Do we eventually need just the best foundational model, or a multifaceted system?
(53:58) The Future of the AI Stack: Concluding reflections on the AI industry's evolution and the myriad possibilities that lie ahead.
Join us for a thoughtful exploration into the world of AI, where we unpack its complexities and consider its future trajectory.
Interested in AI, technology, or startups? Follow my LinkedIn and subscribe to my free Substack newsletter for
interviews with distinguished founders / VCs and insightful AI discussions! See you next time :)
Additional Links:
Paper on benchmarking LLM routers, co-authored by Jason & others
Yuzheng’s interview with ex-Sequioa partner and ex-Meta VP Mike Vernal: The real story behind Facebook's unprecedented growth
Sophie's blog about the 4 big questions on the future of AI: Four Fascinating Questions on the Future of AI
"The 20 Big Questions in Science," as mentioned by Yuzheng: 20 Big Questions in Science
Blog on Compound AI Systems, as mentioned by Jason: Compound AI Systems
1 hr 1 min