Episode Summary Most AI conversations start with parameter counts. This one doesn’t. In this episode, we go inside the origin story of smallest.ai, a company built on the contrarian belief that true intelligence can be achieved with compute-constrained, smaller models — especially when the goal is real-time speech intelligence that can run actual workflows in production. Sudarshan shares how his background in self-driving vehicles shaped his thinking on reliability, active learning loops, and why 90–95% of the work lives in data and labeling, not model training. We then zoom into real-world enterprise use cases like collections, outbound calls, and multilingual customer support, and talk through how CIOs can actually start with voice AI in a messy legacy stack. In the second half, we switch gears into his founder journey: using LinkedIn and Discord as core distribution and learning channels, building the largest voice AI community, and his unfiltered advice on cold outreach, selecting whose advice to listen to, and running asset-light experiments before raising large rounds. If you’re a founder building AI for the enterprise — or an executive trying to separate hype from deployable systems — this episode will give you a grounded way to think about small models, agents, and voice AI. Key Topics - Origin story of smallest.ai and the shift from self-driving to speech AI. - Why “small vs large models” is the wrong framing — and how to think in terms of specialized vs general-purpose agents instead - Building one of the world’s fastest text-to-speech and speech-to-speech systems - Emotional information in audio vs traditional speech-to-text → LLM → TTS pipelines - Handling multilingual, code-switching conversations (Hinglish and Spanish/English) in real-world deployments - The hidden 90–95%: data collection, labeling, and active learning loops inspired by Tesla’s approach - How CIOs and CTOs can actually start: quick-win use cases in collections and outbound calling with simple Excel-based feedback loops - Why legacy call center software is optimized for human agents, not infinite-capacity AI agents - Who ends up making the buying decision: CEOs, CIOs, heads of AI transformation, and VPs of collections Building a founder-led growth engine: - 30K+ LinkedIn connections - The largest voice AI Discord community - Leveraging community feedback to shape product and GTM - Founder advice: cold outreach, whose advice to ignore, asset-light validation, and benchmarking yourself against the best Notable Quotes “We should stop talking about intelligence in terms of models. We should always talk about intelligence in terms of agents that do end-to-end tasks in the economy.” “Training is actually very quick. 90–95% of the work is the data — labeling it, fixing label errors, and feeding it back through active learning loops.” “For enterprises, start with quick wins. Collections is a great one — run outbound calls, compare the agent to your humans, and only then worry about integrating deeply into your systems.” “I wouldn’t take pitch deck advice from someone who’s never raised from a tier-one VC. Or engineering advice from someone who hasn’t written code in five years.” “Talking to a lot of high-agency people is a superpower — and social media is one of the fastest ways to make that happen as a founder.” About Sudarshan Kamath Sudarshan Kamath is the founder & CEO of smallest.ai, a company focused on building compute-efficient, real-time speech intelligence and specialized voice agents. Prior to smallest.ai, he worked on deploying deep learning systems for self-driving vehicles, building safety-critical systems that cannot fail. About Founder to Fortune Founder to Fortune is hosted by Vidya Raman, an investor and former operator who helps founders crack the enterprise market. Each episode dives deep into the realities of building, selling, and scaling products for enterprise customers — with operators, founders, and researchers who’ve actually done it. Subscribe on Spotify, Apple Podcasts, or YouTube, and leave a review if this episode helped you think differently about AI in the enterprise. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.foundertofortune.org
