The Stack Overflow Podcast The Stack Overflow Podcast

SPONSORED BY MONGODB

Ben talks with Shane McAllister, lead developer advocate at MongoDB, Stanimira Vlaeva, senior developer advocate at MongoDB, and Miku Jha, director, AI/ML and generative AI at Google Cloud, about the challenges and opportunities of operationalizing and scaling generative AI models in enterprise organizations. They discuss the importance of data quality, the role of cloud providers in simplifying the deployment and management of AI models, and the challenges of transitioning from experimentation to production.

On this episode: Stack Overflow senior data scientist Michael Geden tells Ryan and Ben about how data scientists evaluate large language models (LLMs) and their output. They cover the challenges involved in evaluating LLMs, how LLMs are being used to evaluate other LLMs, the importance of data validating, the need for human raters, and more needs and tradeoffs involved in selecting and fine-tuning LLMs.

In the wake of the XZ backdoor, Ben and Ryan unpack the security implications of relying on open-source software projects maintained by small teams. They also discuss the open-source nature of Linux, the high cost of education in the US, the value of open-source contributions for job seekers, and what Apple is up to AI-wise.

SPONSORED BY INTEL
In this episode, Ben and Ryan are joined by Ria Cheruvu, an AI evangelist at Intel, to discuss the different approaches to incorporating AI models into organizations. They explore the decision tree for choosing between building your own model, delve into the ongoing challenges of maintenance and sustainability, and highlight the importance of optimizing and directing workloads.

The home team convenes to discuss the XZ backdoor attack, what great software engineers have in common, how GenAI is changing the face of drug development, and the rise of managed service providers for AI.

We sit down with Jessica Clark, a senior data scientist at Stack Overflow, to discuss how our company approaches generative AI and data quality. She discusses how we deal with risk mitigation when deploying non-deterministic systems that can produce different varying outputs to identical queries.