MLOps.community Demetrios Brinkmann

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/.

Benjamin Wilms is a developer and software architect at heart, with 20 years of experience. He fell in love with chaos engineering. Benjamin now spreads his enthusiasm and new knowledge as a speaker and author – especially in the field of chaos and resilience engineering.

Retrieval Augmented Generation // MLOps podcast #237 with Benjamin Wilms, CEO & Co-Founder of Steadybit.

Huge thank you to Amazon Web Services for sponsoring this episode. AWS - https://aws.amazon.com/

// Abstract
How to build reliable systems under unpredictable conditions with Chaos Engineering.

// Bio
Benjamin has over 20 years of experience as a developer and software architect. He fell in love with chaos engineering 7 years ago and shares his knowledge as a speaker and author. In October 2019, he founded the startup Steadybit with two friends, focusing on developers and teams embracing chaos engineering. He relaxes by mountain biking when he's not knee-deep in complex and distributed code.

// MLOps Jobs board
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Website: https://steadybit.com/

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Benjamin on LinkedIn: https://www.linkedin.com/in/benjamin-wilms/

Timestamps:
[00:00] Benjamin's preferred coffee
[00:28] Takeaways
[02:10] Please like, share, leave a review, and subscribe to our MLOps channels!
[02:53] Chaos Engineering tldr
[06:13] Complex Systems for smaller Startups
[07:21] Chaos Engineering benefits
[10:39] Data Chaos Engineering trend
[15:29] Chaos Engineering vs ML Resilience
[17:57 - 17:58] AWS Trainium and AWS Infecentia Ad
[19:00] Chaos engineering tests system vulnerabilities and solutions

[23:24] Data distribution issues across different time zones

[27:07] Expertise is essential in fixing systems

[31:01] Chaos engineering integrated into machine learning systems

[32:25] Pre-CI/CD steps and automating experiments for deployments

[36:53] Chaos engineering emphasizes tool over value

[38:58] Strong integration into observability tools for repeatable experiments

[45:30] Invaluable insights on chaos engineering

[46:42] Wrap up

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

Tom Smoker is the cofounder of an early stage tech company empowering developers to create knowledge graphs within their RAG pipelines. Tom is a technical founder, and owns the research and development of knowledge graphs tooling for the company.

Managing Small Knowledge Graphs for Multi-agent Systems // MLOps podcast #236 with Tom Smoker, Technical Founder of whyhow.ai.

A big thank you to  @latticeflow  for sponsoring this episode! LatticeFlow - https://latticeflow.ai/

// Abstract
RAG is one of the more popular use cases for generative models, but there can be issues with repeatability and accuracy. This is especially applicable when it comes to using many agents within a pipeline, as the uncertainty propagates. For some multi-agent use cases, knowledge graphs can be used to structurally ground the agents and selectively improve the system to make it reliable end to end.

// Bio
Technical Founder of WhyHow.ai. Did Masters and PhD in CS, specializing in knowledge graphs, embeddings, and NLP. Worked as a data scientist to senior machine learning engineer at large resource companies and startups.

// MLOps Jobs board
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links

A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models: https://arxiv.org/abs/2401.01313Understanding the type of Knowledge Graph you need — Fixed vs Dynamic Schema/Data: https://medium.com/enterprise-rag/understanding-the-type-of-knowledge-graph-you-need-fixed-vs-dynamic-schema-data-13f319b27d9e

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Tom on LinkedIn: https://www.linkedin.com/in/thomassmoker/

Timestamps:
[00:00] Tom's preferred coffee
[00:33] Takeaways
[03:04] Please like, share, leave a review, and subscribe to our MLOps channels!
[03:23] Academic Curiosity and Knowledge Graphs
[05:07] Logician
[05:53] Knowledge graphs incorporated into RAGs
[07:53] Graphs & Vectors Integration
[10:49] "Exactly wrong"
[12:14] Data Integration for Robust Knowledge Graph
[14:53] Structured and Dynamic Data
[21:44] Scoped Knowledge Retrieval Strategies
[28:01 - 29:32] LatticeFlow Ad
[29:33] RAG Limitations and Solutions
[36:10] Working on multi agents, questioning agent definition

[40:01] Concerns about performance of agent information transfer

[43:45] Anticipating agent-based systems with modular processes

[52:04] Balancing risk tolerance in company operations and control

[54:11] Using AI to generate high-quality, efficient content

[01:03:50] Wrap up

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

David Nunez, based in Santa Barbara, CA, US, is currently a Co-Founder and Partner at Abstract Group, bringing experience from previous roles at First Round Capital, Stripe, and Slab.

Just when we Started to Solve Software Docs, AI Blew Everything Up // MLOps Podcast #235 with Dave Nunez, Partner of Abstract Group co-hosted by Jakub Czakon.

Huge thank you to Zilliz for sponsoring this episode. Zilliz - https://zilliz.com/.

// Abstract
Over the previous decade, the recipe for making excellent software docs mostly converged on a set of core goals:

Create high-quality, consistent content
Use different content types depending on the task
Make the docs easy to find

For AI-focused software and products, the entire developer education playbook needs to be rewritten.

// Bio
Dave lives in Santa Barbara, CA with his wife and four kids.

He started his tech career at various startups in Santa Barbara before moving to San Francisco to work at Salesforce. After Salesforce, he spent 2+ years at Uber and 5+ years at Stripe leading internal and external developer documentation efforts.

In 2021, he co-authored Docs for Developers to help engineers become better writers. He's now a consultant, advisor, and angel investor for fast-growing startups. He typically invests in early-stage startups focusing on developer tools, productivity, and AI.

He's a reading nerd, Lakers fan, and golf masochist.

// MLOps Jobs board
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Website: https://www.abstractgroup.co/
Book: docsfordevelopers.com
About Dave: https://gamma.app/docs/Dave-Nunez-about-me-002doxb23qbblme?mode=doc
https://review.firstround.com/investing-in-internal-documentation-a-brick-by-brick-guide-for-startups
https://increment.com/documentation/why-investing-in-internal-docs-is-worth-it/

Writing to Learn paper by Peter Elbow: https://peterelbow.com/pdfs/Writing_for_Learning-Not_just_Demonstrating.PDF


--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Dave on LinkedIn: https://www.linkedin.com/in/djnunez/
Connect with Kuba on LinkedIn: https://www.linkedin.com/in/jakub-czakon/?locale=en_US

Timestamps:
[00:00] Dave's preferred coffee
[00:13] Introducing this episode's co-host, Kuba
[00:36] Takeaways
[02:55] Please like, share, leave a review, and subscribe to our MLOps channels!
[03:23] Good docs, bad docs, and how to feel them
[06:51] Inviting Dev docs and checks
[10:36] Stripe's writing culture
[12:42] Engineering team writing culture
[14:15] Bottom-up tech writer change
[18:31] Strip docs cult following
[24:40] TriDocs Smart API Injection
[26:42] User research for documentation
[29:51] Design cues
[32:15] Empathy-driven docs creation
[34:28 - 35:35] Zilliz Ad
[35:36] Foundational elements in documentation
[38:23] Minimal infrastructure of information in "Read Me"
[40:18] Measuring documentation with OKRs
[43:58] Improve pages with Analytics
[47:33] Google branded doc searches
[48:35] Time to First Action
[52:52] Dave's day in and day out and what excites him
[56:01] Exciting internal documentation
[59:55] Wrap up

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/


Cody Peterson has a diverse work experience in the field of product management and engineering. Cody is currently working as a Technical Product Manager at Voltron Data, starting from May 2023. Previously, they worked as a Product Manager at dbt Labs from July 2022 to March 2023.

MLOps podcast #234 with Cody Peterson, Senior Technical Product Manager at Voltron Data | Ibis project // Open Standards Make MLOps Easier and Silos Harder.

Huge thank you to Weights & Biases for sponsoring this episode. WandB Free Courses -http://wandb.me/courses_mlops

// Abstract
MLOps is fundamentally a discipline of people working together on a system with data and machine learning models. These systems are already built on open standards we may not notice -- Linux, git, scikit-learn, etc. -- but are increasingly hitting walls with respect to the size and velocity of data.

Pandas, for instance, is the tool of choice for many Python data scientists -- but its scalability is a known issue. Many tools make the assumption of data that fits in memory, but most organizations have data that will never fit in a laptop. What approaches can we take?

One emerging approach with the Ibis project (created by the creator of pandas, Wes McKinney) is to leverage existing "big" data systems to do the heavy lifting on a lightweight Python data frame interface. Alongside other open source standards like Apache Arrow, this can allow data systems to communicate with each other and users of these systems to learn a single data frame API that works across any of them.

Open standards like Apache Arrow, Ibis, and more in the MLOps tech stack enable freedom for composable data systems, where components can be swapped out allowing engineers to use the right tool for the job to be done. It also helps avoid vendor lock-in and keep costs low.

// Bio
Cody is a Senior Technical Product Manager at Voltron Data, a next-generation
data systems builder that recently launched an accelerator-native GPU query
engine for petabyte-scale ETL called Theseus. While Theseus is proprietary,
Voltron Data takes an open periphery approach -- it is built on and interfaces
through open standards like Apache Arrow, Substrait, and Ibis. Cody focuses on the Ibis project, a portable Python dataframe library that aims to be the
standard Python interface for any data system, including Theseus and over 20
other backends.

Prior to Voltron Data, Cody was a product manager at dbt Labs focusing on the open source dbt Core and launching Python models (note: models is a confusing term here). Later, he led the Cloud Runtime team and drastically improved the efficiency of engineering execution and product outcomes.

Cody started his carrer as a Product Manager at Microsoft working on Azure ML. He spent about 2 years on the dedicated MLOps product team, and 2 more years on various teams across the ML lifecycel including data, training, and inferencing.

He is now passionate about using open source standards to break down the silos and challenges facing real world engineering teams, where engineering
increasingly involves data and machine learning.

// MLOps Jobs board
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Ibis Project: https://ibis-project.org
Apache Arrow and the “10 Things I Hate About pandas”: https://wesmckinney.com/blog/apache-arrow-pandas-internals/

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Cody on LinkedIn: https://linkedin.com/in/codydkdc

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/



Syed Asad is an Innovator, Generative AI & Machine Learning Engineer, and a Champion for Ethical AI
MLOps podcast #233 with Syed Asad, Lead AI/ML Engineer at KiwiTech // Retrieval Augmented Generation.

A big thank you to @ for sponsoring this episode! AWS -

// Abstract
Everything and anything around RAG.

// Bio
Currently Exploring New Horizons:
Syed is diving deep into the exciting world of Semantic Vector Searches and Vector Databases. These innovative technologies are reshaping how we interact with and interpret vast data landscapes, opening new avenues for discovery and innovation.

Specializing in Retrieval Augmented Generation (RAG):
Syed's current focus also includes mastering Retrieval Augmented Generation Techniques (RAGs). This cutting-edge approach combines the power of information retrieval with generative models, setting new benchmarks in AI's capability and application.

// MLOps Jobs board
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Website: https://sanketgupta.substack.com/
Our paper on this topic "Generalized User Representations for Transfer Learning": https://arxiv.org/abs/2403.00584
Sanket's blogs on Medium in the past: https://medium.com/@sanket107

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Syed on LinkedIn: https://www.linkedin.com/in/syed-asad-76815246/

Timestamps:
[00:00] Syed's preferred coffee
[00:31] Takeaways
[03:17] Please like, share, leave a review, and subscribe to our MLOps channels!
[03:37] A production issue
[07:37] CSV file handling risks
[09:42] Embedding models not suitable
[11:22] Inference layer experiments and use cases
[14:00] AWS service handling the issue
[17:35] Salad testing and insights
[22:12] OpenAI vs Customization
[24:30] Difference between Olama and VLLM
[27:16] Fine-tuning of small LLMs
[29:51] Evaluation framework
[32:04] MLOps for efficient ML
[37:12] Determining the pricing of tools
[39:35] Manage Dependency Risk
[40:27] Get in touch with Syed on LinkedIn
[41:46] ML Engineers are now all AI Engineers
[43:01] The hard framework
[43:53] Wrap up

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/



Sanket works as a Senior Machine Learning Engineer at Spotify working on building end-to-end audio recommender systems. Models built by his team are used across Spotify in many different products including Discover Weekly and Autoplay.

MLOps podcast #232 with Sanket Gupta, Senior Machine Learning Engineer at Spotify //
RecSys at Spotify.

A big thank you to LatticeFlow for sponsoring this episode! LatticeFlow - https://latticeflow.ai/

// Abstract
LLMs with foundational embeddings have changed the way we approach AI today. Instead of re-training models from scratch end-to-end, we instead rely on fine-tuning existing foundation models to perform transfer learning.
Is there a similar approach we can take with recommender systems?
In this episode, we can talk about:
a) how Spotify builds and maintains large-scale recommender systems,
b) how foundational user and item embeddings can enable transfer learning across multiple products,
c) how we evaluate this system
d) MLOps challenges with these systems

// Bio
Sanket works as a Senior Machine Learning Engineer on a team at Spotify building production-grade recommender systems. Models built by my team are being used in Autoplay, Daily Mix, Discover Weekly, etc.
Currently, my passion is how to build systems to understand user taste - how do we balance long-term and short-term understanding of users to enable a great personalized experience.

// MLOps Jobs board
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Website: https://sanketgupta.substack.com/
Our paper on this topic "Generalized User Representations for Transfer Learning": https://arxiv.org/abs/2403.00584
Sanket's blogs on Medium in the past: https://medium.com/@sanket107

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Sanket on LinkedIn: www.linkedin.com/in/sanketgupta107

Timestamps:
[00:00] Sanket's preferred coffee
[00:37] Takeaways
[02:30] RecSys are RAGs
[06:22] Evaluating RecSys parallel to RAGs
[07:13] Music RecSys Optimization
[09:46] Dealing with cold start problems
[12:18] Quantity of models in the recommender systems
[13:09] Radio models
[16:24] Evaluation system
[20:25] Infrastructure support
[21:25] Transfer learning
[23:53] Vector database features
[25:31] Listening History Balance
[26:35 - 28:06] LatticeFlow Ad
[28:07] The beauty of embeddings
[30:13] Shift to real-time recommendation
[34:05] Vector Database Architecture Options
[35:30] Embeddings drive personalized
[40:16] Feature Stores vs Vector Databases
[42:33] Spotify product integration strategy
[45:38] Staying up to date with new features
[47:53] Speed vs Relevance metrics
[49:40] Wrap up