MLOps.community

Demetrios
  • 4.6 (24)
  • TECHNOLOGY
  • UPDATED WEEKLY

Relaxed Conversations around getting AI into production, whatever shape that may come in (agentic, traditional ML, LLMs, Vibes, etc)

  1. Performance Optimization and Software/Hardware Co-design across PyTorch, CUDA, and NVIDIA GPUs

    2H AGO

    Performance Optimization and Software/Hardware Co-design across PyTorch, CUDA, and NVIDIA GPUs

    March 3rd, Computer History Museum CODING AGENTS CONFERENCE, come join us while there are still tickets left. https://luma.com/codingagents Chris Fregly is currently focused on building and scaling high-performance AI systems, writing and teaching about AI infrastructure, helping organizations adopt generative AI and performance engineering principles on AWS, and fostering large developer communities around these topics.Performance Optimization and Software/Hardware Co-design across PyTorch, CUDA, and NVIDIA GPUs // MLOps Podcast #363 with Chris Fregly, Founder, AI Performance Engineer, and InvestorJoin the Community: https://go.mlops.community/YTJoinInGet the newsletter: https://go.mlops.community/YTNewsletterMLOps GPU Guide: https://go.mlops.community/gpuguide// AbstractIn today’s era of massive generative models, it's important to understand the full scope of AI systems' performance engineering. This talk discusses the new O'Reilly book, AI Systems Performance Engineering, and the accompanying GitHub repo (https://github.com/cfregly/ai-performance-engineering). This talk provides engineers, researchers, and developers with a set of actionable optimization strategies. You'll learn techniques to co-design and co-optimize hardware, software, and algorithms to build resilient, scalable, and cost-effective AI systems for both training and inference. // BioChris Fregly is an AI performance engineer and startup founder with experience at AWS, Databricks, and Netflix. He's the author of three (3) O'Reilly books, including Data Science on AWS (2021), Generative AI on AWS (2023), and AI Systems Performance Engineering (2025). He also runs the global AI Performance Engineering meetup and speaks at many AI-related conferences, including Nvidia GTC, ODSC, Big Data London, and more.// Related LinksAI Systems Performance Engineering: Optimizing Model Training and Inference Workloads with GPUs, CUDA, and PyTorch 1st Edition by Chris Fregly: https://www.amazon.com/Systems-Performance-Engineering-Optimizing-Algorithms/dp/B0F47689K8/Coding Agents Conference: https://luma.com/codingagents~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExploreJoin our Slack community [https://go.mlops.community/slack]Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)] Sign up for the next meetup: [https://go.mlops.community/register]MLOps Swag/Merch: [https://shop.mlops.community/]Connect with Demetrios on LinkedIn: /dpbrinkmConnect with Chris on LinkedIn: /cfregly

    1h 26m
  2. Serving LLMs in Production: Performance, Cost & Scale // CAST AI Roundtable

    5D AGO

    Serving LLMs in Production: Performance, Cost & Scale // CAST AI Roundtable

    Roundtable CAST AI episode: Serving LLMs in Production: Performance, Cost & Scale. Join the Community: https://go.mlops.community/YTJoinIn Get the newsletter: https://go.mlops.community/YTNewsletter MLOps GPU Guide: https://go.mlops.community/gpuguide // Abstract Experimenting with LLMs is easy. Running them reliably and cost-effectively in production is where things break. Most AI teams never make it past demos and proofs of concept. A smaller group is pushing real workloads to production—and running into very real challenges around infrastructure efficiency, runaway cloud costs, and reliability at scale. This session is for engineers and platform teams moving beyond experimentation and building AI systems that actually hold up in production. // Bio Ioana Apetrei Ioana is a Senior Product Manager at CAST AI, leading the AI Enabler product, an AI Gateway platform for cost-effective LLM infrastructure deployment. She brings 12 years of experience building B2C and B2B products reaching over 10 million users. Outside of work, she enjoys assembling puzzles and LEGOs and watching motorsports. Igor Šušić Igor is a founding Machine Learning Engineer at CAST AI’s AI Enabler, where he focuses on optimizing inference and training at scale. With a strong background in Natural Language Processing (NLP) and Recommender Systems, Igor has been tackling the challenges of large-scale model optimization long before transformers became mainstream. Prior to CAST AI, he worked at industry leaders like Bloomreach and Infobip, where he contributed to the development and deployment of large-scale AI and personalization systems from the early days of the field. // Related Links Website: https://cast.ai/ ~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~ Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore Join our Slack community [https://go.mlops.community/slack] Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)] Sign up for the next meetup: [https://go.mlops.community/register] MLOps Swag/Merch: [https://shop.mlops.community/] Connect with Demetrios on LinkedIn: /dpbrinkm Connect with Ioana on LinkedIn: /ioanaapetrei/ Connect with Igor on LinkedIn: /igor-%C5%A1u%C5%A1i%C4%87/

    1h 6m
  3. The Future of Information Retrieval: From Dense Vectors to Cognitive Search

    FEB 17

    The Future of Information Retrieval: From Dense Vectors to Cognitive Search

    Rahul Raja is a Staff Software Engineer at LinkedIn, working on large-scale search infrastructure, information retrieval systems, and integrating AI/ML to improve ranking and semantic search experiences. The Future of Information Retrieval: From Dense Vectors to Cognitive Search // MLOps Podcast #362 with Rahul Raja, Staff Software Engineer at LinkedIn Join the Community: https://go.mlops.community/YTJoinIn Get the newsletter: https://go.mlops.community/YTNewsletter MLOps GPU Guide: https://go.mlops.community/gpuguide // Abstract Information Retrieval is evolving from keyword matching to intelligent, vector-based understanding. In this talk, Rahul Raja explores how dense retrieval, vector databases, and hybrid search systems are redefining how modern AI retrieves, ranks, and reasons over information. He discusses how retrieval now powers large language models through Retrieval-Augmented Generation (RAG) and the new MLOps challenges that arise, embedding drift, continuous evaluation, and large-scale vector maintenance. Looking ahead, the session envisions a future of Cognitive Search, where retrieval systems move beyond recall to genuine reasoning, contextual understanding, and multimodal awareness. Listeners will gain insight into how the next generation of retrieval will bridge semantics, scalability, and intelligence, powering everything from search and recommendations to generative AI. // BioRahul is a Staff Engineer at LinkedIn, where he focuses on search and deployment systems at scale. Rahul is a graduate from Carnegie Mellon University and has a strong background in building reliable, high-performance infrastructure. He has led many initiatives to improve search relevance and streamline ML deployment workflows. // Related Links Website: https://www.linkedin.com/ Coding Agents Conference: https://luma.com/codingagents ~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~ Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore Join our Slack community [https://go.mlops.community/slack] Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)] Sign up for the next meetup: [https://go.mlops.community/register] MLOps Swag/Merch: [https://shop.mlops.community/] Connect with Demetrios on LinkedIn: /dpbrinkm Connect with Rahul on LinkedIn: /rahulraja963/ Timestamps: [00:00] Vector Search for Media [00:33] RAG and Search Evolution [04:45] Cognitive vs Semantic Search [08:26] High Value Search Signals [16:43] Scaling with Embeddings [22:37] BM25 Benchmark Bias [29:00] Video Search Use Cases [31:21] Context and Search Tradeoff [35:04] Personal Memory Augmentation [39:03] Future of Cognitive Search [44:51] Access Control in Vectors [49:14] Search Ranking Challenge [54:43] Hard Search Problems Solved [58:29] Freshness vs Cost [1:02:12] Wrap up

    1h 3m
  4. Rethinking Notebooks Powered by AI

    FEB 13

    Rethinking Notebooks Powered by AI

    Vincent Warmerdam is a Founding Engineer at marimo, working on reinventing Python notebooks as reactive, reproducible, interactive, and Git-friendly environments for data workflows and AI prototyping. He helps build the core marimo notebook platform, pushing its reactive execution model, UI interactivity, and integration with modern development and AI tooling so that notebooks behave like dependable, shareable programs and apps rather than error-prone scratchpads. Join the Community: https://go.mlops.community/YTJoinIn Get the newsletter: https://go.mlops.community/YTNewsletter MLOps GPU Guide: https://go.mlops.community/gpuguide // Abstract Vincent Warmerdam joins Demetrios fresh off marimo’s acquisition by Weights & Biases—and makes a bold claim: notebooks as we know them are outdated. They talk Molab (GPU-backed, cloud-hosted notebooks), LLMs that don’t just chat but actually fix your SQL and debug your code, and why most data folks are consuming tools instead of experimenting. Vincent argues we should stop treating notebooks like static scratchpads and start treating them like dynamic apps powered by AI. It’s a conversation about rethinking workflows, reclaiming creativity, and not outsourcing your brain to the model. // Bio Vincent is a senior data professional who worked as an engineer, researcher, team lead, and educator in the past. You might know him from tech talks with an attempt to defend common sense over hype in the data space. He is especially interested in understanding algorithmic systems so that one may prevent failure. As such, he has always had a preference to keep calm and check the dataset before flowing tonnes of tensors. He currently works at marimo, where he spends his time rethinking everything related to Python notebooks. // Related Links Website: https://marimo.io/ Coding Agent Conference: https://luma.com/codingagents Hyperbolic GPU Cloud: app.hyperbolic.ai ~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~ Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore Join our Slack community [https://go.mlops.community/slack] Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)] Sign up for the next meetup: [https://go.mlops.community/register] MLOps Swag/Merch: [https://shop.mlops.community/] MLOps GPU Guide: https://go.mlops.community/gpuguide Connect with Demetrios on LinkedIn: /dpbrinkm Connect with Vincent on LinkedIn: /vincentwarmerdam/ Timestamps: [00:00] Context in Notebooks [00:24] Acquisition and Team Continuity [04:43] Coding Agent Conference Announcement! [05:56] Hyperbolic GPU Cloud Ad [06:54] marimo and W&B Synergies [09:31] marimo Cloud Code Support [12:59] Hardest Code to Generate [16:22] Trough of Disillusionment [20:38] Agent Interaction in Notebooks [25:41] Wrap up

    26 min
  5. Software Engineering in the Age of Coding Agents: Testing, Evals, and Shipping Safely at Scale

    FEB 10

    Software Engineering in the Age of Coding Agents: Testing, Evals, and Shipping Safely at Scale

    Ereli Eran is the Founding Engineer at 7AI, where he’s focused on building and scaling the company’s agentic AI-driven cybersecurity platform — developing autonomous AI agents that triage alerts, investigate threats, enrich security data, and enable end-to-end automated security operations so human teams can focus on higher-value strategic work. Software Engineering in the Age of Coding Agents: Testing, Evals, and Shipping Safely at Scale // MLOps Podcast #361 with Ereli Eran, Founding Engineer at 7AI Join the Community: https://go.mlops.community/YTJoinIn Get the newsletter: https://go.mlops.community/YTNewsletter MLOps GPU Guide: https://go.mlops.community/gpuguide // Abstract A conversation on how AI coding agents are changing the way we build and operate production systems. We explore the practical boundaries between agentic and deterministic code, strategies for shared responsibility across models, engineering teams, and customers, and how to evaluate agent performance at scale. Topics include production quality gates, safety and cost tradeoffs, managing long-tail failures, and deployment patterns that let you ship agents with confidence. // Bio Ereli Eran is a founding engineer at 7AI, where he builds agentic AI systems for security operations and the production infrastructure that powers them. His work spans the full stack - from designing experiment frameworks for LLM-based alert investigation to architecting secure multi-tenant systems with proper authentication boundaries. Previously, he worked in data science and software engineering roles at Stripe, VMware Carbon Black, and was an early employee of Ravelin and Normalyze. // Related Links Website: https://7ai.com/ Coding Agents Conference: https://luma.com/codingagents ~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~ Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore Join our Slack community [https://go.mlops.community/slack] Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)] Sign up for the next meetup: [https://go.mlops.community/register] MLOps Swag/Merch: [https://shop.mlops.community/] Connect with Demetrios on LinkedIn: /dpbrinkm Connect with Ereli on LinkedIn: /erelieran/ Timestamps: [00:00] Language Sensitivity in Reasoning [00:25] Value of Claude Code [01:54] AI in Security Workflows [06:21] Agentic Systems Failures [12:50] Progressive Disclosure in Voice Agents [16:39] LLM vs Classic ML [19:44] Hybrid Approach to Fraud [25:58] Debugging with User Feedback [33:52] Prompts as Code [42:07] LLM Security Workflow [45:10] Shared Memory in Security [49:11] Common Agent Failure Modes [53:34] Wrap up

    57 min
  6. Physical AI: Teaching Machines to Understand the Real World

    FEB 6

    Physical AI: Teaching Machines to Understand the Real World

    Nick Gillian is the Co-Founder and CTO at Archetype AI, working on physical AI foundation models that understand and reason over real-world sensor data. Physical AI: Teaching Machines to Understand the Real World // MLOps Podcast #360 with Nick Gillian, Co-Founder and CTO of Archetype AI Join the Community: https://go.mlops.community/YTJoinIn Get the newsletter: https://go.mlops.community/YTNewsletter MLOps GPU Guide: https://go.mlops.community/gpuguide / Abstract As AI moves beyond the cloud and simulation, the next frontier is Physical AI: systems that can perceive, understand, and act within real-world environments in real time. In this conversation, Nick Gillian, Co-Founder and CTO of Archetype AI, explores what it actually takes to turn raw sensor and video data into reliable, deployable intelligence. Drawing on his experience building Google’s Soli and Jacquard and now leading development of Newton, a foundational model for Physical AI, Nick discusses how real-time physical understanding changes what’s possible across safety monitoring, infrastructure, and human–machine interaction. He’ll share lessons learned translating advanced research into products that operate safely in dynamic environments, and why many organizations underestimate the challenges and opportunities of AI in the physical world. // Bio Nick Gillian, Ph.D., is Co-Founder and CTO of Archetype AI with over 15 years of experience turning advanced AI and interaction research into real-world products. At Archetype, he leads the AI and engineering teams behind Newton—a first-of-its-kind Physical AI foundational model that can perceive, understand, and reason about the physical world. Before co-founding Archetype, Nick was a Senior Staff Machine Learning Engineer at Google and a researcher at MIT, where he developed AI and ML methods for real-time sensor understanding. At Google’s Advanced Technology and Projects group, he led machine learning research that powered breakthrough products like Soli radar and Jacquard, and helped advance sensing algorithms across Pixel, Nest, and wearable devices. // Related Links Website: https://www.archetypeai.io/https://www.archetypeai.io/blog/timefusion-newton https://www.nature.com/articles/s41598-023-44714-2https://www.youtube.com/watch?v=Pow4utY9teU https://www.youtube.com/watch?v=uE0jjdzwe9w https://arxiv.org/abs/2410.14724 Coding Agents Conference: https://luma.com/codingagents ~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~ Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore Join our Slack community [https://go.mlops.community/slack] Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)] Sign up for the next meetup: [https://go.mlops.community/register] MLOps Swag/Merch: [https://shop.mlops.community/] Connect with Demetrios on LinkedIn: /dpbrinkm Connect with Nick on LinkedIn: /nick-gillian-b27b1094/ Timestamps:[00:00] Physical Agent Framework[00:56] Physical AI Clarification[06:53] Building a Repair Model[12:41] World Models and LLMs[17:17] Data Weighting Strategies[24:19] Data Diversity vs Quantity[38:30] R&D and Product Creation[41:22] Construction Site Data Shipping[50:33] Wrap up

    52 min
  7. Speed and Scale: How Today's AI Datacenters Are Operating Through Hypergrowth

    FEB 3

    Speed and Scale: How Today's AI Datacenters Are Operating Through Hypergrowth

    Kris Beevers is the CEO at NetBox Labs, working on turning NetBox into the system of record and automation backbone for modern and AI-driven infrastructure. Speed and Scale: How Today's AI Datacenters Are Operating Through Hypergrowth // MLOps Podcast #359 with Kris Beevers, CEO of NetBox Labs Join the Community: https://go.mlops.community/YTJoinIn Get the newsletter: https://go.mlops.community/YTNewsletter MLOps GPU Guide: https://go.mlops.community/gpuguide // Abstract Hundreds of neocloud operators and "AI Factory" builders have emerged to serve the insatiable demand for AI infrastructure. These teams are compressing the design, build, deploy, operate, scale cycle of their infrastructures down to months, while managing massive footprints with lean teams. How? By applying modern intent-driven infrastructure automation principles to greenfield deployments. We'll explore how these teams carry design intent through to production, and how operating and automating around consistent infrastructure data is compressing "time to first train". // Bio Kris Beevers is the Co-founder and CEO of NetBox Labs. NetBox is used by nearly every Neocloud and AI datacenter to manage their networks and infrastructure. Kris is an engineer at heart and by background, and loves the leverage infrastructure innovation creates to accelerate technology and empower engineers to do their best work. A serial entrepreneur, Kris has founded and helped lead multiple other successful businesses in the internet and network infrastructure. Most recently, he co-founded and led NS1, which was acquired by IBM in 2023. He holds a Ph.D. in Computer Science from Rensselaer Polytechnic Institute and is based in New Jersey. // Related Links Website: https://netboxlabs.com/ Coding Agents Conference: https://luma.com/codingagents ~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~ Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore Join our Slack community [https://go.mlops.community/slack] Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)] Sign up for the next meetup: [https://go.mlops.community/register] MLOps Swag/Merch: [https://shop.mlops.community/] Connect with Demetrios on LinkedIn: /dpbrinkm Connect with Kris on LinkedIn: /beevek/ Timestamps: [00:00] Observability and Delta Analysis [00:26] New World Exploration [04:06] Bottlenecks in AI Infrastructure [13:37] Data Center Optimization Challenges [19:58] Tech Stack Breakdown [25:26] Data Center Design Principles [31:32] Constraints and Automation in Design [40:00] Complexity in Data Centers [45:02] GPU Cloud Landscape [50:24] Data Centers in Containers [57:45] Observability Beyond Software [1:04:43] Tighter Integrations vs NetBox [1:06:47] Wrap up

    1h 7m
  8. Cracking the Black Box: Real-Time Neuron Monitoring & Causality Traces

    JAN 27

    Cracking the Black Box: Real-Time Neuron Monitoring & Causality Traces

    Mike Oaten is the Founder and CEO of TIKOS, working on building AI assurance, explainability, and trustworthy AI infrastructure, helping organizations test, monitor, and govern AI models and systems to make them transparent, fair, robust, and compliant with emerging regulations. Cracking the Black Box: Real-Time Neuron Monitoring & Causality Traces // MLOps Podcast #358 with Mike Oaten, Founder and CEO of TIKOS Join the Community: https://go.mlops.community/YTJoinIn Get the newsletter: https://go.mlops.community/YTNewsletter // Abstract As AI models move into high-stakes environments like Defence and Financial Services, standard input/output testing, evals, and monitoring are becoming dangerously insufficient. To achieve true compliance, MLOps teams need to access and analyse the internal reasoning of their models to achieve compliance with the EU AI Act, NIST AI RMF, and other requirements. In this session, Mike introduces the company's patent-pending AI assurance technology that moves beyond statistical proxies. He will break down the architecture of the Synapses Logger, a patent-pending technology that embeds directly into the neural activation flow to capture weights, activations, and activation paths in real-time. // Bio Mike Oaten serves as the CEO of TIKOS, leading the company’s mission to progress trustworthy AI through unique, high-performance AI model assurance technology. A seasoned technical and data entrepreneur, Mike brings experience from successfully co-founding and exiting two previous data science startups: Riskopy Inc. (acquired by Nasdaq-listed Coupa Software in 2017) and Regulation Technologies Limited (acquired by mnAi Data Solutions in 2022). Mike's expertise spans data, analytics, and ML product and governance leadership. At TIKOS, Mike leads a VC-backed team developing technology to test and monitor deep-learning models in high-stakes environments, such as defence and financial services, so they comply with the stringent new laws and regulations. // Related Links Website: https://tikos.tech/ LLM guardrails: https://medium.com/tikos-tech/your-llm-output-is-confidently-wrong-heres-how-to-fix-it-08194fdf92b9 Model Bias: https://medium.com/tikos-tech/from-hints-to-hard-evidence-finally-how-to-find-and-fix-model-bias-in-dnns-2553b072fd83 Model Robustness: https://medium.com/tikos-tech/tikos-spots-neural-network-weaknesses-before-they-fail-the-iris-dataset-b079265c04da GPU Optimisation: https://medium.com/tikos-tech/400x-performance-a-lightweight-open-source-python-cuda-utility-to-break-vram-barriers-d545e5b6492f Hyperbolic GPU Cloud: app.hyperbolic.ai. Coding Agents Conference: https://luma.com/codingagents ~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~ Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore Join our Slack community [https://go.mlops.community/slack] Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)] Sign up for the next meetup: [https://go.mlops.community/register] MLOps Swag/Merch: [https://shop.mlops.community/] Connect with Demetrios on LinkedIn: /dpbrinkm Connect with Mike on LinkedIn: /mike-oaten/ Timestamps: [00:00] Regulations as Opportunity [00:25] Regulation Compliance Fun [02:49] AI Act Layers Explained [05:19] Observability in Systems vs ML [09:05] Risk Transfer in AI [11:26] LLMs and Model Approval [14:53] LLMs in Finance [17:17] Hyperbolic GPU Cloud Ad [18:16] Stakeholder Alignment and Tech [22:20] AI in Regulated Environments [28:55] Autonomous Boat Regulations [34:20] Data Compliance Mapping [39:11] Data Capture Strategy [41:13] EU AI Act Insights [44:52] Wrap up [45:45] Join the Coding Agents Conference!

    47 min
See All (502)

4.6
out of 5
24 Ratings

About

Relaxed Conversations around getting AI into production, whatever shape that may come in (agentic, traditional ML, LLMs, Vibes, etc)

Information

  • Creator
    Demetrios
  • Years Active
    2020 - 2026
  • Episodes
    502
  • Rating
    Clean
  • Copyright
    © Demetrios
  • Show Website
    MLOps.community

You Might Also Like