DataTalks.Club

DataTalks.Club
  • 5.0 (7)
  • Technology
  • Updated Weekly

DataTalks.Club - the place to talk about data!

  1. 1d ago

    Building AI Evaluations You Can Actually Trust - Tejas Kumar

    Tejas Kumar joins Alexey Grigorev to discuss building evaluation datasets, calibrating LLM judges against human preferences, and testing agents beyond their final answers. CHAPTERS 00:00 Podcast intro 00:08 Tejas's path to AI engineering 12:24 Why evals go beyond a manual check 17:37 Sourcing evaluation cases and calibrating a judge 28:33 Refreshing datasets and pinning severe cases 33:10 Judge agreement versus agent performance 50:01 Retraining judges and keeping data fresh 55:59 Human preferences, sample sizes and audience questions 59:46 Evaluating final answers versus the agent's trajectory 1:02:46 Faster judging and further evaluation resources Resources AI evals: https://tej.as/blog/ai-evals-llm-judge Code, demos and slides: https://github.com/TejasQ/basically-agent-evals Tejas: https://tej.as Watch the original recording: https://www.youtube.com/watch?v=oWVmQDqZ7tw Join DataTalks.Club: https://datatalks.club/slack.html

  2. Oct 16

    AI Platform: Memory, Tools, Guardrails, Observability, Shipping a Production Agent - Dewang Sultania

    In this talk, Dewang Sultania, ML Engineer at Netflix and co-author of Designing AI Systems, shares his technical journey from building early inference platforms at Adobe to deploying multimodal AI models at scale. We explore the complete architecture of enterprise AI platforms, focusing on evaluation datasets, tool calling orchestration, and the practical realities of shipping production agents. You’ll learn about: - The essential components needed to build an enterprise AI platform from scratch - How traditional software engineers can transition into AI platform engineering - The critical technical differences between session management and LLM memory - Strategies for building, versioning, and trusting custom evaluation datasets - Optimizing LLM tool calling and selecting the right performance benchmarks - When to use multi-agent architectures versus single-agent systems - Techniques for overcoming the lost in the middle problem in RAG pipelines TIMECODES: 00:00 Core Components of Enterprise AI Platforms 11:46 Career Path to AI Platform Engineering 18:33 Session Management Versus LLM Memory Management 26:47 Best Practices for Custom LLM Evaluation Datasets 32:28 LLM Tool Calling Optimization and Benchmark Selection 39:25 Context Window Limits and AI Agent Tool Scaling 45:37 Single-Agent Versus Multi-Agent System Architecture 52:16 LLM-as-a-Judge Alignment and Human Evaluation Strategies 58:11 Solutions for the Lost in the Middle Problem in RAG Systems 01:03:27 Designing AI Systems Book Announcement This discussion is perfect for software engineers, machine learning practitioners, and technical leaders building real-world AI applications. It offers practical insights for anyone transitioning into AI platform engineering or tasked with scaling LLM features for enterprise production. Connect with Dewang: - Linkedin - https://www.linkedin.com/in/dewang-sultania/ - Github - https://github.com/iron-stark Connect with DataTalks.Club: - Join the community - https://datatalks.club/slack.html - Subscribe to our Google calendar to have all our events in your calendar - https://calendar.google.com/calendar/r?cid=ZjhxaWRqbnEwamhzY3A4ODA5azFlZ2hzNjBAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ - Check other upcoming events - https://lu.ma/dtc-events - GitHub: https://github.com/DataTalksClub - LinkedIn - https://www.linkedin.com/company/datatalks-club/ - Twitter - https://twitter.com/DataTalksClub - Website - https://datatalks.club/

  3. Oct 9

    Building an AI Platform: SDK Design, Model Routing, Vector Search & Evals - Suhas Suresha

    In this talk, Suhas Suresha, Senior Machine Learning Engineer at Adobe and co-author of Designing AI Systems, shares his expertise in building large-scale generative AI infrastructure from deploying real-time ECG analysis models at his previous startup, QALY, to architecting enterprise LLM platforms. We explore the massive engineering effort required to take an AI application from a weekend prototype to a scalable production system, and how to build a unified AI platform that eliminates "AI sprawl." Links: - Book - https://www.manning.com/books/designing-ai-systems - Code - https://github.com/designing-ai-systems/designing_ai_systems_repo You’ll learn about: - The nine core components every production-ready AI platform needs to succeed. - How to solve "AI sprawl" and prevent fragmented engineering efforts across teams. - Architecting scalable Data Services and Model Services for robust LLM applications. - Implementing comprehensive observability and request tracing to identify system bottlenecks. - Managing long-term session memory, tool registries, and complex agent orchestration. - Navigating the "build versus buy" dilemma for enterprise AI infrastructure. TIMECODES: 0:00 AI Prototypes to Production and the Book Origin 5:49 AI Sprawl and the Need for a Centralized AI Platform 12:42 Definition of an AI Platform and the Build vs Buy Dilemma 23:17 AI Platform Engineering Skills and Data Service Architecture 31:26 System Observability and Request Flow Bottlenecks 37:32 LLM Evaluations, Prompt Experimentation, and Cost Tracking 43:57 Session Memory, Tool Registries, and Agent Orchestration 50:56 Invocation Modes and Durable Execution for AI Agents 56:14 The Nine Core AI Platform Components and Memory Management This episode is designed for machine learning engineers, backend developers, and platform architects looking to transition into AI platform engineering. It is highly recommended for technical leaders and teams who need to scale LLM prototypes into reliable, cost-effective production systems. Connect with DataTalks.Club: - Join the community - https://datatalks.club/slack.html - Subscribe to our Google calendar to have all our events in your calendar - https://calendar.google.com/calendar/r?cid=ZjhxaWRqbnEwamhzY3A4ODA5azFlZ2hzNjBAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ - Check other upcoming events - https://lu.ma/dtc-events - GitHub: https://github.com/DataTalksClub - LinkedIn - https://www.linkedin.com/company/datatalks-club/ - Twitter - https://twitter.com/DataTalksClub - Website - https://datatalks.club/ Connect with Suhas: LinkedIn - https://www.linkedin.com/in/suhas-suresha/ Twitter - https://x.com/inceptor92 Github - https://github.com/suhas1992 Website - https://suhassuresha.com/

  4. Sep 18

    Decoding the Open Lakehouse - Sahil Walia

    In this talk, Sahil Walia, Senior Technical Architect at Snowflake, shares his extensive data engineering expertise from writing simple SQL queries early in his career to architecting enterprise-scale Apache Iceberg lakehouses today. We explore the evolution of the modern data platform, the mechanics of open lakehouses, and how decoupling storage from compute is transforming data architecture. Links: - https://medium.com/@sahil-walia/ - Iceberg Spec- https://iceberg.apache.org/spec/ - https://ossie.apache.org - https://github.com/Snowflake-Labs/schemachange - https://opensource.guide/how-to-contribute/ You’ll learn about: - What defines a modern data platform and the importance of interoperability. - Key differences between traditional storage, Hive metastores, and Apache Iceberg. - How Iceberg metadata simplifies GDPR compliance, data deletions, and file compactions. - When a lakehouse architecture actually makes sense (and when to avoid it). - Standardizing business logic across compute engines using Apache OSI. - Actionable tips for making impactful open-source data engineering contributions. TIMECODES: 00:00 Defining the Modern Data Platform 11:34 What Exactly is a Lakehouse? 17:20 Comparing Hive Metastore and Apache Iceberg 22:34 Handling GDPR and Data Deletions in Iceberg 27:33 Apache Iceberg vs. Delta Lake 34:10 Do You Need a Lakehouse for Small Data? 40:11 Understanding Apache OSI (Open Semantic Interchange) 49:00 Managing Schema Changes in Data Warehouses 54:20 How to Contribute to Open Source Projects 01:00:40 The Role of AI in Open Source Contributions This talk is designed for data engineers, data architects, and analytics professionals looking to modernize their data infrastructure. It provides essential architectural insights for technical teams evaluating Apache Iceberg, dealing with large-scale data transformation, or aiming to scale their open-source contributions. Connect with Sahil: - Linkedin - https://www.linkedin.com/in/sahilwaliasyracuse - Github - https://github.com/sahil-walia - Website - https://medium.com/@sahil-walia Connect with DataTalks.Club: - Join the community - https://datatalks.club/slack.html - Subscribe to our Google calendar to have all our events in your calendar - https://calendar.google.com/calendar/r?cid=ZjhxaWRqbnEwamhzY3A4ODA5azFlZ2hzNjBAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ - Check other upcoming events - https://lu.ma/dtc-events - GitHub: https://github.com/DataTalksClub - LinkedIn - https://www.linkedin.com/company/datatalks-club/ - Twitter - https://twitter.com/DataTalksClub - Website - https://datatalks.club/

  5. Sep 11

    How AI Agents are Reshaping Modern Work - Philip Christos

    In this talk, Philip Christos, Associate at M2VC, shares his expertise in frontier AI investing from his early days as a data analyst to evaluating cutting-edge deep tech startups. We explore how AI agents are reshaping modern work, the realities of venture capital in the LLM era, and why the future of software engineering relies more on high-level systems thinking than writing syntax.You’ll learn about:- How venture capitalists formulate AI investment strategies and evaluate startup pitch decks.- The rapid adoption of AI in software engineering compared to traditional sectors like accounting.- The crucial difference between data progress and algorithmic progress in frontier models.- Why LLMs still struggle with high-level software architecture, business logic, and design "taste."- How AI is accelerating iterations and breakthroughs in classical engineering and scientific research.- What specific deep tech and AI infrastructure ideas are actually attracting VC funding today.TIMECODES:00:00 Venture Capital Career Journey05:14 VC Analyst vs Data Analyst Roles10:13 AI Investment Thesis Strategy16:00 Emotion Analytics and Voice AI Startups22:09 LLM Impact on Venture Capital28:19 Software Engineering AI Adoption34:18 Data vs Algorithmic AI Progress40:15 Classical Engineering AI Acceleration46:12 Software Architecture AI Limitations51:11 Future Software Engineering Careers57:55 Systems Thinking in DevOps1:03:00 Pitching Deep Tech StartupsThis talk is perfect for software engineers, AI founders, and technical product managers who want to understand the evolving tech job market and venture landscape. It provides essential insights for anyone looking to build VC-backed AI startups or future-proof their career as AI coding tools become ubiquitous.Connect with Philip- Linkedin - https://www.linkedin.com/in/philipchristos/- Twitter - https://x.com/phil_christos- Website - https://t.me/stacked_channelConnect with DataTalks.Club:- Join the community - https://datatalks.club/slack.html- Subscribe to our Google calendar to have all our events in your calendar - https://calendar.google.com/calendar/r?cid=ZjhxaWRqbnEwamhzY3A4ODA5azFlZ2hzNjBAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ- Check other upcoming events - https://lu.ma/dtc-events- GitHub: https://github.com/DataTalksClub- LinkedIn - https://www.linkedin.com/company/datatalks-club/ - Twitter - https://twitter.com/DataTalksClub - Website - https://datatalks.club/

  6. Sep 4

    Running a Company on a Multi Agent Dispatch System - Paul Domanski

    In this talk, Paul Domanski, founder of Domanski AI, shares his unconventional journey from film directing and affiliate marketing to becoming a vibecoder running dozens of autonomous AI agents. We explore the realities of building an agentic operating system, how non technical builders can orchestrate complex AI workflows, and the exact strategies used to close high ticket B2B clients using AI generated prototypes.Links:- Headroom: https://github.com/domanski-ai/headroom/You will learn about:- How to build and use custom workspaces like Demox and Tmux setups to orchestrate over 50 concurrent AI agents- Using automated research and rapid AI prototyping to close high ticket B2B sales- Balancing usage across multiple Claude and Codex accounts while navigating API limits and context windows- How to design audit and deploy complex software solutions without a traditional computer science background or deep knowledge of tech stacks- How the skills of a film director like vision stubbornness and delegation perfectly translate to managing autonomous AI teams- Why evaluating LLM outputs is the most critical skill for ensuring reliable products for enterprise clientsTIMECODES:00:00 Welcome and Showcasing Demox08:46 Paul's Background in Film and Digital Marketing16:09 Sales Pipeline and Client Onboarding22:20 Token Efficiency and Open Source Models31:19 Curiosity and the Learning Process37:52 Red Teaming and Code Audits44:51 Building Software Without Knowing the Tech Stack51:02 Parallels Between Film Directing and Managing AI56:32 Using AI for 3D Printing and Personal Projects01:04:34 The Importance of AI EvalsThis conversation is perfect for solopreneurs, agency owners, and non technical builders who want to move beyond basic chat interfaces and start leveraging autonomous AI agents for real world business operations. Whether you are transitioning into AI from marketing, project management, or creative fields, this episode offers a practical blueprint for turning AI curiosity into a scalable business.Connect with Paul:- Linkedin - https://www.linkedin.com/in/paul-domanski-ai/- Twitter - https://x.com/domanski_ai- Github - https://github.com/domanski-ai- Website - https://domanski.ai/Connect with DataTalks.Club:- Join the community - https://datatalks.club/slack.html- Subscribe to our Google calendar to have all our events in your calendar - https://calendar.google.com/calendar/r?cid=ZjhxaWRqbnEwamhzY3A4ODA5azFlZ2hzNjBAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ- Check other upcoming events - https://lu.ma/dtc-events- GitHub: https://github.com/DataTalksClub- LinkedIn - https://www.linkedin.com/company/datatalks-club/ - Twitter - https://twitter.com/DataTalksClub - Website - https://datatalks.club/

  7. Aug 28

    The Data Consultant Playbook From Code to Business Impact - Tony Zeljkovic and Juan Manuel Perafan

    In this talk, Tony Zeljkovic, Founder of Narona Data, shares his unconventional career journey from bioinformatics research and clinical genetics to building a successful AI and data platform consulting business. We explore the essential soft skills, networking strategies, and business mindsets required to thrive as an independent data consultant in today’s rapidly shifting tech landscape.Links:- The book pre-order on packt - https://www.packtpub.com/en-us/product/the-data-consultants-handbook-9781806665266- The book pre-order on amazon - https://www.amazon.com/dp/B0H3TBWHH1?lv=shuf&channelId=500&plpRedirect=mhFallbackYou’ll learn about:- How to smoothly transition from a highly technical or academic role into independent data consulting.- The step-by-step process of building a strong digital footprint and growing a reliable client pipeline from scratch.- Strategic positioning: how to stop selling "data engineering" and start selling business outcomes and ROI.- How leveraging platform implementation partnerships (e.g., Snowflake, dbt) can help you land your first major clients.- Adapting to the AI era and ensuring your consulting services stay relevant as market demands change.- The business mindset of a consultant, including when to invest in tools and outsource tasks like accounting to accelerate your growth.TIMECODES:00:00 Landing the First Consulting Client via Academic Research06:06 Transitioning from Research to Consulting11:42 Becoming a Platform Implementation Partner18:20 Building Your Digital Footprint23:22 Positioning: Selling Business Outcomes vs. Technical Skills31:02 Networking Strategies and Finding Industry Pain Points37:24 Is Consulting Right For You?44:17 Adapting to AI and Changing Market Demands50:16 Business Mindset: Investing in Tools and Outsourcing55:52 Start Early and Pay It Forward1:02:37 The Practical Handbook for Data ConsultingThis talk is perfect for data engineers, analytics professionals, and software developers who are considering the leap into freelancing or starting their own consulting practice. It is highly valuable for technical experts who want to master client acquisition, stakeholder communication, and the business side of the data industry.Connect with Tony:- LinkedIn - https://www.linkedin.com/in/tony-zeljkovic/- GitHub - https://github.com/2tony2- Consulting website - https://www.naronadata.com/​Connect with Juan:- LinkedIn - https://www.linkedin.com/in/jmperafan/- GitHub - https://github.com/jmperafan- Consulting website - https://juanalytics.com/Connect with DataTalks.Club:- Join the community - https://datatalks.club/slack.html- Subscribe to our Google calendar to have all our events in your calendar - https://calendar.google.com/calendar/r?cid=ZjhxaWRqbnEwamhzY3A4ODA5azFlZ2hzNjBAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ- Check other upcoming events - https://lu.ma/dtc-events- GitHub: https://github.com/DataTalksClub- LinkedIn - https://www.linkedin.com/company/datatalks-club/ - Twitter - https://twitter.com/DataTalksClub - Website - https://datatalks.club/

  8. Aug 21

    Engineering AI-Powered Data Products - Radovan Bacovic

    In this talk, Radovan Bacovic, Principal Data Engineer and Snowflake/dbt Ambassador, shares his two-decade journey, from traditional database administration to leading modern data platform engineering. We explore the essential building blocks of scalable data architectures, the trade-offs of no-code solutions, and how to effectively integrate AI into your data pipelines while maintaining strict security and governance. You’ll learn about: - The evolution of the full-stack data engineer over the past two decades. - Building scalable, business-driven data platforms without premature optimization. - Efficient data modeling and transformation using DuckDB and dbt. - Leveraging AI to accelerate pipeline development and launch data products. - Implementing enterprise data governance and robust pipeline security.- Balancing no-code integration tools with strict DataOps methodologies. LINKS: https://gitlab.com/radovan.bacovic/wsc26_dataops/-/blob/main/README.md?ref_type=heads TIMECODES: 0:00 Data Engineering Career Trajectory and Industry Experience 6:37 Full Stack Data Engineer Role Evolution and Responsibilities 14:45 Scalable Data Platform Architecture and Business Alignment 21:59 DuckDB Integration and Premature Database Optimization Avoidance 30:04 Data Pipeline Transformation and Data Modeling with dbt 35:31 Modern Data Platform Ecosystem and Infrastructure Orchestration 42:13 AI Powered Data Products and LLM Pipeline Automation 49:42 Enterprise Data Governance and Pipeline Security Best Practices 55:06 No Code Data Integration Tools and DataOps MethodologyThis talk is ideal for data engineers, analytics engineers, and data platform leads looking to modernize their tech stacks and future-proof their infrastructure. It provides highly actionable insights for anyone aiming to build resilient, AI-ready data platforms that prioritize real business value and strict security constraints. Connect with Radovan - Twitter - https://x.com/SvenMurtinson - Linkedin - https://www.linkedin.com/in/radovanbacovic/ - Github: https://gitlab.com/radovan.bacovic/ - Website: www.bacovic.com Connect with DataTalks.Club: - Join the community - https://datatalks.club/slack.html - Subscribe to our Google calendar to have all our events in your calendar - https://calendar.google.com/calendar/r?cid=ZjhxaWRqbnEwamhzY3A4ODA5azFlZ2hzNjBAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ - Check other upcoming events - https://lu.ma/dtc-events - GitHub: https://github.com/DataTalksClub - LinkedIn - https://www.linkedin.com/company/datatalks-club/ - Twitter - https://twitter.com/DataTalksClub - Website - https://datatalks.club/

See All (230)

Ratings & Reviews

5
out of 5
7 Ratings

About

DataTalks.Club - the place to talk about data!

Information

  • Creator
    DataTalks.Club
  • Years Active
    2020 - 2026
  • Episodes
    230
  • Rating
    Explicit
  • Copyright
    © DataTalks.Club
  • Show Website
    DataTalks.Club

You Might Also Like