The AI Kubernetes Show

The AI Kubernetes Show

The Kubernetes AI Show dives deep into the real-world challenges of adopting AI on Kubernetes platforms.

  1. LLM Inference on Kubernetes: New Primitives, Real Challenges

    −4 d

    LLM Inference on Kubernetes: New Primitives, Real Challenges

    Running LLM inference on Kubernetes requires new primitives for routing, autoscaling, and GPU scheduling. Here's what platform engineers need to know.

    44 min
  2. Running Multi-agent AI on Kubernetes: Lessons from Imagine Learning

    17 juni

    Running Multi-agent AI on Kubernetes: Lessons from Imagine Learning

    In this episode of The AI Kubernetes Show, Blake Romano, Staff Software Engineer at Imagine Learning, walks through what it actually looks like to build and run AI agents on Kubernetes at scale. He talks about the architecture choices, the failures, and why the organizational context you bring to the LLM matters more than which Software Development Kit (SDK) you use. Imagine Learning is a K-12 education company building digital platforms for students and educators, and Blake has been driving AI and platform engineering initiatives there.

    48 min
  3. AI Agents Security: Guardrails & Production on Kubernetes

    3 juni

    AI Agents Security: Guardrails & Production on Kubernetes

    Learn platform-level security patterns for AI agents on Kubernetes. Close the production gap with LLM guardrails, tool filtering, and short-lived tokens.

    42 min
  4. Platform Engineering & Kubernetes: Guardrails For AI Code

    20 maj

    Platform Engineering & Kubernetes: Guardrails For AI Code

    Learn how Schonfeld scaled their internal AI platform, SchonAI, using Kubernetes and established guardrails to manage AI agent code volume. Build your AI-native workflow now.

    54 min

  5. 6 maj

    One Dependency Away: Supply Chain Security in the Age of AI

    Secure your Kubernetes environment. Learn why zero trust cybersecurity is the only defense against AI agents and non-deterministic agentic software in your supply chain.

    50 min
  6. AI: Bubble or Bug? A CTO’s Perspective on Engineering in the AI Era

    14 jan.

    AI: Bubble or Bug? A CTO’s Perspective on Engineering in the AI Era

    Is the AI boom a bubble, or is it a new technological wave? Dinesh Majrekar, CTO of Civo, breaks down the current state of software development, explains why data sovereignty is the paramount security concern, and details how AI's real value lies in increasing code auality, not just velocity. In this episode of The AI Kubernetes Show, Civo CTO Dinesh Majrekar tackles the AI bubble hype, suggesting it is a blend of market speculation and genuine, disruptive innovation, drawing a comparison to the historical hardware monopoly of IBM during the mainframe era. He dives into the challenge of data sovereignty in the age of large language models, explaining Civo's solution of using an "on-prem public cloud" to run an OpenAI-compatible endpoint on private GPUs. This approach ensures maximum security for sensitive data, like medical records, by guaranteeing the data "never leaving your building." We also discussed the flattening curve of open source LLM capabilities, noting that models like the Kimi K2 model are now matching and even beating proprietary benchmarks while using fewer resources. Majrekar challenges the prevailing focus on speed, arguing the true value for software development teams is in boosting code quality. He champions code generation as the best AI use case but stresses it must be a "partnership" where saved time is reinvested in tackling technical debt and strengthening the code base. This is important for managing deployment risk. Finally, he addresses the dilemma of non-deterministic outputs in deterministic processes, which engineers simply call "a bug," emphasizing that AI is not a universal solution. Read the blog post: www.buoyant.io/ai-kubernetes-episode/ai-bubble-or-bug-a-ctos-perspective-on-engineering-in-the-ai-era Key Takeaways ✓ Code Quality is the true benefit of integrating AI; the time saved on initial generation should be used to fix technical debt and strengthen code. ✓ Achieving true Data Sovereignty requires running LLMs on private infrastructure (e.g., an on-prem public cloud) to keep data securely contained. ✓ The non-deterministic outputs of LLMs can be considered a "bug" in core engineering processes that demand algorithmic certainty. ✓ Code generation is the strongest AI use case, but developers must maintain ownership and set a high context standard for the LLM to follow. ✓ Open source LLM capabilities are now "on par" with proprietary models. Hit the like button and subscribe to The AI Kubernetes Show for more AI content!  What is your engineering team prioritizing with AI: velocity or quality? Let us know in the comments below! #AI #CodeQuality #DataSovereignty #SoftwareDevelopment #PlatformEngineering #Kubernetes #LLM

    27 min
  7. Moving from Single Agents to AI Agent Fleets

    14 jan.

    Moving from Single Agents to AI Agent Fleets

    The future of software development isn't about single agents—it's about building AI agent fleets! Dive into this conversation with Okteto CEO Ramiro Berrelleza to understand how this shift is fundamentally changing platform engineering and accelerating developer productivity.  In this episode of The AI Kubernetes Show, we sat down with Ramiro to discuss AI adoption and the need for constant experimentation in the current "Cambrian explosion" of AI tooling. Berrelleza highlights the move from single-threaded AI tools to large, asynchronous AI agent fleets, which solves the bottleneck of waiting for a single AI response. This agentic model is a game-changer, with some early adopters seeing a massive increase in output. Organizations need to adapt for AI-native workflows, because the focus on traditional metrics like measuring code production (lines of code, number of PRs) for AI is flawed. Instead, organizations should identify and focus their AI projects on their real constraints, such as slow CI workflows. Ramiro also addresses the disproportionate challenge of open source maintainer overload caused by AI-generated contributions, proposing a policy of "human-proof code." Finally, AI agents are presented as a powerful technical context multiplier for everyone from sales engineers to the CEO, significantly speeding up the onboarding process and improving communication across the organization.  Read the blog post:  Takeaways ✓ The future is moving from single-threaded AI tools to "AI agent fleets" to solve productivity bottlenecks. ✓ Traditional metrics like lines of code or PR count are now ineffective for measuring AI-driven developer productivity. ✓ The new focus for AI investment should be on organizational bottlenecks, such as optimizing slow CI workflows. ✓ Open source projects should adopt policies like "human-proof code" to manage maintainer overload from AI contributions. ✓ AI agents can serve as a technical context multiplier, speeding up onboarding and improving organization-wide understanding of complex code. Hit the like button, subscribe for more content on platform engineering and AI, and ring the notification bell.  What is the biggest productivity bottleneck you've solved with AI agents? Let us know in the comments! #AIAgentFleets #PlatformEngineering #DeveloperProductivity #Kubernetes #KubeCon #Okteto #AgenticAI #OpenSource #SoftwareDevelopment #TechTrends

    27 min
  8. Why Testing and Validation are the Unsolved AI Code Challenges

    14 jan.

    Why Testing and Validation are the Unsolved AI Code Challenges

    Is your engineering org ready for the speed of AI? Grant Miller, CEO of Replicated, breaks down the intersection of AI and platform engineering, revealing why testing and validation are the biggest unsolved problems in the industry. In this episode of The AI Kubernetes Show, we sit down with Replicated CEO Grant Miller to discuss how the pace of AI is fundamentally reshaping software development. Miller argues that engineering velocity has become the core competitive differentiator and shares the concept of "leadership empathy," where leaders contribute to a pull request with AI to understand the new tools. This increased velocity, however, puts significant system pressure on platform engineering teams, leading to "Frankenstein-y" application footprints and a greater need for top-notch observability and optimized CI/CD pipelines to improve "iteration speed total." The unique distribution challenges of self-hosted AI applications and the difficulty of validating AI code generation, especially for templated infrastructure-as-code like Helm charts and Terraform. Unlike front-end code, the human validation loop for infrastructure-as-code is not intuitive, making the complexity of testing and validation the industry's most significant hurdle. Read the blog post:  Takeaways ✓ AI turns engineering velocity into the ultimate competitive advantage, requiring organizations to move incredibly fast. ✓ Leaders must develop "leadership empathy" by using AI tools to understand the modern developer experience. ✓ Rapid AI code generation can lead to complex, "Frankenstein-y" application architectures, increasing pressure on platform engineering for troubleshooting and observability. ✓ The biggest challenge in AI-generated code is the lack of an intuitive validation loop for infrastructure-as-code like Helm charts. ✓ Testing and validation are the key unsolved problems and future areas for discovery and job creation. Liked this podcast? Hit the like button, subscribe for more AI and platform engineering insights, and let us know in the comments: What is the biggest challenge your team faces with AI-generated code? #AI #PlatformEngineering #EngineeringVelocity #AIGeneratedCode #TestingAndValidation #Kubernetes #Replicated #TechPodcast #CloudNative

    27 min
Visa alla (24)

Om

The Kubernetes AI Show dives deep into the real-world challenges of adopting AI on Kubernetes platforms.

Information

  • Skapad av
    The AI Kubernetes Show
  • Aktiva år
    2 tn
  • Avsnitt
    24
  • Åldersgräns
    Barnvänligt
  • Copyright
    © The AI Kubernetes Show
  • Programmets webbplats
    The AI Kubernetes Show