What does successful AI adoption actually look like inside an organization? For technology leader Jessica Kalinowski, the answer starts with people, not tools. With more than 15 years of experience leading engineering, DevOps, site reliability, and cloud transformations across e-commerce, financial services, and healthcare, Jessica shares how her teams have used automation and AI to improve reliability, accelerate delivery, and reduce operational friction. In this episode of the AI Accelerator Podcast, Jessica explains why organizations should approach AI adoption much like cloud transformation: with a clear strategy, appropriate guardrails, measurable outcomes, and a willingness to learn. She discusses how psychological safety helps teams embrace new technology, why mandating AI usage can backfire, and how involving engineers in decisions creates stronger adoption. Jessica also shares a practical healthcare example: rebuilding clinician onboarding in-house, consolidating fragmented systems, and using AI to help reduce onboarding time from approximately 30 days to as little as 24 hours in some cases. If you're a business leader looking to move beyond AI experimentation and achieve meaningful results, this conversation offers practical lessons on building trust, empowering teams, and making technology serve real business needs. In This Episode, Jessica Kalinowski Discusses How her career evolved from quality assurance and test automation to cloud, DevOps, site reliability engineering, and AI. Why automation should focus on removing work that does not create value. How her organization increased deployment frequency from once or twice a month to more than 120 times a month. Why psychological safety and transparency are essential for successful AI adoption. How to introduce AI tools through pilots, feedback, and practical use cases instead of mandates. Why AI usage statistics, such as the number of agents created, are not meaningful measures of business productivity. How AI-assisted document reviews can reduce repetitive operational work. How the team rebuilt clinician onboarding and reduced the process from approximately 30 days to as little as 24 hours in some cases. What cloud transformation can teach organizations about AI governance, security, and adoption. Why teams need measurable outcomes, appropriate guardrails, and a clear strategy before scaling AI. Chapters 00:00 – Introduction to Jessica Kalinowski 01:38 – From Quality Assurance to AI and Engineering Leadership 02:42 – Building a Culture of Automation and Reliability 04:47 – How the Team Increased Delivery Speed and Reliability 05:12 – Creating Shared Ownership of Security and Reliability 06:30 – Applying AI Beyond Engineering 07:16 – Psychological Safety and Incremental AI Adoption 08:44 – Transparency, Team Feedback, and AI Guardrails 09:58 – What AI Adoption Can Learn From Cloud Transformation 10:36 – Introducing AI Safely in a Healthcare Environment 11:20 – Starting With Practical Tools and Human Oversight 12:16 – Why Mandating AI Usage Can Backfire 13:30 – Rethinking AI Productivity Metrics 15:02 – Measuring Business Outcomes Instead of AI Activity 16:30 – Leadership, Trust, and Retaining Engineering Talent 17:55 – Building a Better Clinician Onboarding Experience 19:00 – Reducing Operational Complexity and Vendor Sprawl 20:00 – Using AI to Improve Onboarding Workflows 21:53 – Building and Launching an In-House Platform 23:26 – Moving Beyond One-Size-Fits-All Enterprise Tools 24:45 – Building, Iterating, and Owning Your Technology 27:46 – Testing and Rolling Out Changes in Stages 29:36 – Practical Advice for Business Leaders 30:25 – Creating an AI Strategy With Measurable Outcomes 31:24 – Final Thoughts on AI Adoption and Engineering Leadership Key Learnings Successful AI adoption starts with people. Technology alone does not create transformation. Teams need transparency, opportunities to provide feedback, and confidence that AI is being introduced to improve their work rather than simply monitor or replace them. Learn from cloud transformation. Organizations that moved to the cloud without planning for architecture, performance, cost, and operational change encountered challenges. AI adoption also requires preparation, governance, and a realistic understanding of how new tools affect existing workflows. Introduce AI incrementally. Start with practical use cases, pilot programs, and tools that solve recognizable problems. Let employees evaluate the results and help shape adoption before expanding across the organization. Measure outcomes, not AI activity. The number of AI agents created or how frequently employees use a tool does not necessarily indicate productivity. Track meaningful measures such as delivery speed, reliability, quality, cost, and the amount of work completed. Give teams a voice in the transformation. Engineers are more likely to embrace AI when they are involved in decisions, understand the reasoning behind them, and can share honest feedback about what is and is not working. Use AI to remove friction from real workflows. Jessica's clinician onboarding example illustrates the potential of combining automation, AI, and process redesign. Bringing fragmented workflows into a more integrated system helped reduce duplication, operational effort, and onboarding time. Governance becomes more important as AI scales. Organizations need visibility into what their AI tools and agents are doing, appropriate security guardrails, and oversight of usage and cost. Not every process is a good candidate for AI. Build around business needs, not vendor limitations. When existing tools do not adequately support a business process, organizations should assess whether a tailored solution makes sense. The decision should be grounded in the problem, feasibility, and long-term value. 💬 Jessica Kalinowski's Key Insights “The number of agents you have spun up in a day is not a productivity measure.” “How can we empower them to use the tools?” “People need a voice.” “Have a strategy. Have a plan. Have measurable outcomes that you're tracking.” These insights capture Jessica's central message: AI adoption is a leadership and organizational challenge as much as a technology challenge. The strongest results come from empowering teams, solving meaningful problems, and measuring what actually improves. About Jessica Kalinowski Jessica Kalinowski is a technology leader with more than 15 years of experience building and scaling engineering organizations, leading cloud transformations, and delivering measurable business outcomes. Her experience spans e-commerce, financial services, and healthcare, with a focus on DevOps, site reliability engineering, platform transformation, automation, and AI-driven improvements. At ConnectRN, she has helped scale engineering capabilities, advance CI/CD practices, improve platform reliability, and increase developer productivity. Previously, she led large-scale SRE and cloud initiatives at Chewy and MassMutual, delivering cost savings while strengthening system resilience and developer experience. Jessica is passionate about empowering teams, simplifying complexity through automation, and aligning technology strategy with business value. Follow Jessica Kalinowski LinkedIn: https://www.linkedin.com/in/jessica-kalinowski-4159636b/ Follow Matt Zembruski Website: https://leadingaiagility.com LinkedIn: https://www.linkedin.com/in/mattzembruski/ Email: matt@leadingaiagility.com Phone / Text / WhatsApp: +1 978-618-5778