Human in the Loop: Two Ex-Microsoft Builders on AI

Chris Shanku
  • 0.0 (0)
  • Technology
  • Updated Weekly

Chris Shanku and Sash Mohapatra - two ex-Microsoft builders - demo one real AI tool per episode, live and unedited. Real people. Real tools. No retakes.

  1. 4d ago

    Why Your Agent Rollout Stalls: 3 Pillars, Not One

    Sash drove this one, and it was a conversation rather than a demo: why so many enterprise AI pilots stall, and why the answer is almost never the model. His framing comes straight out of the ITIL discipline he and Chris both carried at Microsoft, updated for agents - people, data, and agent ops. Miss any one of the three and the other two stop mattering. He anchored it on BCG's latest numbers: 82% of CEOs are more optimistic about AI gains than they were last year, while only 6% of companies report meaningful value. The 6% are the ones working all three pillars at once. On people, Sash argued for three tiers rather than a single training push - baseline AI fluency for everybody, an orchestration layer of the people who actually own operational processes, and a small group of stewards who carry the message. Chris pushed on why this is the pillar that fails quietly: teams treat an agent like a new hire who is not allowed to ask questions and whose work goes straight to a customer unread. Generic training does not fix that. Training on your own workflows, with the requirement that every AI-assisted result be explainable, does. On data, the number that landed was 7% - the share of organizations that believe their data is ready for AI - and the practical advice was deliberately small: pick one workflow, find the data sources it needs, go to the owners, fix and document the lineage for that one path, then reuse the process. If an answer cannot be traced to a source, the agent does not ship. Agent ops got the most concrete treatment. Map a workflow step by step before deciding where AI belongs, and leave the steps that code already handles deterministically alone rather than forcing tokens at them. Put a human checkpoint ahead of anything touching finance, external sending, or a system of record. Give every agent a named owner and a traceable action log, and stand up the eval layer before the agent count grows, not after - Sash's point being that without tracing, agents quietly atrophy as models change underneath them. Chris connected that to where the industry is heading: Mastercard and Visa building a Know Your Agent framework, Mastercard expecting one in ten purchases to be agent-made by 2030, and Microsoft Scout putting an agent into Entra ID as a named identity that inherits DLP and compliance. Outside the enterprise, as Chris put it, there is still no way to prove your agent is yours. Key takeaways: Pick one workflow before touching the data estate. Identify its sources, find the owners, fix and document the lineage for that single path, then reuse the process on the next one.Do not ship an agent on data whose answers cannot be traced back to a source. If the trace is missing, you will not be able to reconstruct what went wrong.Put a human checkpoint in front of any agent action that touches finance, external sending, or a system of record.Give every agent a named owner and a traceable action log, and stand the eval layer up before the agent count grows. Agent behavior shifts as models change underneath it.Find the 5 to 10% of people already building without waiting for permission and make them your stewards. Train on your own workflows, not generic courses, and require that every AI-assisted result be explainable.Leave the steps that code already handles deterministically alone. Not every automation needs a model.Watch the video version on YouTube · Full show notes, transcript + takeaways

  2. Sep 9

    Quit Babysitting Your Terminals

    Chris has spent months trying to get his agents to talk to each other, and this episode is the honest tour of three attempts: what broke, what held, and what he actually runs now. He frames it against four trust levers borrowed from how a person climbs through the ranks - human IS the loop, human IN the loop, human ON the loop, human OUT of the loop - and pins the current shift to Gary Vaynerchuk's 15/80/5 model: own the first 15 percent of an initiative, get out of the way for the 80 in the middle, come back for the last 5. Attempt one was AgentMail, a webhook service that bills itself as Gmail for agents. Silas, Chris's Hermes chief-of-staff agent, had his own inbox and would email Neo, the coder agent, to hand off work. It worked on vibes. Conversations died silently after about the third message, and worse, they died without telling Chris - the failure mode every manager recognizes. Sash draws the distinction that explains it: these are separate Hermes profiles, peers rather than sub-agents under one parent, so there is no hierarchy to carry the handshake. Attempt two, Hermes's Kanban feature, fixed exactly that. Kanban is really a wake-up mechanism with a native acknowledgement handshake: the orchestrator posts a task to another profile's board, that profile gets a heartbeat, picks it up, and links its subtask back. Chris runs it autonomously at 4 AM against a backlog board, with a Postgres ledger tracking the traffic. It still runs, and it still only works between Hermes profiles on one gateway. The gap that Kanban could not close is that Claude Code is Chris's real building harness and it cannot clear its own context window. His SDLC runs the superpowers plugin - brainstorming on Sonnet, design on Opus, writing plans on Sonnet, then the build loop on Opus or Fable - and every handoff between those steps needed a human to clear context, relaunch, and switch models. That is the thing that killed autonomy. Grove Rooms are the answer: a dedicated tmux session, synced to the Postgres ledger, that Silas spins up and then drives. Silas watches for the spec file to be committed, types /exit, relaunches Claude Code in the same session on the right model, and kicks off the next skill, unattended. Chris answers the brainstorming questions, walks away, gets a notice when the room moves from plan to build, and comes back to review and run close. The next move he talks himself into on air is cross-harness: have Silas hand Claude Code's finished work to a Codex-backed agent for review before it ever reaches him, and eventually replace the whole arrangement with the native A2A protocol that Hermes, ElevenLabs, Copilot Studio, and Foundry Agents already speak. Key takeaways: Separate agent profiles are peers, not sub-agents. If your orchestration depends on a parent-child hierarchy that does not exist, messages will die silently. Check whether your transport has a real acknowledgement handshake before you trust it with unattended work.A ledger is what makes stepping back survivable. Chris tracks every agent-to-agent transition in Postgres, so oversight is a lookup rather than an argument about what happened overnight.The bottleneck in autonomous building is usually context, not capability. Claude Code cannot clear its own context window, so any pipeline that spans multiple skills needs something outside the session to relaunch it.Run different models for different phases. Brainstorming on Sonnet, design on Opus, planning on Sonnet, build on Opus or Fable - and let the orchestrator do the switching so the phase boundary costs you nothing.Apply 15/80/5 to a build pipeline literally: answer the brainstorming interview, leave, and come back for review and close. If you cannot name your first 15 percent and your last 5, you are not delegating, you are just hoping.Watch the video version on YouTube · Full show notes, transcript + takeaways

  3. Sep 2

    Copilot Cowork: Enterprise Breakdown

    Eric Brown, a Solution Engineer at Microsoft focused on Copilot and agents, joined Chris and Sash to walk through Copilot Cowork the way he demos it for enterprise customers. The framing up front: the bottleneck in most organizations was never a lack of knowledge, it is coordination across email, meetings, chat, and scattered documents - and Cowork is built to act as an agentic teammate inside that mess rather than another place to search. Eric ran a live screen share showing the difference between Copilot Chat (single question, single response) and Cowork (connects to tools and data, reasons, and executes multi-step work). A big piece of what makes that possible inside Microsoft 365 is Work IQ - the graph layer that already knows a user's files, meetings, emails, and chats, now paired with an LLM that can reason over it. Eric was clear that Cowork's access is bounded by the same permissions the user already has in Exchange, SharePoint, and OneDrive - "your digital assistant only has access to the items that you have access to" - and described the governance posture as wanting to build "a casino, not a prison," enabling exploration inside real guardrails rather than locking things down entirely. The back half moved into automations: Eric used voice to ask Copilot to pull meeting transcripts off his calendar and generate a recurring daily digest, live, showing the step-by-step planning Cowork does before it commits to a scheduled task. Chris connected it to his own "orchestrator method" - define the end state with clarity first, then let the agent work backward into steps - and the conversation was headed into how Microsoft 365's credit system prices this kind of agentic work when the recording cut short. Key takeaways: Copilot Cowork's real differentiator over Copilot Chat is that it acts on tasks (connects to tools, plans, executes) rather than just answering a single question - if you are still only using it for one-off Q&A, you're leaving the agentic half on the table.Enterprise Copilot's data access is bounded by the user's own existing permissions (Exchange/SharePoint/OneDrive) - it doesn't grant new access, so oversharing risk is closer to "existing permission hygiene" than "AI-specific risk."Voice input plus Cowork's task/automation builder can turn a recurring manual chore (pulling meeting transcripts into a summary) into a standing background job with a few spoken sentences - worth trying on your own most-repeated weekly task first.Start with "auto" model selection in Copilot if you're not sure which underlying model fits your task; Copilot will route it for you.Define the end state before you prompt for the task - Chris's orchestrator framing (clarity on the output shape first, then the steps) mapped directly onto how Eric was structuring his own Cowork requests.Watch the video version on YouTube · Full show notes, transcript + takeaways

  4. Aug 26

    Stop Running Agents Without Shared Memory

    Sash drives this one, and he does not start with the tool. He starts with the failure. He was running Claude Code across two Macs, so his first move was the obvious one: put his intelligence folder of Markdown context files in a git repo and sync it to both machines. That worked until it did not. Every session ended with a push he had to remember, and every session started with a pull he had to remember, and forgetting either one meant an agent was working from yesterday's truth. Then he added a Hermes agent on a Hostinger VPS, then revived an old OpenClaw agent on another one, and suddenly there were four agents and no shared understanding between them. The fix was not better discipline, it was a different layer. Sash looked at Obsidian vaults and Karpathy's LLM Wiki before landing on gbrain, the open-source project from Y Combinator's Garry Tan, which Tan describes on the repo as "the production brain behind my OpenClaw and Hermes deployments." gbrain sits on top of the git repo as a semantic layer: it chunks the Markdown, indexes it into Postgres with pgvector, and exposes an MCP endpoint agents can query. The git repo stays the durable source of truth and stays human-editable. What changes is retrieval. Instead of loading CLAUDE.md plus status.md plus decisions.md into every context window, an agent asks gbrain for the chunk that matches the task and pulls more only if it needs it. That is the moment Chris stops the demo to repeat it back, because just-in-time context is the part that actually changes a working day. The second half is the part most people skip, which is access. Sash runs three agents against one brain with deliberately different permissions: Nuro is the only thing with write access to the git repo, Kyra has full read/write on gbrain but read-only on the repo, and Blaze can only read the course-related slice and cannot see client work at all. His framing is that this is not novel, it is just how you build an IT team - help desk does not get admin on the database, engineers do. Sash closes with the smallest version worth doing: if you run one agent and nothing always-on, hand the gbrain repo to Claude Code or Codex, point it at the files you already work in, and describe the thing your agent keeps forgetting. Key takeaways: Put the durable source of truth in a git repo of plain Markdown you can still open and edit by hand, then treat gbrain as an index on top of it rather than a replacement for it.Stop loading whole context files into every session. Wire a hook so the agent queries the brain for the chunk the task needs, and let it pull the full page only when the chunk is not enough.Separate the three things an agentic system needs: context lives in the brain, canonical instructions stay in CLAUDE.md or AGENTS.md, and tools are wired per agent.Decide read versus write per agent before you connect the second one. Give exactly one agent write access to the source repo and scope every other agent to the slice it actually needs.If you are starting cold, do not install it by hand. Give the gbrain repo URL to Claude Code or Codex, tell it what your agent keeps forgetting, and let it work out the setup and the sync job.Watch the video version on YouTube · Full show notes, transcript + takeaways

  5. Aug 19

    AI for Enterprise: Stop Starting with AI

    Kevin Johnson - KJ - leads the AI and automation practice at Pinnacle Financial Partners, a bank that closed a merger of equals in January between two institutions north of $60 billion each. He has been doing this hand-to-hand since ChatGPT launched in November 2022. Chris and Sash brought him on to answer the question every enterprise team is quietly stuck on: what actually has to be true before generative AI works in production. His answer is that almost none of it is about AI. Pinnacle's ability to move fast on generative AI traces back five or six years, to a modernization journey that ripped workloads out of a traditional data center, rebuilt the data and networking and security layers, and reorganized teams into pods around the new work. That unglamorous foundation is the reason the AI work has anywhere to land. KJ's sharper point is that the same modernization has to happen to people, and that half gets far less attention: reinventing yourself as a contributor after two decades of doing the job one way is hard, and the organizations that win will be the ones that move fluidly through that change. As he put it, that is as human as it gets. The back half is about discipline. Value in generative AI is genuinely hard to measure, and KJ is blunt that if you are not already measuring your KPIs well, layering AI on top makes measurement harder rather than easier. He and Sash land in the same place on tooling: automation and AI are two different tools in the toolbox, deterministic solutions never actually died, and you do not need a cannon or a bazooka for a mosquito. Chris connects that back to skills as the middle ground - a way to get an agent to follow the same process every time, even when the wording varies - and notes how often his skills end up becoming automations. The advice KJ would give his 2023 self is not technical at all: get real leadership buy-in, make continuous innovation part of the DNA, and meet the organization where it actually is. Key takeaways: Do the modernization work first. Cloud, data, networking, and security foundations are what make a generative AI program possible, and skipping them is the same mistake teams made pushing BI projects ahead of data cleansing.Budget for team member modernization, not just technology modernization. The hard part is people reinventing how they work, and that is a change management problem rather than a tech problem.Fix your measurement before you add AI. If your KPIs are not instrumented well today, proving generative AI value on top of them gets harder, not easier.Match the tool to the problem. Deterministic automation and RPA still solve plenty of things more cheaply and more reliably than a frontier model, and orchestration plus observability matter more than horsepower.Use skills to put determinism back into agent work. Define the pattern once so the process runs the same way every time, and expect the good ones to graduate into automations.Ship tools that make someone's day measurably easier and adoption becomes organic. Finding an answer in ten seconds instead of two and a half minutes is the kind of win a frontline team feels immediately.Watch the video version on YouTube · Full show notes, transcript + takeaways

  6. Aug 14 ·  Bonus

    Here Comes Watermarking!

    Anthropic started watermarking every piece of text Claude produces, worldwide, three days before this episode and Sash and Chris jumped on an unplanned morning show to work through what it actually does. The watermark has two parts: an invisible mark woven into the text itself, and C2PA metadata (the same provenance standard Microsoft and Google use) attached to generated artifacts like images and documents. It is not opt-in - Anthropic is required to ship it everywhere under Article 50 of the EU AI Act, the same transparency requirement that is starting to do for AI content what GDPR did for data. The conversation kept circling one unresolved question: the mark does not distinguish between text Claude wrote from scratch and text a human wrote and ran through Claude for a light edit, a fact-check, or a translation. Both cases get flagged the same way. Sash pointed out the mark also strips cleanly the moment you run the output through a second model to rephrase it, which means the ambiguity cuts both directions - it can wrongly implicate a human editor, and it is trivially defeated by anyone trying to hide real AI generation. Anthropic says a public text-detection API is coming, which is the piece that would actually make this enforceable rather than theoretical. Both hosts landed on the same personal practice out of this: get in front of it and disclose AI use yourself instead of waiting to be caught by a detector. Sash already footnotes every newsletter issue as written with Claude Code; Chris said the episode convinced him to start doing the same with his own AI-generated text going forward. Key takeaways: If you publish AI-assisted writing, disclose it yourself in the footer or byline rather than waiting for a detection tool to flag it later - both hosts already do this and treat it as the safest default under the new mark.Don't assume a light-touch AI pass (grammar check, fact-check, translation) is exempt from the watermark - the mark applies to anything that passed through Claude, not just fully AI-authored text, so budget for that before you rely on it staying invisible.Know that the watermark is currently defeated by running Claude's output through a second model to rephrase it - if you or your org are planning to lean on detection for policy or trust decisions, that gap matters today.Watch for Anthropic's promised public text-detection API - that is the step that turns "a mark exists" into "someone can actually check for it," and it is not live yet.Watch the video version on YouTube · Full show notes, transcript + takeaways

  7. Aug 12

    3 Lessons from Blowing up my AgentOS

    Rene Charbonneau returns three months after EP.009 to report on the promise he made on his way out the door: refactor his AgentOS. He didn't refactor it. He blew it up. What started as a two-week attempt to untangle a ball of yarn he and Claude had built together turned into a full rebuild once he realized the old assumptions (a single hand-built harness, a builder-first workflow, memory scattered across machines) no longer held. The teardown included literally retiring his always-on agent, Jim, mid-project, and returning to rebuild the connection layer (SSH, approvals, skills) from scratch on a fresh machine profile. The conversation centers on what Rene replaced the old setup with: less building, more configuring; a shift from Claude Code alone to a Hermes/OpenClaw split, backed by an LLM Wiki-style agentic memory layer (Y Combinator's G-brain) synced read-only across multiple machines so every agent works from one source of truth. He also dropped his Claude subscription from $200 to $20/month, using Hermes and OpenClaw against OpenAI instead, partly for cost, partly because routing agent traffic that way violates Anthropic's terms of use, and partly because watching Fable get geo-restricted in Canada made vendor lock-in feel like a real risk rather than a hypothetical one. The back half zooms out to where this goes for non-technical users: Microsoft Scout and rocket-branded turnkey "agent PCs" (Auto, VioLoop) as the consumer packaging layer for the same OpenClaw/Hermes stack the hosts run by hand, POKE as a prepackaged agent you negotiate with over iMessage, and a BYOA ("bring your own agent") framing for how enterprise hardware procurement might look in five years. Chris teases next week's guest, Kevin Johnson of Pinnacle Financial, who will cover the enterprise version of this same journey: foundation-first before AI, not AI bolted onto ungoverned data. Key takeaways: If something about your agent setup keeps nagging at you ("is this friction normal?"), treat it as signal - spin up a parallel instance with a stock model and no custom harness, and compare results before assuming your custom setup is still worth its context budget.When you retire an agent or harness, don't delete its memory - archive it as a tombstoned "V1," ask it what it wants carried forward into "V2," and start clean from there.A shared, synced memory layer (wiki-style, read-only mirrored across machines) matters more than which harness you pick - organize it so every agent can actually find what it needs, not just access it.Watch for vendor lock-in risk directly, not just cost: geographic restrictions (Fable's brief US-only limitation) and terms-of-use constraints (routing autonomous-harness traffic through Anthropic directly) are real reasons to keep a second provider live.Scope what an always-on agent can do to itself - Rene's agent needed a human to step in and grant runtime access precisely because it was walled off from troubleshooting its own failures, which is what kept a bad actor or prompt injection from having free rein.Watch the video version on YouTube · Full show notes, transcript + takeaways

  8. Aug 5

    Stop Doing Video Production Manually

    Chris opened the show with roughly thirty seconds of himself that was not himself. The voice, the face, and the script were all generated, and the reveal was the point: the pipeline that produced that cold open is the same one that produces the trailer, the highlight reel, and the product demos. He then kicked off two real jobs live on stream, the EP.020 trailer and an iLink product demo, and let them run in the background while he and Sash talked through how the machinery works. The shape of the pipeline is the same everywhere Chris uses it. A research skill finds material inside a defined niche. A script skill drafts it, grounded in enough of his own writing that the draft comes back sounding like him rather than like a model. A critic agent checks the draft against the storytelling rules he cares about and sends it to Telegram for approval. Only after that gate does a produce skill start spending money, calling ElevenLabs for voice and rendering images. A release skill then schedules the finished video to YouTube against a defined cadence, cuts the Shorts, builds the Facebook and Instagram posts and reels, and pushes the audio to the podcast RSS feed. Nothing in the orchestrating skills holds logic; they call subskills, so a single stage can be changed without touching the rest. Sash's framing for all of it was deconstruct and delegate. His argument was that a complex multi-step task should never be one pipeline, and that the three things that decide whether long-running agent work succeeds are context, instructions, and tools, set up per piece rather than once globally. He tied it back to spec-driven development: nobody writes production software without planning the user journey and the features first, and agent pipelines deserve the same treatment. The honest moment came at 29:21, when the product demo Chris had kicked off did not finish end to end despite testing clean three times beforehand. Nobody cut away. He showed a previously rendered demo instead and moved on, which is roughly the whole thesis of the show. Key takeaways: Put the human gate where the money starts. Chris approves the script on Telegram before the produce skill runs, because everything upstream of that point is free and everything downstream costs real API spend.Keep orchestrating skills free of logic. Have them call subskills instead, so you can change the voice step without editing the whole pipeline or hoping the model does not break something adjacent.Track cost per minute of finished video as your KPI. Chris lands around 50 cents per minute for a story and a little over a dollar per minute for a product demo, counting images, voice, digital twin, and music.Deconstruct before you automate. Write down the whole pipeline from research to release first, then turn each phase into its own skill with its own context, instructions, and tools.Taste is not a prompt, it is time. The way to get output that matches your creative vision is to iterate and review repeatedly, and to define what good looks like in words the machine can act on.Watch the video version on YouTube · Full show notes, transcript + takeaways

See All (25)

About

Chris Shanku and Sash Mohapatra - two ex-Microsoft builders - demo one real AI tool per episode, live and unedited. Real people. Real tools. No retakes.

Information