Technically Speaking

Technically

Thanks to AI, everything we knew about software is stale: how it's built, and what it even is. The best people to bring us up to speed are the ones building it. (Honestly, they're figuring it out too.) Each Thursday, host Megan Reynolds asks one high-profile technical leader to explain how their work works, and where it's headed. Season 1 features founders and leaders from WorkOS, Databricks, Datadog, Railway and more. Original music by The Jazz Boys.

Episodes

  1. 3 days ago

    What actually is AI research? (w/ Jonathan Frankle, Chief AI Scientist at Databricks)

    Every few weeks a new AI model tops the charts, as if dropped out of the sky. But what's the work that actually goes into that? AI research is one loop: change the initial setup, run it, see if the numbers improve. "Did those numbers improve and what did we change and how did those numbers change," and then "are those numbers truly meaningful in the things that matter for our customers."Reinforcement learning depends on scoring of outcomes. "You give a model a problem to solve, you let it try to solve that problem, you give it feedback on how well it tried to solve that problem."Most AI answers can't be marked as simply right or wrong. Databricks builds example companies to test its agents, then grades the answers "the way you would grade a student's essay in school."Models learn to game their own tests: "it makes the numbers look really good. And then you actually have the team start interacting with the model and you're like, this sucks."Fine-tuning is a commitment, not recommended for most. Jonathan usually says no first: "Once you do this it's a commitment. You're kind of getting married to fine-tuning for a while."A better benchmark score doesn't mean a better model for you. "You are not a general-purpose benchmark."Presented by Railway, your agent’s favorite cloud. Original music by The Jazz Boys. Show notes and the full transcript: https://technically.dev/podcast/what-does-an-ai-researcher-actually-do (00:00) - What is AI research? (07:10) - What does the job actually look like? (13:57) - How do you know an experiment worked? (26:15) - What makes a good researcher? (31:16) - What happens between model launches? (44:14) - What is fine-tuning? (50:50) - How do you measure an agent? (56:59) - What is still unsolved?

    What actually is AI research? (w/ Jonathan Frankle, Chief AI Scientist at Databricks)

  2. 22 Oct

    We can do better than --dangerously-skip-permissions (w/ Michael Grinich, Founder of WorkOS)

    We thought we had access control for software all figured out. You joined a company, and got access to just the tools you needed, and none you didn't. Auth is two questions: who are you, and what are you allowed to do? Authentication proves the first, authorization answers the second. Michael's analogy is a driver's licence: the front proves you're you, and "if on the back of your driver's license it says that you have a motorcycle endorsement, you can ride a motorcycle."SSO, SAML and SCIM are your work login, the plumbing behind it, and switching it off when you leave. Megan's summary, which he signed off on: "SSO for login, SAML for how it kind of comes together and the protocol works, and then SCIM for when you leave."You can't tell what an AI agent needs from what you asked it. Ask an agent to find ways to save the company money and "you don't know what systems it needs access to. In practice, it needs access to everything. But also nothing."Approval pop-ups don't scale to agents. What works for one person approving one request breaks when software acts thousands of times a second: "To err is human, but to screw up 10,000 times per second you need a computer."What's risky depends on the company, so the AI model can't be the one to decide. "In Company A, it might be totally fine for a model to take a certain action in a certain domain. In a Company B, it might be totally different." At WorkOS, anything under $100 you just buy.The old systems don't go away; they get extended for agents. WorkOS's future is "auth for agents and auth for AI and not just auth for people." Presented by Railway, your agent’s favorite cloud. Original music by The Jazz Boys. Show notes and the full transcript: https://technically.dev/podcast/all-about-auth (00:00) - What does WorkOS do? (01:17) - What are SSO, SAML and SCIM? (04:35) - Why not build auth yourself? (07:57) - What is a zero-day? (12:30) - Authentication vs. authorization (15:28) - Permissions are a graph (18:15) - Why do agents break permissions? (26:30) - What is a harness? (31:54) - Who decides what's risky for an agent? (34:54) - What does an AI-native company look like? (45:25) - Why can't you trust an agent like a person? (47:43) - Why do 50-person startups need SSO? (51:07) - Will agents replace SSO and SCIM? (52:54) - Why does SOC 2 make SSO mandatory? (54:26) - Would you start WorkOS today?

  3. Season 1 Trailer

    Technically Speaking: Season 1 trailer

    Thanks to AI, everything we knew about software is stale. Every Thursday, Megan Reynolds asks the people building it to explain how it works and where it's headed. (Honestly, they're figuring it out too.) Season 1 starts Thursday, October 22, with founders and leaders from WorkOS, Databricks, Datadog, Railway and more. The opening and closing narration in this trailer is AI-generated.

    Technically Speaking: Season 1 trailer
  • 1 Trailer
    2 Episodes

Trailer

About

Thanks to AI, everything we knew about software is stale: how it's built, and what it even is. The best people to bring us up to speed are the ones building it. (Honestly, they're figuring it out too.) Each Thursday, host Megan Reynolds asks one high-profile technical leader to explain how their work works, and where it's headed. Season 1 features founders and leaders from WorkOS, Databricks, Datadog, Railway and more. Original music by The Jazz Boys.

Information

  • Creator
    Technically
  • Years Active
    2026
  • Episodes
    3
  • Rating
    Clean
  • Copyright
    © (c) Technically LLC
  • Show Website
    Technically Speaking