030: Yes, founders trick themselves. Unless they measure reliability.

When founders analyze discovery interviews, they trick themselves. Entrepreneurs today are like medieval bakers judging an oven by feel and then despairing when the bread comes out burnt or raw. Baking got reliable when thermometers replaced the guesswork, and founders need that same upgrade for their interview analysis. Good news! Entrepreneurs can leverage tools from other fields. Specifically, this episode introduces "content analysis" and a metric called Krippendorff's Alpha, which let founders measure the reliability of their analysis.

This episode also details how Mike measured the reliability of Nascent using Krippendorff's Alpha and the opportunity that it revealed: founders do not have an innate ability to analyze interviews, but it's a skill that founders can learn. Specifically, before training, founders' analysis was essentially random (K-alpha = 0.1). But after founders learned how to use Nascent to analyze interviews, their analysis became reliable (K-alpha = 0.8).

In this episode:

(00:00) An exciting unlock in entrepreneurship

(03:47) Content analysis meets customer discovery

(05:44) Analyzing discovery interviews is a type of data transformation

(06:53) Content analysis and Krippendorff's Alpha to measure the reliability of data transformation

(10:34) Measuring K-alpha for Nascent (the jump from 0.1 to 0.8)

(13:55) Analyzing interviews reliably isn't innate, but a learnable skill

Are you new to the Nascent podcast? Start with Episode 019

Let's connect:

Newsletter: https://nascentstartups.com

Work with Mike: https://nascentidea.com

Miro board with visuals and frameworks: https://miro.com/app/board/uXjVGhU2Xoo=/

LinkedIn: https://www.linkedin.com/in/mikevladimer

YouTube: https://www.youtube.com/@nascentidea

Spotify: https://open.spotify.com/show/5BtFtYF6nVUkLu7d7VVnSY

Apple Podcasts: https://podcasts.apple.com/us/podcast/nascent-startups-podcast/id1728760830

Twitter: https://x.com/NascentIdea

Reach Mike at Podcast@NascentIdea.com

Links and sources:

Nascent Episode 028: Yes, the easiest way to analyze discovery interviews is with the Yardstick of Pain.

Nascent Episode 027: No, don't interpret discovery interviews by feel.

Nascent Episode 022: Yes, search for People in Pain. But they're invisible.

Thinking, Fast and Slow⁠ by Daniel Kahneman

Calculator for Krippendorff's Alpha

Nascent frameworks referenced:

Yardstick of Pain™ — A three-bucket emotional scale (hurting, fine, joyous) that founders apply to categorize the emotions of interviewees. Introduced in Episode 027.

People in Pain™ — The first signal a startup might have a customer. Every customer starts as a Person in Pain. Defined in Episode 022.

ERNY™ (Estimated Revenue Next Year) — A metric for startups with no customers: Estimated Revenue Next Year from the ground up with no customers today, calculated only using discovery interviews. See Episode 019.

Tools from content analysis:

Content analysis — A research technique for making replicable and valid inferences from texts (Klaus Krippendorff). The field that lets founders make reliable inferences from discovery interviews.

Krippendorff's Alpha (K-alpha) — A metric for inter-rater reliability that ranges from 0 (completely random) to 1 (perfect agreement).

Many thanks to Paul Tepper for letting me know about content analysis and providing feedback on how I’m applying it to Nascent.