Summary When H.R. 1 put new pressure on states to cut SNAP payment error rates, Nava PBC had six weeks to help Pennsylvania stand up a way to catch bad document uploads before they reached a caseworker. That rapid-response build became Document AI, an open-source service that checks whether an uploaded document is legible and the right type, then extracts key fields for review. Product manager Sophia Philip and senior software engineer Laurence Goolsby of Nava Labs join Ryan to talk about the project, which has since been adapted for a Maryland pilot that's set to scale statewide by December. They cover: - Why the tool is deliberately kept out of eligibility decisions, with no policy coded into it- How applicants can opt in, opt out, or proceed anyway when a document is flagged- How it handles gig-work and handwritten invoices, plus Spanish-language documents- Why a confidence score is not the same as accuracy- How they're approaching AWS dependence and data security- How other agencies can fork the repo and adapt it to their own needs This episode is sponsored by Nava PBC. Keywords Document AI, intelligent document processing, Nava PBC, Nava Labs, open source, SNAP, H.R. 1, payment error rates, Maryland, Pennsylvania, benefits delivery, administrative burden, human-in-the-loop, consent, Amazon Bedrock, government technology, civic tech Key Topics - How AI-assisted prototyping ("demos over memos") is changing collaboration between product and engineering- Document processing as a shared problem across benefits, licensing, permits, and more- Building a six-week rapid response to H.R. 1 in Pennsylvania, then adapting it for Maryland- Why open source: avoiding vendor lock-in while keeping PII inside each agency's environment- The three users: applicants, caseworkers, and the people who administer the service- Consent that is plain-language, just-in-time, and reversible- Supporting nonstandard documents like handwritten invoices from gig and informal work- Guarding against caseworker over-reliance: confidence scores, and no determinations made by the tool- Under the hood: Amazon Bedrock and Bedrock Data Automation, plain-language document schemas, and Textract- Meeting states on the infrastructure they already use, and the path to being cloud-agnostic- Security by default: data that never leaves the agency, and extracted values that aren't returned unless requested- How to contribute, deploy, or request demo access Sound Bites - "Confidence does not mean accuracy."- "Fundamentals are the building blocks of fun."- "Document AI isn't making any decisions about whether the document should be uploaded. That lives with the person who's uploading it."- "It's not the person who uploads it's fault that we haven't defined it."- "You are not in a vacuum." Chapters 00:00 Intro 00:21 Meet Sophia Philip and Laurence Goolsby of Nava Labs 00:49 Their personal "why" 02:16 What's changed, and what hasn't, in product management and engineering 05:59 How AI prototyping changes collaboration across practices 08:33 Why document processing is a problem worth solving 12:38 Designing around burdens imposed by policy, and building reversible consent 15:15 What is Document AI? 16:26 Origin story: a six-week rapid response to H.R. 1 in Pennsylvania 20:10 Did it move payment error rates? 21:21 What "open source" means here, and why it matters 24:25 The three user personas 26:23 The applicant experience and opt-out fallback 28:41 The caseworker experience, gig-work invoices, and multilingual documents 33:26 Avoiding rubber-stamping: human-in-the-loop and no automated determinations 37:36 Shaping the product vision, from Postman demos to a management layer 41:32 Handling surprises: agile delivery and the Maryland pilot timeline 46:04 Under the hood: how Document AI uses AI 50:12 Why confidence isn't accuracy 51:13 Unsupported languages and undefined document types 54:07 AWS dependence and the path to cloud-agnostic 58:39 Security, privacy, and keeping data inside the agency 1:01:37 Governing Document AI after adoption 1:02:26 How to contribute 1:04:10 Final takeaways Links - Document AI open-source GitHub repository- Nava demo day
