Ben Lengerich on why AI trained on hospital records mistakes the sickest for the safest. Ben is a statistician at the University of Wisconsin-Madison and the founder of Intelligible, which makes software that records what an AI finds in data. Parts of this conversation are close to his business. In this conversation - In hospital records the sickest patients can look the safest, because they get the most care, and it works. - A perfect test score only proves an AI passed that test, so ask what the test measured. - An MOT for AI: rules written for one use, and checks that never stop. - AI note-takers at the GP, and the line the UK regulator drew in July. - An AI read Amer's X-ray in seconds. Did the doctor really check? - When a sealed-box AI is fine, and when you want pieces you can check. - In the UK, a computer can turn down your loan on its own only if you can challenge the decision. - Ben's prediction: within two years hallucination stops being the worry, and the question becomes what an AI is for. Chapters (00:00:00) What the hospital data hides (00:01:31) Why kidney failure looks safe (00:07:48) Ten thousand ways to lie (00:10:31) 100% on the test, still wrong (00:16:08) An MOT for AI (00:23:12) Round-number birthdays (00:28:19) AI note-takers at the GP (00:33:00) Did anyone check the X-ray? (00:39:31) The last ten minutes in A&E (00:42:07) Sealed box, or pieces you can check (00:49:57) True, or right? (00:54:13) Show your working (00:58:19) Turned down by a computer (01:05:42) The prediction Links Watch on YouTube: https://youtu.be/RUmr6Ai8Qt8 Ben Lengerich: https://www.linkedin.com/in/benlengerich/ Intelligible: https://www.intelligible.ai The paper, npj Digital Medicine: https://www.nature.com/articles/s41746-025-02079-y The Control Layer, subscribe free: https://thecontrollayer.arkava.ai/?utm_source=spotify&utm_medium=description Arkava: https://arkava.ai Sources Lengerich et al., "The hidden risk of round numbers and sharp thresholds in clinical practice", npj Digital Medicine, 21 Nov 2025 Caruana et al., "Intelligible Models for HealthCare", KDD 2015 Derksen et al., survey of 598 UK GPs on AI scribes, npj Digital Medicine, 2026 MHRA, regulatory status of ambient voice technologies in the NHS, 29 Jul 2026 Dratsch et al., automation bias in AI-supported mammography, Radiology, 2023 NICE HTG739, AI to help detect fractures on X-rays in urgent care, 14 Jan 2025 IFS, "The four-hour NHS waiting time target saves lives", 17 Jan 2020 Data (Use and Access) Act 2025, s.80 (UK GDPR Art. 22C), in force 5 Feb 2026 Bank of England and FCA, "Artificial intelligence in UK financial services", 21 Nov 2024 Anthropic, "Claude's new constitution", 22 Jan 2026 Hosted by Amer Altaf, Founder and CEO of Arkava and Managing Editor of The Control Layer. Views expressed by guests are their own. Some figures are the speakers' own accounts; independently verified statistics are attributed on screen. Written by a real person, assisted by AI tools, with every claim checked.