This is Fine! A podcast about resilience engineering and software

Colette Alexander and Clint Byrum

A podcast about resilience engineering and software. Ever wondered why things on the internet break? Do you work in software and wish that you could have a Dear-Abby-Like call-in show that could answer your deepest questions about how to make your workplace suck less? We're here to help!  Write us anonymously at our open question form Email us at: thisisfine.softwarepodcast@gmail.com Call us and leave a voicemail, or text us at: ‪(401) 592-7574‬

  1. FEB 12

    Outsourcing and Resilience

    Colette mentioned Menlo Innovations https://menloinnovations.com/ and Atomic Object https://atomicobject.com/ who both build custom software for folks. The CEO of Menlo is Richard Sheridan who wrote Joy, Inc. - https://bookshop.org/p/books/joy-inc-how-we-built-a-workplace-people-love-richard-sheridan/7677689?ean=9781591847120&next=t Chad Todd’s thesis on Handovers in Software Operations is worth a read: https://lup.lub.lu.se/luur/download?func=downloadFile&recordOId=9076274&fileOId=9076276 Clint refers to Zingerman’s and their servant leadership model, one of Colette’s favorite places to learn about leadership from. If you want to know more, go to https://www.zingtrain.com/ and in particular, read https://shop.zingtrain.com/products/a-lapsed-anarchists-approach-to-being-a-better-leader

    42 min

  2. FEB 1

    The Messy 9 and Coding with AI - A Panel Discussion

    Special thanks to John Allspaw, Sheeri Cabral, Martin Smith, and David Woods for joining us! Ben Affleck’s been making the promo rounds, but the specific convo we reference is recapped here: https://www.moviemaker.com/ben-affleck-ai-explains/ The Messy 9 are: congestion cascade conflict lag saturation friction tempo surprise tangles Dave’s been doing a set of videos on Resilience Engineering, some of which have some crossover with the Messy 9 - you can find the first one here: https://resiliencefoundations.github.io/video-1-introduction-pt-1-it's-all-about-viability.html Previous TiF episode on the messy 9: https://www.thisisfinepod.com/the-pod/complex-systems-and-the-messy-nine-wspecial-guests-dave-woods-and-john-allspaw Richard Cook on Above the Line/Below the Line:  Written - https://dl.acm.org/doi/pdf/10.1145/3379510 A good excerpt from a talk from John Allspaw on Above the Line/Below the Line: https://www.youtube.com/watch?v=8bxj-FLEi10&list=PLb1aZTnPf3-OMChMkrr6WsokRI6LOnuem   Colette mentioned the competence knowledge model: https://en.wikipedia.org/wiki/Four_stages_of_competence There’s a good argument based on the conversation here that AI makes it harder for Consciously Incompetent people to graduate to Conscious Competence. And, in Martin’s case, it makes Unconsciously Competent folks need to backtrack into Conscience Competence to “teach” it how to do things they don’t always think about. We can reset the clock to 0 episodes since we’ve mentioned the Ironies of Automation: https://ckrybus.com/static/papers/Bainbridge_1983_Automatica.pdf There is a good blog on Jamie Zawinski’s saying on regular expressions here: https://regex.info/blog/2006-09-15/247 Alex Gorbachev and The Battle Against Any Guess seems to have become a paper https://www.researchgate.net/publication/251255185_Battle_Against_Any_Guess Dave talks about Robust Yet Fragile as part of Resilience Engineering here: https://www.youtube.com/watch?v=gFotUdLL2zs Lorin Hochstein’s blog post that Dave is referencing is https://surfingcomplexity.blog/2026/01/19/amdahl-gustafson-coding-agents-and-you/ Fred writes a good one on the Law of Stretched Systems: ​​https://ferd.ca/the-law-of-stretched-cognitive-systems.html The 1985 paper Dave keeps mentioning could be any number of things he released that year, but I have a hunch it’s this one: https://ojs.aaai.org/aimagazine/index.php/aimagazine/article/view/511 or this one: https://link.springer.com/chapter/10.1007/978-3-642-50329-0_11 Dave references a lot of things around the economic sustainability around AI, and Ed Zitron has been writing quite a bit about that for the last year and change. See: https://www.wheresyoured.at/wheres-the-money/ https://www.wheresyoured.at/big-tech-2tr/ Among others.

    1h 43m

  3. JAN 17

    Going Solid

    If you’re feeling like you need to do more to respond to our moment: Lots of place to donate to in the twin cities are listed here: https://mspmag.com/arts-and-culture/general-interest/ice-minnesota-support-immigrant-communities-fundraisers-food-drives-trainings/ You can always find mutual aid networks in your own area, including immigrant aid networks https://immigrantdefensenetwork.org/ does good work, too The Hometown Holler podcast with Tressie McMillan Cottom was a wonderful discussion: https://www.youtube.com/watch?v=2gr4mW8aR-g The Ruth Wilson Gilmore’s interview that I quoted clumsily is here: https://www.nytimes.com/2019/04/17/magazine/prison-abolition-ruth-wilson-gilmore.html  The paper itself: https://qualitysafety.bmj.com/content/14/2/130.short If you haven’t seen The Pitt, you should, it’s super good: https://en.wikipedia.org/wiki/The_Pitt Charles Perrow’s Normal Accidents has more definitions/examples of coupling: https://bookshop.org/p/books/normal-accidents-living-with-high-risk-technologies-updated-edition-professor-charles-perrow/cad38a43fcffa1f8?ean=9780691004129&next=t Some stuff on microservices and coupling here: https://microservices.io/post/architecture/2023/03/28/microservice-architecture-essentials-loose-coupling.html Colette’s #notanad endorsement for paper organizing is https://paperpile.com/ Rasmussen’s boundary model comes initially from his paper here: https://www.sciencedirect.com/science/article/abs/pii/S0925753597000520 And if you want a good writeup on Rasmussen’s boundary model explaining it, you can always read Lorin’s blog: https://surfingcomplexity.blog/2021/05/31/transgressing-the-boundaries-rasmussen-and-woods/ Dr Cook’s talk at Velocity is a classic, and goes over Rasmussen’s boundary model really well: https://www.youtube.com/watch?v=PGLYEDpNu60 Fred does a great job writing about the Law of Stretched Systems and how it applies to his own work on his blog: https://ferd.ca/the-law-of-stretched-cognitive-systems.html “Plans are nothing, but planning is everything” is a paraphrase of Eisenhower: https://www.presidency.ucsb.edu/documents/remarks-the-national-defense-executive-reserve-conference Want to chat about this paper with other folks? Come to the RISF live event for a Paper Party! https://resilienceinsoftware.org/events/157553

    1h 2m

  4. 12/31/2025

    The Year in Resilience w/special guest John Allspaw

    Seriously though, can’t wait to gtfo of this year. Palisades fire links: https://www.nbclosangeles.com/investigations/anonymous-letter-demands-independent-palisades-fire-investigations/3800442/ https://internationalfireandsafetyjournal.com/palisades-fire-report/ https://www.latimes.com/california/story/2025-12-20/lafd-report-on-palisades-fire-was-watered-down-in-editing-process-records-show Corey Quinn’s commentary on the AWS outage in October is here: https://www.theregister.com/2025/10/20/aws_outage_amazon_brain_drain_corey_quinn/ Time to reset the clock on how many episodes it’s been since we’ve mentioned the Ironies of Automation: https://ckrybus.com/static/papers/Bainbridge_1983_Automatica.pdf Also on Rasmussen’s Boundary Model, which Lorin does a great write up on: https://surfingcomplexity.blog/2021/05/31/transgressing-the-boundaries-rasmussen-and-woods/ Lorin’s Law is our favorite law: https://surfingcomplexity.blog/2017/06/24/a-conjecture-on-why-reliable-systems-fail/ You can ask us questions or write to us using our form linked from our website: thisisfinepod.com Resilience in Software Foundation is at resilienceinsoftware.org

    53 min

  5. 11/28/2025

    Incident Status: On Hold w/special guest Will Gallego

    Mentioned multiple times, Em Ruppe’s amazing talk on incident severity: https://www.usenix.org/conference/srecon24americas/presentation/ruppe We talk about the RIS Slack sometimes - you can join us in the slack, by joining the Foundation here: https://resilienceinsoftware.org/ Please ask us a question at thisisfinepod.com

    43 min

  6. 11/13/2025

    Complex Systems and the Messy Nine w/special guests Dave Woods and John Allspaw

    The writeup on the AWS outage from AWS themselves, if you haven’t seen it: https://aws.amazon.com/message/101925/ Dave’s department at OSU, Cognitive Systems Engineering: https://ise.osu.edu/human-systems-integration/cognitive-systems-engineering is a part of the larger Integrated Systems Engineering school: https://ise.osu.edu/human-systems-integration  Dave was talking early on about the discussion on the war on expertise, it was this webinar through the NDM association: https://vimeo.com/1129606494?fl=pl&fe=sh&mc_cid=c807a504fb Dave was a part of the  Paul Feltovich got a shout out - he wrote a lot, but one of the best is with Gary Klein on Common Ground and Coordination in Joint Activity: https://www.academia.edu/download/31764257/Common_Ground_Single.pdf And Studies of Expertise from Psychological Perspectives: https://www.researchgate.net/profile/Paul-J-Feltovich/publication/200772882_Studies_of_expertise_from_psychological_perspectives/links/58bd18b2aca27261e528de07/Studies-of-Expertise-from-Psychological-Perspectives.pdf Dave mentions his “Command-Adapt Paradox chapter” - you can find that here: https://library.oapen.org/bitstream/handle/20.500.12657/88327/1/978-3-031-45055-6.pdf#page=77 Shout out to Norbert Weiner, the godfather of cybernetics: https://www.jstor.org/stable/24945913 For just two studies on how private equity in hospitals causes worse outcomes for patients you can see: https://hsph.harvard.edu/news/private-equitys-appetite-for-hospitals-may-put-patients-at-risk/ And https://www.sciencedirect.com/science/article/pii/S0304405X25001151 Dave talks a bit about saturation and crossing boundaries towards failure - it’s worth familiarizing yourself with Rasmussen’s boundary model - Lorin Hochstein writes a good summary over at his blog: https://surfingcomplexity.blog/2021/05/31/transgressing-the-boundaries-rasmussen-and-woods/ Dave also mentions graceful extensibility - this is a concept he’s written quite a bit about, you can start here: https://link.springer.com/article/10.1007/s10669-018-9708-3 Shout out to Slight Reliability: https://slightreliability.com/ One of the great Woods/Cook write ups on anticipation in anesthesiology: https://www.sciencedirect.com/science/article/pii/S0952818096900094 In case you’re unfamiliar with the Chicago Seven: https://en.wikipedia.org/wiki/Chicago_Seven The Messy 9 are: congestion cascade conflict lag saturation friction tempo surprise tangles Keep an eye on the merch store over at https://www.bonfire.com/store/risf/ if you want the t-shirt.

    1h 8m

  7. 10/30/2025

    All the things about Incident Command

    It’s Spamton G (not J) Spamton, Clint! Get hip to the game characters!  https://deltarune.fandom.com/wiki/Spamton There are a couple of incident command trainers out there who tend to get recommended in the tech world (that we know of): https://www.blackrock3.com/ and Great Circle: https://greatcircle.com/im/

    37 min

  8. 10/16/2025

    Root Cause Analysis vs. Resilience Engineering

    A history of the 5 whys and root cause analysis from papers Some critiques of the 5 whys: From John Allspaw: https://www.oreilly.com/radar/the-infinite-hows/ From Alan J Card: https://qualitysafety.bmj.com/content/26/8/671 James Reason and the Swiss Cheese Model:  https://pmc.ncbi.nlm.nih.gov/articles/PMC8514562/ James Reason’s book Human Error: https://bookshop.org/p/books/human-error/9e06d8a100a07537?ean=9780521314190&next=t And a classic from Sidney Dekker (et al.) on the implication of complexity within safety investigations: https://www.sciencedirect.com/science/article/abs/pii/S0925753511000105?via%3Dihub We always recommend the Howie Guide: https://howie-guide.pagerduty.com/ STAMP is starting to get popular: https://functionalsafetyengineer.com/introduction-to-stamp/ Google’s STAMP paper: https://www.usenix.org/publications/loginonline/evolution-sre-google Google’s STAMP discussion on ProdCast: https://sre.google/prodcast/#season4-episode7 And presentation at SRECon: https://www.usenix.org/conference/srecon25americas/presentation/klein Nancy Leveson’s google scholar is always worth browsing: https://scholar.google.com/citations?user=78y4sEcAAAAJ&hl=en Allspaw’s LinkedIn post that we quoted: https://www.linkedin.com/posts/jallspaw_important-reminders-about-learning-effectively-activity-7378775591447183360-c_eD Lorin’s Law: https://surfingcomplexity.blog/2017/06/24/a-conjecture-on-why-reliable-systems-fail/ Want to talk more about this subject? We’re doing a live event co-sponsored by RISF and you can sign up for it here: https://resilienceinsoftware.org/networks/events/146485

    1 hr
See All (30)

Ratings & Reviews

5
out of 5
4 Ratings

About

A podcast about resilience engineering and software. Ever wondered why things on the internet break? Do you work in software and wish that you could have a Dear-Abby-Like call-in show that could answer your deepest questions about how to make your workplace suck less? We're here to help!  Write us anonymously at our open question form Email us at: thisisfine.softwarepodcast@gmail.com Call us and leave a voicemail, or text us at: ‪(401) 592-7574‬

Information

You Might Also Like