James Maconochie | Architecture & Attention Podcast

James Maconochie

Essays on Augmented Human Intelligence, the Wisdom Gap, and the architecture of attention in an AI-mediated world. Read in James Maconochie's own voice. jamesmaconochie.substack.com

  1. 5d ago

    Fifty-Two Weeks

    When I started this series, I made myself a promise: an essay every week. I have kept it every week but one. That one, I wrote and thought I had posted from a few days at the beach, but when I got home, I realized I'd missed the last step. This is the fifty-third essay. The first fifty-two went out in just under eleven months. From here, I am going to publish every two weeks. The next essay will arrive on October 27, and every second Tuesday after that. Thanks for reading James Maconochie | Architecture & Attention! Subscribe for free to receive new posts and support my work. I have plenty to say, and I am not stopping. But I wrote one recent essay the day before it was due, and the next on the day itself. That told me something I had been avoiding. When I began, weekly was a decision. I made it on purpose, for reasons I could have explained to you. At some point it stopped being a decision and became simply what I do each week. I hadn't asked whether it still fitted my life since the day I made it. This essay is me asking. It turns out the weekly essay isn't the only routine I stopped examining, and it isn't the one that cost me most. Why Weekly When I began, writing was the whole job. I had spent twenty-five years in technology consulting, and I had already written several long working papers on how I think artificial intelligence should be built and used. The material was there. What I lacked was a track record as a writer. So I set a rule: one essay a week, each one good enough that I was happy to stand behind it. My thinking was simple. A new writer has no record, and the only way to build one is to turn up on time, again and again, until people can see what you are about. A weekly essay was how I would lay a base. I still think that was the right call for the conditions. I had the time, and I had plenty of material. The discipline turned a stack of papers into fifty-two essays that someone can read from the start. It also pushed me, week by week, to write more plainly for readers who do not work in technology. I am still working at that. But the rule was made for those conditions. The conditions changed, and I left the rule as it was. What Went, and in What Order Start with the walking. In 2013, my doctor told me I was pre-diabetic. I started measuring my blood sugar, and I started walking every day. Within a year, I had lost 60 lbs. By then, the walking was no longer a treatment. It was my time: anywhere from 4 to 14 miles a day, in any weather, with music or a book in my ears. For ten years I was up at four or half past to fit it in before work. By my count I have walked once round the Earth and started a second lap. I kept it up for most of twelve years, through the year I was out of work and looking, and into the months after, when I began my research and writing. Then, in the autumn of 2025, I slowed down. Some months later I had more or less stopped. The slowing began before I had published an essay here, and well before the driving. No single thing ended it, and time wasn't the reason. I began to question a lot I had taken for granted, and the walking got caught up in that. I had lost sight of why I started. The benefits had been there so long that I doubted walking caused them. And, if I am honest, part of it became about how I looked and what other people thought of me. So I let it go, and I am worse off for it. The books went too, because I listened while I walked. I am still 30 lbs lighter than at my worst, but I am not where I was. Five months ago, I started driving for Uber, 20 to 25 hours a week. I had promised my wife that I would bring in income and keep writing, and this was how. For a while, the essays didn't suffer, because I had plenty stored up. But something had to give. A month or two ago it was the time I spent reading and replying to other writers here. In the last few weeks it has been the essay itself, squeezed into the final day. The plain account of that is not very interesting. I took on more than twenty hours of driving a week and expected everything else to fit around it. It did not. What interests me is that it took me about five months to ask whether the weekly essay still made sense. Two Standing Answers In The Fasten Your Seatbelts Century — Again, I wrote about living with a seatbelt sign that never goes off. You can ignore it, or you can stay white-knuckled. Either way, you have stopped asking whether this particular warning means anything. I called those settings, not decisions. A decision looks at the conditions. A setting was once a decision, and now runs without anyone checking it. I have been running two of them. The walking was a setting that was good for me. It began as a decision, driven by a doctor’s warning and a blood sugar reading. Over the years, the reason faded, and only the habit remained. When I finally questioned it, I had lost sight of what it was for, so I had nothing to defend it with. I dropped a good routine because I had lost its reason. The weekly essay was a setting that no longer fit. It also began as a decision, for the reasons I gave above. Then the driving filled my weeks, and I did not ask whether a rule made for open weeks still suited full ones. I kept a routine because I never went back to its reason. Those look like opposite mistakes, and the causes were different. The walking faded through apathy. The weekly essay stopped fitting because my time changed. But I think the mistake underneath is the same one. In both cases, I had the routine and had lost track of why. I do not think routines are the problem. Twelve years of walking happened because I did not have to decide each morning. The trouble starts when the reason gets obscured by the mist of life, because then I cannot tell a routine worth keeping from one that needs changing. Is this adjusting, or giving up? Fifty-two essays show that I can keep going. They do not show that slowing down is right. What tells me this is an adjustment is that the conditions changed. The weeks I made the rule for are not the weeks I have now. What Changes for You An essay will arrive every second Tuesday, starting October 27. In the weeks between, I will go back to reading and replying to other writers here. I stopped a month or two ago, when the driving squeezed it out. Many of the ideas in these essays began as a comment on someone else’s work, or as an objection someone raised to mine. I do not want to write without listening. The walking is a separate matter. It is not for the weeks between. It is for every week. I want to be careful here. A lighter schedule will not put me back on the road, because time was never the reason I stopped. The change gives me room. The rest is up to me. I am saying so here, in front of you, because a promise made in public is harder to let slide quietly. I know I am saying that in the same essay in which I change an earlier public promise. The difference, I would claim, is that I am changing that one in the open and telling you why. With the walking come the books, and the books feed the writing. I cannot promise the essays will be better for being fewer. I do intend that I will not write them on the day they are due. What I Do Not Know I do not know that every two weeks is right either. It is my best reading of the conditions I am in now: the driving, the walking I have started again, and the conversations I want to get back to. Those will change again. So this time I am writing the reason down. That is partly what this essay is. When I next wonder why I publish every other Tuesday, the answer will be here, dated, and I can check whether it still holds. I mean to do that in the spring, after a dozen essays at the new pace. Fifty-two essays showed me I can hold to a routine. The walking showed me that holding to one is not the same as remembering what it is for. I would like to manage both. Thank you for reading the first fifty-two. The next one is on October 27. Thanks for reading James Maconochie | Architecture & Attention! Subscribe for free to receive new posts and support my work. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit jamesmaconochie.substack.com

  2. Oct 6

    The Batting Average

    Some years ago, while consulting, I spent many hours in a car with a younger colleague. Our client was three or four hours away, and for most of two projects we made the drive nearly every week. That is a great deal of time to talk. She was early in her career, bright and ambitious, and good at the work. From time to time the conversation came round to careers, and it was plain she was frustrated with how slowly hers was moving. Thanks for reading James Maconochie | Architecture & Attention! Subscribe for free to receive new posts and support my work. I should say at once that I knew the feeling. I had it many times in my own career, and I am not sure I ever entirely lost it. So I tried to encourage her. I pointed to what I had watched happen over those two projects: the progress she had made, and the trust and respect she had earned from the client and from our own team. I told her that the frustration was common, and that measuring yourself against people you believe are at your level is a reliable way to stay unhappy. I said something like this more than once. If you have ten goals on an engagement, and you achieve six of them, you have not failed. You have six wins. I did not think it landed. She heard me, she was gracious about it, and as far as I could see, the frustration stayed where it was. In her accounting, as I read it then, anything short of ten was a loss. I have thought about those drives often since, because over a consulting career that ran from 1998 to 2023 I met that accounting more and more often, and found it harder and harder to argue with. No Running Total Baseball has a number for this. A hitter’s batting average is simply how often he gets a hit. One who manages it three times in ten is among the best in the game, which means the best hitters fail far more often than they succeed. Nobody watching thinks of them as failures, because nobody judges a hitter on a single trip to the plate. The average only exists because someone keeps the record. That is what I was trying to offer my colleague on those drives, though I did not have the words for it then. I was asking her to keep a ledger: a running record of what she had attempted and how it turned out, read as a whole. Six of ten, on a ledger, is a good season. What I was arguing against has no ledger. Each goal, each review, each argument is scored as a win or a loss on its own, and then the score is thrown away. Nothing accumulates. With no running total, the only result that feels safe is a perfect one, and anything less reads as defeat. I think this is one way absolutism gets made. We usually describe it as stubbornness or certainty, and sometimes that is all it is. There are also matters of principle on which people are right not to settle for six of ten. But much of the all-or-nothing I met was neither. I have come to see that part as a bookkeeping problem. It is judgment with no record behind it. In August, in The Ledger We Refused to Read, I wrote about a government that had a record of its own past judgments and, as far as we were shown, did not consult it. This is the same failure at the scale of one person. The government had a ledger and left it closed. The absolutist never started one. What Gets Counted I said I met this accounting more often as the years went on. I should be careful about what that is worth. It is one consultant’s impression, formed over twenty-five years of working alongside people younger than me, and I have no study to back it up. But I do not think I imagined it, and I think I know one place it is taught. It is taught at work. In several companies I worked for, the review process ran the same way. Each year started more or less from zero. What counted was this year, and often the last few months of it. The result that mattered was a yes or a no on promotion, so six strong achievements and no promotion registered as a no. And where places were limited, the firm itself measured you against the people at your level. I told my colleague not to compare herself with her peers. The system she worked in may have been doing it for her. None of these firms lacked records. The reviews were on file, year after year. I am not aware that anyone read them as a whole. It would have taken too long. Whether that was oversight or convenience, I cannot say. The same accounting runs well outside the office. In politics, on television, online, a deal that secures most of what a side wanted is reported as a surrender, and conceding a point is treated as losing the exchange. I cannot prove what caused this. I can point to something that would reinforce it. Much of public conversation now happens on platforms that keep score, and what they count matters. They count reactions: how many people responded to this post, this clip, this reply, today. Nobody’s page shows how their judgments held up over five years. These platforms do keep a running total. It is a total of attention. So the scoreboards most of us look at every day, at work and online, reward the recent and the visible. Neither has a place to show how your judgment held up over the years. If you wanted to teach people to score each exchange alone and discard the result, I am not sure you could design a better classroom. Where The Frame Could Break This argument is weaker in at least three places than I have made it sound. First, a batting average treats every attempt as equal, and decisions are not equal. I trained as a civil engineer. Nobody wants a bridge designer who gets six out of ten right. In September, in Who Is Left Holding the Napkin, I described a model that scored every bridge on two things: its condition, and the consequence if it failed. A ledger of judgment needs that second column. Counting six of ten as a win is sound advice for a career, an argument, or a negotiation. It is dangerous advice anywhere one error is permanent, because the people who pay for it do not get another turn. It is also the first question I would ask of any AI system offered to me on the strength of its average: is this use a negotiation, or a bridge? The second concerns my colleague, and I owe her this. I told her to temper her expectations. Her expectations may have been reasonable, and the pace may have been the problem. Capable people early in their careers are sometimes held back for reasons that have nothing to do with their outlook, and “be patient, count your wins” is convenient advice for an employer to have in circulation. I believed what I told her, and I still think it was true. But I was a senior colleague describing the view from further along, and I cannot rule out that some of her frustration was an accurate reading of her situation. The third is the hardest. A ledger you keep for yourself does not change the one your employer keeps. If the firm counts only this year, and only yes or no, your private record of steady progress settles nothing at review time. I do not have a full answer to that. Companies are the way they are, and short of starting your own, there is little you can do about how they count. I wrote last week, in The Better Design Waits for the Rule, about why a better arrangement everyone can see still doesn't happen until the rule changes for everyone at once. Review systems are no different. This is the individual’s side of that problem. What you can change is how much their count weighs with you. I am not the person to preach this. For most of my life, I have wanted other people to confirm my worth, and I have been disappointed by promotions, raises, bonuses, and awards more often than I would like to admit. It is less than it was. It has not gone. But I have slowly learned to give my own opinion, and my family’s, at least as much weight as an employer’s. That only works if my own opinion is worth trusting, and an honest record is what makes it so. None of these undoes the argument. All three narrow it. Keeping a ledger is no reason to accept whatever you are given. It lets you tell the difference between a bad season and a bad game. Keeping The Count So here is what I would say now, with better words than I had in the car. Keep a ledger of your own judgment. The full version is simple to describe. Before something matters, write down what you expect, why, and what you would count as a good result. Afterward, go back and read it. An entry written beforehand cannot be adjusted to suit how things turned out, and that is what separates a ledger from a diary. Its job is to make it harder to fool yourself about how you are doing, no matter how the truth turns out. It is roughly what the best forecasters do: commit to a prediction in advance, then score themselves against what happened. I should admit that I have never kept one that carefully, and I doubt many people will. It takes time, which is why those firms never read their own files. So I would settle for less. I could be persuaded to do it properly for the few decisions that matter most. For the rest, before you call something a failure, ask two questions: what did I actually expect, and what does my record on things like this look like? Note what happened, and also whether the call was sound. A good decision can turn out badly and a poor one can get lucky. Over enough entries, the record starts to show which is which. Then read the record as a whole. Six of ten cannot be judged alone. Read against a record, you can tell whether it was a good season or a poor one. And notice what a ledger does to being wrong. If each argument is the entire score, conceding a point means losing everything, so of course people refuse. With a running total, a concession is one entry among many. You can afford it. That is how it worked for me. I like to think I grew wiser. I know I acquired a longer record, and it told me that being wrong on a Tuesday had never yet finished me. I also had the standing by then to survive a bad Tuesday, and I know that not everyone does. The most important thing I kno

  3. Sep 29

    The Better Design Waits for the Rule

    The Model That Changed Nothing Around 2010, I worked on a model that scored every bridge in the National Bridge Inventory, the federal record of highway bridges on America’s public roads, on two things at once: what condition it was in, and what it would cost if it failed. A lightly used rural bridge in poor condition and a busy interstate crossing in fair condition carry very different risks. Yet most bridges were inspected on the same two-year cycle. I presented the model at the Federal Highway Administration (FHWA) headquarters in Washington, alongside colleagues from Lloyd’s Register. In the room were the Associate Administrator for Infrastructure and several of his colleagues from the bridge side. He was supportive, but he made no decision and no commitment. Bridge leads at state transportation departments gave it the same reception. We had linked the model to Google Earth, so we could fly over each state’s highest-risk bridges. None of the state bridge engineers who saw it disagreed with what it showed. There was real interest, and then nothing. Thanks for reading James Maconochie | Architecture & Attention! Subscribe for free to receive new posts and support my work. The inspection program did not change. Not then. The information existed, the people closest to the problem found it credible, and it still did not turn into action. I have come to think this is not unusual. It is one of the common ways a better design loses. This series argues that the answer to AI’s judgment problem is design: build systems so the people who bear the consequences of a decision stay part of making it. If that is right, there is an uncomfortable follow-up. Why does the design answer so often lose in practice? The bridge model is my best evidence of how it happens, because it happened in front of me. Two Columns The usual reason a better design loses is cost. Doing things properly takes more money up front, and budgets are set in the present. That is often a fair objection, and it would be dishonest to pretend the design answer is always cheap. The bridge model removed that excuse. It worked from the data states were already collecting for the inventory. Nothing new had to be gathered. Rolling it out wouldn't have been free, but it was modest compared with the size of the program. The benefits were real, but they took a different form. Few people would argue that a simple concrete beam bridge, under twenty years old and used mostly by farm vehicles, needs the same attention as a major interstate crossing. Inspecting such bridges less often would free up effort for better inspection and targeted maintenance on the large, busy bridges where failure matters most. Those bridges are costly to maintain, but replacing one costs far more. If that attention added five or ten years to the life of a major crossing, the savings could be large. The biggest benefit of all is a failure that never happens. That is the problem. A failure that never happens leaves no record. No one can point to the bridge that did not close, or the replacement that was pushed back a decade because a problem was caught early. The benefits go in the ledger as estimates about events that, if the design works, will never occur. The other column, it turns out, was not money. It was the risk of change itself, and who would carry it. That is where the rest of this essay goes. Tonie-Marie Gordon, writing about AI at work, describes the same accounting error from the other direction. The productivity gain from a new tool gets booked in full, while the human effort needed to check its output is left off the ledger. Either way, what gets counted tends to win. The Number and the Money At the time, the inspection data did more than describe bridges. It helped decide how much federal money each state received for them. Under the Highway Bridge Program, a state’s share of the funding was based on the estimated cost of repairing or replacing its deficient bridges, those formally rated as in poor condition or outdated for today’s traffic, calculated from the inventory. That made any change to how bridges were inspected more than a technical question. The same data that recorded a bridge’s condition also fed the formula that divided up the money. A state’s bridge staff might find a risk model persuasive and still have good reason to be cautious. If the new approach changed what was collected, or how it was read, it might change what the state received. Nobody would want to be the one who found out the hard way. This is not a claim that anyone gamed the numbers. The caution was reasonable, given how the money worked. A slow federal process added to the caution. Changes to a national program move through years of rules, comments, and consultation, and a program that seems to be working invites the natural response: if it isn’t broken, don’t fix it. The trouble is that “working” meant the inspections got done and the numbers came in. It did not mean the effort was going where it did the most good. Once a number decides where money goes, protecting that number can become a goal in itself. The Congressional Research Service noticed a version of this in 2008, observing that a formula paying states according to their deficient bridges gave them little reason to reduce the count. Through her reporting, Rana Foroohar has traced the same pattern across the wider economy. When firms were judged on financial figures, managers came to steer by the figures rather than by the work they measured. The figures were persuasive, and too few people asked what they left out. In this case, the better design did not lose on cost. I cannot prove why it lost. I believe it lost at least in part because it threatened the numbers the money depended on. Not Just Bridges The bridge model is the clearest case I have, but I have seen versions of it in technology consulting. One project stays with me. I worked with a large software company on how its sales system handled leads: how a first contact became a qualified lead, and then a real prospect. Marketing was judged on how many leads it passed along, and sales on how many it turned into customers. Sales believed marketing was passing along weak leads to boost its numbers. Those leads rarely went anywhere, so the sales team wasted time chasing them and looked worse when they failed to convert. Nearly everyone recognized the problem when we laid it out. Fixing it meant changing how both teams were measured, at the same time. Our engagement ended with recommendations to do that, and an expectation of follow-on work to implement them. That work never came. As far as I know, the company kept things as they were. The rule there was not a federal regulation but a pair of targets. The pattern was the same: a better arrangement everyone could see, and no one in a position to change the rule for both sides at once. The Asset Nobody Counts One more thing is missing from both columns, and it matters most for AI. Most organizations depend on judgment that lives in their people rather than their procedures. It shows up in the inspector who knows which joint on which bridge to look at twice, or the sales manager who can tell a real lead from a hopeful one within a few minutes on the phone. Little of it appears in a budget. You learn it through practice, pass it on informally, and notice it mainly when it is gone. I think of it as cognitive collateral: the capacity an organization relies on to catch its own mistakes, which never shows up on its balance sheet. If offloading judgment is a kind of borrowing, cognitive debt is what accrues, and cognitive collateral is what is lost when the debt comes due and cannot be paid. AI tools put this capacity under pressure in a particular way. When a system does the routine work, people get less practice at it. In 1983, the engineering psychologist Lisanne Bainbridge pointed out the irony: the skills that fade during smooth automated operation are the ones needed when the automation fails. A design that keeps people exercising their judgment, rather than only approving what a system produces, protects that capacity. But the protection is another benefit that never appears in any column. The simpler design's efficiency gain is immediate and measurable. The judgment it quietly spends is neither, until the day it is needed and turns out to be thinner than anyone knew. That is the design answer’s problem in its most general form. What it protects is invisible, and what it asks people to change is not. Asking to Be Ruled This month, the heads of several of the largest AI labs renewed a call that industry leaders have made, on and off, for years: they asked for rules. At the United Nations Security Council, several told members that AI needed global oversight. Anthropic’s Dario Amodei and OpenAI’s Sam Altman have both said development should slow down, though Amodei has argued the leading labs should slow down only as far as they can while still leading China. Earlier this month, the White House rejected calls for federal guardrails. There are two serious ways to read the request. The first follows this essay's logic. A lab that slows down on its own bears the whole cost: it falls behind competitors who did not. The benefit, a failure that never happens, is shared by everyone and credited to no one. That is the bridge problem again. A state could not change the inspection schedule on its own; the change had to come from the body that set it for everyone. It eventually did, but only after Congress required it. The second reading is less generous, and just as structural. Rules a large company can absorb easily can be heavy for a small one, and the biggest labs are doing the asking. Vice President JD Vance has called the push a possible Trojan horse. This is the lesson of the bridge funding formula in another setting: a rule that decides what counts also decides who gets ahead, and those shaping it have intere

  4. Sep 22

    The Fasten Your Seatbelts Century - Again

    I turned on my own seatbelt sign about four years ago. I had been reading the same headlines as everyone else, and I reached a conclusion I found hard to argue with: too many things could go wrong at once, and most of them would hit the markets before they hit anywhere else. So I moved a good part of my retirement savings somewhere I judged safer. It wasn’t a trade. It was a posture. I buckled in and waited. Thanks for reading James Maconochie | Architecture & Attention! Subscribe for free to receive new posts and support my work. Nothing happened. Four years on, the sign is still lit. Recently I made a list of the warnings that keep it lit, and introduced them with the phrase “only a couple of blinking warnings that I’m aware of.” Here is the list. * Government debt that nobody has a plan to repay ($40T) * A bond market that hasn’t settled * Share prices that assume nothing goes wrong * A technology boom that is partly financing itself (AI) * Whatever that boom does to jobs * A war with no sign of ending (Ukraine) * A standoff that could become another (reader’s choice - US/Iran, China/Taiwan, North Korea/South Korea) Then I counted. There were seven. That gap between “a couple” and seven is what this essay is about. It isn’t evidence that things are worse than I thought. It might be evidence that I’ve stopped being able to read my own instruments. Here is what I’ve done since. I’ve quietly moved some of that money back. Not all of it. Enough to notice. I don’t know whether that was judgment or fatigue. The Sign On a plane, the seatbelt sign works because of three things you rarely think about. The first is that someone with better information than you switches it on. The captain can see the weather radar and hear reports from aircraft further ahead. You can see a cloud. The second is that the captain is on the plane. If the turbulence turns out worse than expected, the captain feels it too. That doesn’t make the captain infallible. It means the person switching the sign on pays when they get it wrong, which is why you can obey it without being told the reason. The third is the one this essay depends on: the sign goes off. On a smooth flight it stays dark most of the time, so when the chime sounds, it means something. The sign carries information because it isn’t always on. Take away the first two and the sign becomes harder to trust. Take away the third, and it becomes part of the cabin. The Last Time the Sign Was On First, what this essay isn’t claiming. The twentieth century was the bumpy one: two world wars, a depression, a pandemic that killed tens of millions, and four decades in which two governments could have ended most life on earth in an afternoon. If a century ever had its seatbelt sign on, that one did. Many of us in the West who came of age around the end of the Cold War believed it had gone off. Since the autumn of 2001 it has been on for most of the time: two long wars, a financial crisis that came close to freezing the banking system, a pandemic that closed much of the world, inflation back after a generation away, war in Europe and the Middle East, and political turmoil in democracies long thought stable. That is a selective list. The same years brought falling extreme poverty and longer lives, and I’m not claiming the world got worse. I’m describing how the sign has behaved. What stands out is how short the quiet stretches have been. When the Alarm Never Stops Hospitals have studied what happens when a warning never turns off, because there the cost is counted in patients. A patient on a modern ward can be connected to several monitors at once, tracking heart rhythm, oxygen, and blood pressure, each set to sound when a number crosses a line. Each alarm makes sense on its own. Together they can produce several hundred alarm signals per patient per day. The Joint Commission, the body that accredits most American hospitals, reviewed the research and found that between 85 and 99 percent of those alarms required no action. Nurses adapt, as anyone would. They learn which beeps usually mean a loose sensor or a patient who shifted in bed. They turn the volume down, widen the limits, and sometimes switch alarms off. The field calls this alarm fatigue. In 2013, the Commission issued a formal warning to hospitals about it. Over about three and a half years, it had received reports of 98 alarm-related events, 80 of which ended in a patient’s death. It noted that reporting is voluntary and that these cases are only a small share of what actually happens. What matters here is how the failure happens. It is tempting to call it carelessness. It isn’t. A nurse who stops reacting is usually responding sensibly to what the alarms have taught them. When nine alarms in ten mean nothing, treating the next one as probably nothing is a reasonable bet. The trouble is that habit places the bet, not judgment, and the tenth alarm sounds exactly like the other nine. The alarm didn’t stop working. It stopped carrying information. The sound stayed the same, but its meaning drained away. Maybe the Air Is Rough There is a fair objection here. Hospital alarms can be scored: afterward, we know which ones needed a nurse. The warnings on my list can’t be scored that way. Some take decades to resolve, and some never will. By a warning I mean a claim that something bad has become more likely and that you should act differently because of it. This difference has a name. Risk is what can be measured; uncertainty is what can’t. Hospital alarms are risk. Most of my list is uncertainty. Economists argue over some of it with better tools than mine, but nobody can yet say which of my seven are right. So maybe the air really is rough, and the sign is on because it should be. The argument doesn’t depend on ruling that out. Even if the sign is right, a sign that is always on can’t tell you when things get worse. Pilots already deal with this. When turbulence is expected to be severe, the captain adds a second announcement, “flight attendants, please take your seats,” because the seatbelt sign alone no longer marks the difference. In steady rough air, the useful signal is the one that separates this bump from the others. That second announcement is what I think we’re missing: a way to tell which of seven lit signs deserves more attention than the rest. Unbuckled or White-Knuckled Without that second signal, people tend to go one of two ways. The first is to unbuckle. After enough warnings that go nowhere, you stop listening. Headlines about debt, war, or the next crisis become background, like the hum of the engines. This feels like calm, and sometimes it is. But it is the nurse’s bet: the next warning is probably nothing. Most of the time that’s a reasonable bet, and it fails precisely when it matters. The second is to stay white-knuckled. You keep the belt tight and never relax, treating every warning as the real one. This feels like prudence. But vigilance that never stands down has its own costs. It is tiring, it crowds out other things, and it has its own blind spot. If you treat everything as an emergency, you can’t pick out the one that is. The relaxed passenger and the white-knuckled one look like opposites, but the sign has stopped telling either of them anything. One ignores it; the other obeys it constantly. Neither is reading it. Both are settings, not decisions. Each replaces the question “does this warning mean something?” with a standing answer: no for one, yes for the other. Judgment is the ability to give different answers to different warnings. A sign that never goes off makes that very hard to do. What I Can’t Tell So what does another twenty or thirty years of this produce? I’ve kept hoping things would settle down. It’s a hunch rather than a forecast, but I no longer expect the sign to go off in my lifetime. I don’t think the answer is collapse. Societies have absorbed worse than anything on my list. The more likely result is quieter. Some institutions warn too often; others warn too little. After enough years of a sign that never goes off, a public finds it harder to tell those apart. Which brings me back to the money. The simplest explanation for what I did is that I made a bad call four years ago and am slowly correcting it. That may be all it is. When I moved some of the money back, I told myself it was recalibration: four years is a long time for a warning to go unconfirmed, and a sensible person updates. But that is also exactly what the nurse tells herself about the tenth alarm. From inside, recalibration and fatigue lead to the same action and bring the same relief. I can’t tell which one I’m doing. What I do know is that I’m not a passenger watching someone else’s sign. If I was right four years ago and have now relaxed, I’ll pay for it. If I was wrong all along, I’ve already paid in the growth I didn’t get. Either way, the cost lands on me, which puts me on the plane. In July I wrote about the difference between a ledger and a scar. A ledger records what happened. A scar changes the one it happened to. Whatever this decision turns out to have cost, it won’t just be written down somewhere; I’ll carry it. I can’t tell yet whether the scar will come from relaxing too soon or holding on too long. Who Sets the Threshold The answer on offer for a public that can’t sort its warnings is to let something else do the sorting. News feeds already decide which stories reach you and how urgently, and hospitals are trying smarter alarms. Some of this will help. But a system that sorts warnings is a system that sets thresholds. It decides what gets through, what gets flagged as urgent, and what stays quiet. Three questions matter about any threshold. Who sets it? Who pays when it’s wrong? And does it ever switch the sign off? On the plane, the answers are simple. The captain sets it, the captain is in the turbulence t

  5. Sep 15

    Who Is Left Holding The Napkin

    I have told this story before. Six weeks ago I told it to make a point about software. I want to tell it again to make a point about myself, because I think I put the emphasis in the wrong place. In 1993, I was a final-year civil engineering student at Imperial College, and my design project was an exhibition hall with a column-free interior and a long-span roof carried on a pair of massive tubular arch beams. I modeled it in I-DEAS, a finite element package running on a Silicon Graphics workstation that took up half a desk. The package computed the stresses, checked them against code, and rendered a deformed shape that looked entirely reasonable. It said the structure would stand. Thanks for reading James Maconochie | Architecture & Attention! Subscribe for free to receive new posts and support my work. The night before the report was due, for reasons I still cannot reconstruct, I did a hand calculation of the stress at one of the arch supports. It came back off by a factor of ten. I had entered the loads in kilograms, but the package read them as newtons. My model was carrying about a tenth of what the real building would have to carry. When I corrected the units and re-ran it, the deformed shape came back flat as a pancake. The structure I had designed would not have stood up. Last time, I said the software never pushed back; the analysis was fluent all the way down. The correction had to come from outside it. That is true, and it is the smaller half of the story. The larger half is the napkin. I had a way to check. It was crude; it took three minutes, and it was decisive. And I had it for a reason that had nothing to do with being careful. I had been taught to size a beam by hand before I was ever allowed near a package that would size it for me. The check wasn’t diligence. It was residue. The Word We Only Half Unpacked Five weeks ago I wrote about the word reward, and what happened to it when AI borrowed it from neuroscience. In the brain, reward is not one thing. Kent Berridge’s work separates it into two systems that can be pulled apart experimentally. Wanting is driven by mesolimbic dopamine; it makes you pursue the thing. Liking is a distinct hedonic system running on different neural substrates, and it registers how the thing actually landed. Disable one and the other carries on without it. An animal can be made to want, intensely, something it does not like at all. Machine learning borrowed the word and built the pursuit. I want to state this more carefully than I did last time, because it is easy to overclaim. Reinforcement learning gives a system an explicit optimization signal and a great deal of patience in following it. What it does not supply, by itself, is anything corresponding to the second system, an independent register of how the result actually landed, sitting outside the objective and capable of disagreeing with it. There are constraints, regularizers, safety layers, human feedback, evaluators. What’s missing is something that stands apart from the optimization and asks whether the objective was the right one. Drive without a brake, with the caveat that this is a metaphor, and a metaphor is not a schematic. The precise version is duller and harder to knock down: a system optimized to produce an answer is not thereby equipped to establish that the objective, or its reading of the objective, was correct. I left it there, as a fact about how these systems are built. It is a fact about how they are built. But nothing is dangerous all by itself. An uncovered hole in the ground is not dangerous sitting in an empty field. It becomes dangerous when someone walks near it. The question I skipped last time is who is walking near this one, and how close. Condition Is Not Risk Engineers separate two things that ordinary language runs together. Years after the exhibition hall, I spent part of my career in transportation consulting. One project, with colleagues at Cambridge Systematics and Lloyd's Register, built a risk model for the bridges on the US highway system. Every state reports the condition of its bridges to a national inventory, and we used that data to place each structure on a five-by-five grid. One axis was condition: how structurally sound the bridge actually is. The other was consequence: what happens if it fails, largely how much traffic crosses it, adjusted for how far the detour would be if it closed. Those two axes aren't the same thing, and keeping them apart is the point. A badly deteriorated bridge on a farm track is in poor condition and carries almost no consequence. A sound bridge carrying a hundred thousand vehicles a day is in fine condition and carries enormous consequence. Neither is where you look first. You look at the corner where both are bad at once: a deteriorating structure carrying heavy traffic with no easy detour. Risk is what you get when condition and consequence coincide. The structure's condition is a fact about the structure. It is true whether anyone drives across it or not. The risk exists only because people do. That is the distinction I skipped past five weeks ago. An optimizer without an independent evaluator is a fact about how the thing is built. It is a condition. It is true of the system sitting on its own, doing nothing at all. What turns a condition into a risk is traffic. Two Ways To Hand Something Over So: who is driving over this bridge, and how heavily? Not “who uses AI.” Everyone does, and that tells you nothing. The question is narrower. What happens to your own ability to check the answer? Handing work to something else is not one thing. It comes in two forms that look identical from outside and behave very differently when something goes wrong. The first is delegation. You give away the task and keep the ability to judge what comes back. A managing editor does not write the article, but can read it and say it isn’t working. I gave the stress calculations to I-DEAS and kept, without ever thinking about it, a way to test the answer. The work moved. The judgment stayed. The second is offloading. You give away the task, and the ability to judge the result goes with it. What comes back is a conclusion, arriving without the reasoning that produced it, and there is nothing left on your side that can weigh it. You can accept it or refuse it on instinct. You cannot check it. From outside, these look the same. A person asks, a system answers, the person proceeds. The difference is invisible until the answer is wrong. The obvious objection is that delegation is just offloading on a longer timescale, that if you never do the work yourself, the ability to judge it must eventually go. That objection is correct, and it is the most important thing in this essay. Delegation is not a stable condition. It decays into offloading unless something actively holds it in place. This raises the question of what could hold it there, and whether anything does. Bainbridge, 1983 The obvious answer is that this is a matter of discipline. Keep your hand in. Check the work. Lisanne Bainbridge answered that in 1983, in a paper on industrial process control written long before any of this existed. Her observation came to be called the ironies of automation. When you automate a process, you leave a human to monitor it and take over if it fails. But monitoring something that runs smoothly is exactly the condition under which manual skill decays. The operator does less, for longer, and the ability to intervene erodes quietly while nothing appears to be wrong. Then the automation fails, which is the moment it needs a person, and the person it gets is the one whose skills have been eroding for years, worn down by the very smoothness that made everything look fine. The skill you need at the moment of failure is precisely the skill that the absence of failure has been taking away. So the decay isn't hypothetical, and vigilance isn't the answer. Which leaves the question of what is. What The Napkin Actually Was Here is where I have to be careful, because the obvious reading of my own story is wrong. The obvious reading is: hold on to the old skill. Learn to do it by hand so you can always do it by hand. And that cannot be the lesson, because it does not survive contact with how anything works. I cannot fabricate a semiconductor, sequence a genome, or derive the cryptography my bank runs on. I use all three every day, and I should. If the answer to automation were personally retaining the underlying method, the answer would be useless. Look again at what I actually had that night. It was not the ability to reproduce what I-DEAS did. The package solved a system of equations across thousands of elements. I could not have done that by hand in a month, and nothing in my training suggested I should try. What I had was cruder and far more valuable: a way to arrive at an approximate answer by a route that did not pass through the package. That is the whole thing. Not reproduction — verification by an independent route. Which gives the test: Is there a way to check the output that does not run through the thing that produced it? Sometimes that route is retained expertise, as mine was. But it can be many other things. An empirical test the world adjudicates. A formal constraint the output must satisfy. A second system built on genuinely different principles rather than a near-copy of the first. An audit trail that lets someone reconstruct how a conclusion was reached. A deliberately adversarial process staffed by people whose interests differ. The napkin is one instance of the category. It is not the category. And that changes what is at stake. The scarce resource is not craftsmanship, and this is not an argument about the dignity of doing things by hand. What is scarce is independent routes, and unlike a bridge, a route can disappear without anyone noticing. There is a collective version of this, and it is worse. My napkin worked because it was a different method. Diffe

  6. Sep 8

    Turtles All The Way Down

    A scientist finishes a lecture on the solar system. An elderly woman in the audience stands up and tells him the whole account is wrong: the world rests on the back of a giant turtle. The scientist, politely, asks what the turtle is standing on. She has the answer ready. It is turtles all the way down. The story is usually told about Bertrand Russell. It is also told about William James, and about various astronomers whose names change with the telling. Each version reaches us through someone who did not say where they got it. Nobody has established which lecture hall, or which century, or whether it happened at all. The provenance of the story about infinite regress has no bottom either. Thanks for reading James Maconochie | Architecture & Attention! Subscribe for free to receive new posts and support my work. The Number I Was Holding Up For several years I worked for a consulting firm that ran a Delivery Excellence dashboard. Every project reported weekly against a set of components: scope, schedule, budget, client satisfaction, team performance, and one or two others I no longer remember. Each one got a color. I entered the scores myself. On a program for a biotech client, I reported client satisfaction as green, week after week, and the score was accurate. The client contact was satisfied. Every concern he raised got addressed in the order it arrived. He was told early that the scope began to shrink because we could not deliver all of it, and hearing it early is most of what makes a shrinking scope tolerable. Underneath that, a team was struggling on several counts, and had been almost from the start. So the numbers were true. Scope was green because we had already cut scope. Satisfaction was green because I spent a substantial part of every week producing it. Every component reported a state, and none had a field for what that state cost to maintain. The effort was the risk, and the effort was the one thing the format could not carry. I did not falsify anything. I reported what the components asked for, and I raised the delivery problems internally, every week, in the space the form left for them. The technical lead and the others read the situation as managed, and I did not want to go out on a limb by myself against my own team’s assessment. I also thought that if we all knew, we would fix it before it came due. So a room full of people who each knew something was wrong went on filling in a form that had nowhere to put it. I have watched a version of this happen more than once, and never once because anyone was hiding anything. What eventually surfaced the problem was not the dashboard. In January, I sat down with the office general manager and argued that we should put three hundred and fifty thousand dollars of our own money into a three-and-a-half-million-dollar job to fix what we had not delivered. The technical lead and two others backed the position. The technical lead told him the only reason we were still in the room was the account management. He agreed to the money. We spent it carefully, the work landed, and the client was pleased in the end. The dashboard recorded none of this. It had been green throughout, and it was green because of what I was doing underneath it. The Tower We Are Building Now The dashboard failed in a specific way, and the way matters more than the failure. It published an accurate number and gave nobody access to what produced it. Whoever read the green had to take on trust that the process behind it was sound, and the process behind it was me. Reported figures come loose from the work that generated them. That is not a property of consulting. It is a property of reporting. A model’s output is evaluated by another model, because human evaluation does not scale. The evaluating model was trained on data that a model filtered for quality. Both are scored against a benchmark, and benchmark content has repeatedly been found in the training data of the systems being scored. The results are then summarized for the people who have to decide something, increasingly by a model, because the volume is large and the summary is fast. Every one of those steps is defensible on its own. Each is documented, published, and argued for by people who know more about it than I do. None of them is hiding anything. What is missing is not honesty. It is anyone whose job is to ask what the bottom layer is standing on. Each layer treats the one below it as ground, and the question of grounding gets handed down the stack until it runs out. In Defense of Turtles I want to concede this properly rather than in passing, because the concession is most of the argument. Abstraction is how anything large gets built. Nobody writing an application re-derives the behavior of a transistor. Proxies are how you manage what you cannot measure directly, and almost everything worth managing cannot be measured directly. I have never inspected a bridge before driving over it, and I am not going to start. Stacking is not the sin. A working tower of abstractions is one of the more impressive things people do. The engineer who trusts the steel certificate is not being lazy. That certificate is doing real work, and behind it there is a mill, a test, a standard, and a name. The failure is narrower than the stacking. It is losing track of whether there is a bottom at all. Deferring the question of grounding to the layer below is fine, and unavoidable, as long as the deferral terminates somewhere. The tower becomes a problem only when every layer in it has been built on the assumption that somebody further down already checked. Where It Stops The regress has to stop somewhere, and the only place I have ever seen it stop is at someone who pays if it is wrong and cannot hand the paying to anyone else. I want to be careful about how much that carries. It is not a guarantee of anything. People with their names on things sign off on work they have not understood. Systems with clear lines of liability fail badly and often, and the person carrying the cost is frequently the last to see it coming. The engineer who signs the beam did not derive the metallurgy, watch the testing, or stand in the mill. Finding the name does not tell you the answer is right, and it does not mean the person can check the work. It tells you the chain has a bottom instead of continuing indefinitely. The claim is that narrow: without someone in that position, there is no bottom to have. I have spent most of this series on that claim, and I am not going to re-argue it here. The engineer who signs off on the beam carries it if the beam fails. The certificate is not the ground, and the signature is not the ground. The ground is that a person’s license, career, and name are attached to the load, and that the attachment cannot be undone by delegating. On the biotech program, the ground floor was not the dashboard. It was a room in January, a general manager who had to spend money he would answer for, and an account lead recommending it who would answer for it too. Neither of those costs could be passed sideways. That is why the problem got fixed. It is also why it took until January. What You Can Use The woman in the story was wrong about cosmology and right about almost everything else. So here is the test. Pick any validation you currently rely on. A green status, a certification, a score, a rating, a benchmark result, a maturity level. Ask what it is standing on. Then ask the same question of the answer. Keep going until the answer stops being a process and starts being a name. If you work down through the whole stack and there is nobody there, you are standing on a turtle. It may well hold. A great many of them do, for years at a time. But you should at least know that is what you are standing on. I stood on one for most of a year, and I entered the number myself every week. Thanks for reading James Maconochie | Architecture & Attention! Subscribe for free to receive new posts and support my work. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit jamesmaconochie.substack.com

  7. Sep 1

    The Problem Nobody Set

    There is a piece of work your body performs every waking second, and you will almost certainly never think about it, unless one day it stops. I injured my lumbar spine. What I learned afterward was not really about pain, which is the part everyone expects and the part I will spare you. It was about administration. Somewhere well below anything I could call thinking, my brain had been issuing a continuous stream of instructions to the small stabilizing muscles around my spine, adjusting them constantly, correcting for every shift of weight and every turn of my head, every hour I had been alive. Standing still turns out not to be still. It is a controlled fall, caught and re-caught, in every direction at once. Thanks for reading James Maconochie | Architecture & Attention! Subscribe for free to receive new posts and support my work. I never chose that work. I never noticed it. I became aware of it only when it started arriving late and imprecise, and the ordinary business of getting up from a chair became something I had to attend to deliberately, the way you attend to a difficult sentence. Here is the part that took me longer to see. The work was not executing a standing instruction. Nothing in me was running an order that said stay upright. At every instant something was deciding which of an enormous number of signals mattered right now, and therefore what the problem currently was: a floor that turned out to be wet, a bag heavier on one side than expected, a step that arrived sooner than the last one. The instruction never changed. The problem changed constantly, and something reformulated it, continuously, without asking me. Nobody set that problem. Something I have never met noticed it and re-noticed it thousands of times an hour. The Ledger We Keep Of Difficulty Hans Moravec noticed the consequence of this in 1988, in a book called Mind Children, and the observation now carries his name. It is comparatively easy, he wrote, to make computers perform at adult level on intelligence tests or at checkers, and difficult or impossible to give them the skills of a one-year-old at perception and mobility. The paradox is normally presented as a fact about machines. I want to present it as a fact about us. We rank difficulty by how hard a thing feels while we do it. Chess feels like intellect. It is effortful, available to introspection, and we can watch ourselves doing it, so it sits at the top of the ledger. Catching a set of keys tossed across a room feels like nothing at all, so it sits nowhere. The ranking is not a judgment about the problems. It is a report on which problems we happen to be conscious of solving. Let me be careful about how much weight that carries. I am not claiming effort measures unfinished learning. Expert musicians work hard at passages they have played for decades, vigilance is exhausting after thirty years of practice, and creative work stays effortful precisely because there is no stable routine to settle into. The claim is narrower: felt effort is an unreliable proxy for computational difficulty, and we have been using it as our main one. An objection deserves to arrive here rather than later. The computer scientist Arvind Narayanan has argued that Moravec’s paradox is not a dependable predictor of what will prove easy or hard for machines, and is better read as a statement about what the field has found worth working on. I think that is largely right, and I am not going to use the paradox as a forecast. Whatever it does or does not predict, it correctly identifies a bias in our accounting. That informal ledger has a formal descendant, and it now runs the industry. We call them benchmarks. A benchmark is our difficulty ledger written down and made scoreable, and it inherits the same property: it can only contain what somebody thought to put in it. The Objective Was Never the Hard Part Let me concede the thing that arguments of this kind usually avoid. Football has an exceptionally clean objective. Score more goals than the opposition inside the allotted time. You can write it on a napkin. There is no ambiguity about success, and a scoreboard settles it. If difficulty lived in the difficulty of stating an objective, football would be among the simplest things humans do. So the difficulty is not in the objective. It is in the distance between having one and knowing what to do next. Watch what that distance contains. Twenty-two people are running the stabilization work I described at the start of this essay, continuously, while sprinting, turning, being shoved and landing badly. Air moves across the stadium in ways nobody has modeled. The surface changes through the match. The light shifts. The altitude sets what their bodies can spend. And each player is reading the intentions of everyone else on the field while being read in return. A pass is a prediction about where a teammate will be in two seconds, made by a body in mid-fall, against defenders making their own predictions about that prediction, who will change what they do the moment they detect it. I want to be precise about what the players are doing, because there is a mystical version of this claim and I do not want it. They are solving the problem. Obviously they are; the ball arrives. What they cannot do is say how. The computation is distributed through the body, learned over years, and closed to introspection, which is different from absent. But the deepest part is not the policy either. It is that the problem does not hold still. Somewhere around the hour mark, a match becomes a different match. The referee has been letting contact go, so the cost of a challenge has changed. Someone is carrying a knock and is now a target. The other side has gone to three at the back, and every space on the pitch means something new. Nobody announces any of this. No player is told the problem has changed. Each one works it out while still playing, revises what matters, and acts on the revision before being able to articulate it. It is worth being exact about what changed, because adapting to a moving situation is something good control systems already do well, and I do not want to dress that up as something rarer than it is. What changed was not the difficulty. It was which information mattered to success. A defender’s body position meant one thing at twenty minutes and means something else now, and nothing in the original instruction said so, or could have. That is the line I want to hold throughout this essay. Adjusting your behavior because conditions moved is one thing. Discovering that a different set of facts has become the relevant ones is another, and it is the second that nobody set. The objective did not move. It was to score more goals in the first minute, and it is to score more goals in the ninetieth. What moved was everything about what that requires. Take Away The Opponent It would be easy to conclude this all comes from the adversary, so it is worth removing the adversary and seeing what remains. Ballet has no opponent in the relevant sense. Nobody on that stage is trying to make a dancer fall. I should concede at once that she faces an evaluator. Two thousand people are watching and judging, and their judgment is opaque, plural and impossible to reduce to a number. That is real pressure, and I have not removed all of it. What I have removed is the thing football is usually assumed to be hard because of. And the difficulty does not drop. She is running the same continuous stabilization work at the limit of what a skeleton tolerates, on a floor with its own particular give, in time with music played live and therefore slightly differently tonight, alongside other dancers whose small variations she absorbs without appearing to absorb anything. And through it all, she is deciding what tonight’s performance is. Not the choreography, which is fixed and which she knew months ago. Suppose her partner arrives a fraction early and the lift no longer sits where the phrase wants it. She has perhaps half a second, and the choice is not how to execute the lift better. It is whether tonight the moment is the lift at all, or the recovery out of it, and she has to decide which one the audience is now watching before she can have any view about how to do it well. Nothing in the choreography addresses that, because the choreography could not have known her partner would be early. The choreography is the specification. It is complete, and it is not the performance. Before I go on, a correction to an impression these two examples will have created. I chose a World Cup final and a stage because they are legible, not because the capacity is rare. Nothing in the argument depends on excellence. The same continuous re-noticing runs in an aide steadying someone out of a chair, whose sense of what this particular person can bear today was never written into any care plan, and in a warehouse picker rerouting around an aisle that is blocked differently than it was an hour ago. The elite cases are the ones we buy tickets for. They are not the ones where the capacity lives, and if anything, the fact that we pay handsomely for the visible version and almost nothing for the ordinary one is its own small piece of evidence about the ledger. What A Measurement Can See Take drone light shows, which are the cleanest case available. Thousands of aircraft, physically present, holding formation in real wind. Every objection about digital versus physical dies on contact with a drone show. The lazy version of this argument is wrong, so I won't make it. Those aircraft are not blindly following fixed paths. Modern autonomous systems execute policies rather than trajectories: wind arrives, and the aircraft recomputes what to do to stay where it needs to be. That is real adaptation and real engineering. Notice what is nonetheless fixed. The image was decided before anything left the ground. Nothing in that system is built to ask whether the picture is still the

  8. Aug 24

    The Kindest Thing Pain Does

    There are people whose bodies have to be checked at the end of the day. Not out of anxiety, and not because anything feels wrong. The checking happens because damage arrives without announcing itself, and looking is the only way to find it. A burn that never registered. A cut that has been open for hours. A joint that has moved further than it should and stayed there. In childhood, a parent does the looking. Later it is the person themselves, or a clinician, or both. Either way, the inspection is deliberate, scheduled, and conscious, and it stands in for something most of us are handed without asking: a signal that arrives on its own at the moment of the damage, whether or not we happen to be paying attention. The condition is congenital insensitivity to pain, and it is rarer and less tidy than the name suggests. Thanks for reading James Maconochie | Architecture & Attention! Subscribe for free to receive new posts and support my work. It is not one condition. Several genetic routes lead to it, and I am going to take two of them, not because they exhaust the picture but because the contrast between them is useful. In one, a gene called SCN9A encodes part of a sodium channel that pain-sensing nerves need to send their signal. When both copies carry disabling changes, that signal does not get through. This route comes closest to a clean experiment because most other sensations remain. Touch works, or nearly: testing has found some differences in how touch is felt at the margins. Position works. The known exception is smell, because the same channel is used by the nerves in the nose, and people with this form are typically unable to smell. The second route runs through a gene called NTRK1 and produces what is usually called congenital insensitivity to pain with anhidrosis. Here, more is missing. Along with pain, the ability to sweat is absent, which leaves the body unable to cool itself. Sensitivity to temperature is affected. Intellectual disability of varying degree is characteristic. This is the form where the confounds are worst. I want to be careful here. The injuries are documented, and they recur across both routes: biting of the tongue, lips, and fingertips, sometimes severely; damage to the eye; fractures walked on and healed badly; joints worn down; bone infections that took hold because nothing reported them early. What I cannot honestly tell you is that this accumulation is what shortens life. In the second form, early deaths are substantially driven by fever the body cannot shed, which follows from the missing sweat rather than from the missing pain. Reported mortality varies across the groups studied, and I am not going to hand you a figure that sounds more settled than the evidence warrants. So the narrow claim, the one that survives both routes and all the confounding, is this. The injuries accumulate because the signal that normally reports damage as it happens is missing. Living with this condition means substituting a deliberate audit for a signal that would otherwise have arrived on its own. Which is worth sitting with because it reveals the pain underneath the flinch. It is not sensation for its own sake. It is a feedback loop rendered in flesh. It takes a bad outcome and converts it into something carried, something the body cannot set down, something that reaches forward and changes the next reach of the hand. Three weeks ago I described the fast loop that living things run inside a single life: the animal that touches the fire pulls back, carries the burn, and approaches the fire differently tomorrow, the same animal, changed. Pain is part of how that sentence gets implemented. It is the part of the mechanism in us that makes a burn teach. Knowledge tells you the stove is hot, and knowledge is not nothing here. People with this condition learn a great deal, and much of what keeps them safe is learned rather than felt. What knowledge does not do is bind the lesson to the moment on its own. A person with this condition can see the flame, know the word, and explain to you exactly why fire is dangerous. What is missing is the small insistent return that says this one cost you something, arriving without having to be asked for. One Word For A Lot Of Different Things And here I have to correct something in my own vocabulary. Across these essays I have said that consequence returning to the actor is what turns knowledge into wisdom, and I have said it as though consequence were a single substance arriving in varying amounts. It is not. Being embarrassed in a meeting, breaking a bone, losing a reputation, and watching an institution fail are not four sizes of the same thing. They arrive at different speeds. They reach different distances. Some can be argued with and some cannot. Two of my own, for contrast. Earlier this year, while driving for Uber, I hit a parked car. Low sun, a dark grey car on the shaded side of the road, and a fraction of a second spent looking down to change the radio station. The airbag went off. The smell of the powder is the detail I have kept. I called the emergency services, then my wife. She came round the corner from the house, where her parents were loading the car for the drive back to Maine. The damage looked limited to me. The insurer wrote the vehicle off. What that taught me is narrow, and it is permanent. I am careful about parked cars on my right, leave more clearance than I used to, and have not had to decide on either since. What it never told me was which part of what I did mattered. The sun was real. The radio was real. I cannot separate them, and I have stopped trying. The cost arrived, the behavior changed, and no account came with it. The other one cost a great deal more and taught a great deal less. Seven or eight years before that, I was the account lead on a large data program for a retailer. Every two weeks we prepared slides for an executive update. Every two weeks, the client's project lead edited down or removed the risks in those slides before the meeting. My colleague and I both understood what was happening. We had other problems at the time, in staffing and in delivery, and we chose not to force it. So the meetings went ahead twice a month without the real state of the work being included. The problems did not go away. After about six months, they were obvious to everyone, and we were left explaining that we had known and had deferred to the client on the contents of the client’s own slides. We were asked to stop work. It took another six months to recover a seven-figure final payment, and it nagged at me for all of them. Worth noting where that landed. The commission on the sale had been paid long before delivery started, to someone who was not in the room for any of this. The people holding the work bore the cost. Ask me what that changed, and the honest answer is: a little, perhaps. Because I could argue my position, and I did, and it was not a bad position. That is the thing worth noticing. The slow arrival came with a complete account attached, and an account can be reasoned with, and reasoning with something is most of the way to setting it down. The expensive one taught less than the cheap one. What those two suggest to me is that pain sits at one end of a range. It arrives during the event rather than after it; it lands on the body that acted, and it comes with no reasoning you can take issue with. Embarrassment is social and fades. Financial loss arrives as a document, and a document can be answered. Collapse arrives once, by which time the people who might have learned from it are rarely still in the room. What This Comparison Can Carry, And What It Cannot It would be neat to say that these machines, by which I mean here and throughout this series the AI systems many of us now talk to daily, are painless bodies grown large. I don't think that holds, and I would rather say so than hope you didn't notice. Someone with the first form I described, the one that leaves most other sensation intact, is missing one channel out of an otherwise working set. They can still be embarrassed. They still learn from outcomes, and they learn protective rules well enough that the rules become much of what keeps them safe. What is absent is a single protective signal inside a loop that is otherwise running. The absence in these systems is larger. They do not carry a correction forward as the same continuing thing; the durable fixes are made to later versions, by processes outside the system itself. This is not a small version of that. It is a different absence, in a body that is not otherwise like them. There is a harder objection beneath the surface, and I would rather take it up than leave it for you to find. People live with this condition. Not easily, and not without cost, but they manage it through rules, inspection, treatment, and the attention of other people. If knowledge and procedure can stand in for the missing signal, then perhaps consequence was never what mattered, and what matters is having a working system for finding and fixing errors. I think that is right. It sharpens the question rather than ending it: what exactly was the signal supplying that now has to come from somewhere else? For my purposes, separate four jobs that event was doing at once. It registered that something harmful had happened. It made that hard to ignore. It tied the harm to the moment it occurred. And it applied pressure on what the hand did next. Note the third one, because it is smaller than it looks. Pain ties damage to the moment, not reliably to the cause. My own collision is the proof. Something arrived that changed my driving permanently and never told me whether the sun or the radio was the problem. Working out what actually caused a failure is one of the things a deliberate process does far better than a body does. Take the signal away, and none of those four becomes impossible. Each becomes something that has to be built. But what goes missing is n

See All (50)

About

Essays on Augmented Human Intelligence, the Wisdom Gap, and the architecture of attention in an AI-mediated world. Read in James Maconochie's own voice. jamesmaconochie.substack.com

Information