James Maconochie | Architecture & Attention Podcast

James Maconochie

Essays on Augmented Human Intelligence, the Wisdom Gap, and the architecture of attention in an AI-mediated world. Read in James Maconochie's own voice. jamesmaconochie.substack.com

  1. 1d ago

    Prevention Has a Timing Problem. So Does Everything Else.

    In a previous essay I argued that Pope Leo XIV’s encyclical on artificial intelligence did something remarkable and then walked away from it. It reached the architectural diagnosis almost no one reaches, that displacement is a choice made at the point of deployment, not a fact of nature descending on the labor market, and then it turned downstream, toward retraining and transfer and oversight, toward managing the consequences of the choice rather than contesting the choice itself. I argued that this turn was not cowardice but gravity: the same pull that takes the state toward the transfer, the market toward faith in growth, and the moral authority toward the language of repair. The default is the slope of the ground. Even the best diagnosis slides down it. I ended that essay with a question I did not answer, because I do not think it has an easy answer. If the gravity is real, can it be resisted? Is the alternative the diagnosis implies, configuring deployment so the worker is augmented rather than replaced, actually buildable, at a cost we would pay, and quickly enough to matter against a technology that moves in months? Thanks for reading James Maconochie | Architecture & Attention! Subscribe for free to receive new posts and support my work. This essay is my attempt to reckon with that question honestly. I will tell you now that I do not arrive at a clean victory. I arrive somewhere narrower and, I think, truer. The Objection That Should Worry Me Here is the strongest case against everything I have argued, and I am going to put it more forcefully than a hostile reader would, because if it stands, the rest of this is decoration. The alternative I am calling for takes time the displaced do not have. My own best example proves it. When I want to show that deployment can be configured to keep the human in the loop, I reach for radiology, a field where the AI arrived as augmentation rather than replacement, where the radiologist still reads, judges, signs, and bears the consequence. But radiology did not get that way by accident or overnight. It took half a century of medical liability law, professional standards, credentialing, and reimbursement structure to build the scaffolding that made augmentation the rational deployment. Fifty years. Now look at the work actually being hollowed out right now. Customer service. Copywriting. Transcription. First-pass legal document review. Entry-level analysis. None of it has radiology’s scaffolding. None of it has the liability shield, the licensing board, the standard of care, the reimbursement code. And it is not being displaced over fifty years. It is being displaced over fiscal quarters. Or so the story goes, and I should say plainly the story is contested. The pace of displacement is partly real and partly the marketing department’s. The gap between what these systems are sold as doing and what they reliably do in production is wide, the promised productivity gains keep arriving late and uneven, and some of what looks like a hollowed-out department is a pilot that still leans on the humans it was supposed to replace. A careful skeptic is right to demand the denominator before accepting that whole categories of work are vanishing in months. I take the point, and I notice it cuts in a direction that helps me rather than hurts: if displacement is slower and narrower than the hype insists, then prevention has more runway than the worst case allows, not less. So let me grant the objection its strongest form anyway, because the argument should survive it. Suppose the displacement is as fast as the alarmists say. Then what? This is the asymmetry that should keep more of us up at night than it does. Cure can be deployed after the fall, you can stand up a retraining program or a transfer payment once the job is already gone. Prevention cannot. Prevention has to be poured like a foundation, before the building goes up, before the displacement happens. And if the configuration that would protect a category of work takes years to build while the displacement of that work takes months, then the window I keep invoking is not open. It closed before I finished describing it. For the people most exposed, I am offering a foundation for a house that has already burned down. I want to be honest: this objection genuinely worries me. It is not a strawman. It is the thing I have to answer before I am entitled to any of the hope in the rest of this piece. What Prevention Is Not Before I try, I have to clear away the version of this argument that deserves every bit of scorn it gets. There is a reading of “prevent displacement” that means: force companies to keep workers doing jobs a machine could do more cheaply, freeze the org chart, hold the economy in amber against the tide of productivity. That reading is economically incoherent and I am not making it. You cannot order a firm to employ people to do nothing, and you should not want to. A serious economist will tell you that technological change displaces labor, that it always has, and that the humane question has always been whether we manage the transition well or badly. On that, the economist is right. But that is not the choice I am pointing at, because it is not the only choice on the table. The displacement debate keeps collapsing two different things into one. One is whether the work gets more productive, which it will, and should, and no one sane is trying to stop. The other is what shape the productivity gain takes: whether the technology is deployed to augment the worker and strip the drudgery from her day, or deployed to remove the worker and keep the wage she used to earn. Both are productive. Both capture the gain. They are not the same deployment, and the difference between them is not dictated by the technology. It is chosen. Prevention, as I mean it, is not the refusal of productivity. It is the contest over which of two equally productive configurations gets built: the one that keeps the human in the loop, or the one that empties the loop out. Anyone who tells you that contest isn’t real, that only one configuration was ever economically available, is smuggling the conclusion into the premise. Cure Isn’t Built Either So let me start by noticing what the objection quietly assumes. It assumes that cure is ready and prevention is not, that on one side of the ledger sits a functioning safety net, waiting, and on the other sits my hypothetical foundation that takes too long to pour. Time the two against each other and prevention loses. But cure is not built either. Consider what cure actually requires to work, not to be announced, but to work. Retraining has to take a forty-five-year-old customer service representative and return her to comparable income and comparable dignity in something other than a worse job. The evidence that retraining programs do this is, to put it gently, poor; decades of trade-adjustment and workforce-retraining efforts have a track record that ranges from modest to dismal. The infrastructure that would do it well does not exist at scale. It would have to be built. Or consider the transfer the encyclical leans toward, the social protection, the redistribution, in its strongest form a universal basic income. None of that is built. It is not funded. It is not politically coalitioned. Standing up an income-transfer regime large enough to absorb mass displacement is at least as slow, at least as institutionally demanding, and at least as politically captured as anything I am proposing on the prevention side. The Pope’s own remedies are a fifty-year project that no one has broken ground on. So when the defeatist holds a stopwatch to prevention, fairness requires holding the same stopwatch to cure. And when you do, the comparison stops favoring cure. I am not going to overclaim here, I am not going to tell you prevention is obviously faster, because I do not know that. The real claim is narrower and it is enough: the speed objection, applied evenhandedly to both sides, is not a reason to prefer the safety net. Both the net and the foundation have to be built, both are slow, both are contested. The only question left is which one is worth starting, and “it’s too late for prevention” cannot be the answer when cure is exactly as unbuilt. Radiology, Honestly Let me give the critics their due on radiology, because they are right about it, and conceding that is the only way to extract the lesson that survives. Radiology is the slow case. It is the high-liability, high-status, heavily regulated, professionally fortified case. It is, in almost every respect, the least representative of the work AI is displacing fastest. If I lean on it as proof that prevention is easy, I deserve the truck that gets driven through the argument. A copywriter has no equivalent of the FDA. A transcriptionist has no standard of care. Pointing at radiology and saying “see, it can be done” is like pointing at a cathedral to prove that anyone can put a roof over their head. So I will not claim radiology’s timeline transfers. It doesn’t. You cannot grow an entire profession’s regulatory edifice in the time you have. But that is the wrong lesson to draw from it, and the critics stop one step too early. Radiology is not valuable as a timeline. It is valuable as an anatomy. It shows you what scaffolding actually is, disassembled into parts: a liability rule that names a specific human as accountable when the automated decision is wrong. A standard that defines what competent practice requires. A gate that governs what the software is allowed to decide on its own. A payment structure that funds the human-in-the-loop rather than penalizing it. Those are the load-bearing elements. And here is the thing the fifty-year objection obscures: most of that half-century was spent building the profession, not the configuration. The liability principle, the accountable-human rule, the single most important piece, is not a

    25 min

  2. 4d ago

    Pope Leo Found the Cause. Then Gravity Took Over.

    This is the first of two essays on Pope Leo XIV's encyclical Magnifica Humanitas. Part 2 lands this Friday rather than next Tuesday; the argument doesn't survive a week's gap. When the most authoritative moral document ever written about artificial intelligence arrives at the same structural conclusions you have been arguing toward by an entirely different road, the temptation is to declare victory and stop reading. I want to resist that because the more interesting thing about Pope Leo XIV’s first encyclical is not where it agrees. It is where it stops. Thanks for reading James Maconochie | Architecture & Attention! Subscribe for free to receive new posts and support my work. Magnifica Humanitas runs to roughly forty-two thousand words and reasons from a hundred and thirty-five years of Catholic social doctrine, beginning with Leo XIII’s Rerum Novarum and its defense of the worker in the Industrial Revolution. It is not a technology document dressed in vestments. It is a moral anthropology that happens to take AI as its occasion. And reasoning from that tradition, from premises that have nothing to do with the vocabulary of systems design, it lands on three claims that anyone who has thought structurally about AI deployment will recognize at once. The first is that AI does not bear consequences. The encyclical is blunt about it: these systems do not undergo experience, do not mature through relationships, and do not, in the Pope’s framing, judge good and evil or carry responsibility for what follows from their outputs. They imitate; they do not answer for the result. The second is that systems are being built the wrong way around, designed so that workers must adapt to the speed and demands of the machine, rather than the machine being designed to support the people who work. The third is what the encyclical calls the “architecture of visibility”: platforms engineered to capture attention, amplify what is visible, and shape opinion, treating the finite human capacity for attention as a resource to be mined. Consequence-bearing. The design of work. The capture of attention. An independent line of moral reasoning, starting from the dignity of the human person rather than from any analysis of deployment, arrived at an architectural diagnosis. That convergence is worth naming plainly, not because it flatters anyone, and not because everyone serious sees it this way. Plenty of capable people read AI primarily as a productivity expansion and think the displacement worry is overstated. The convergence I mean is narrower and more striking for it: two arguments built from unrelated foundations, one theological and one structural, traced the problem to the same place. That is some evidence that the diagnosis is structural rather than idiosyncratic. The Word That Matters The encyclical’s sharpest move is a single word. Leo writes that technology is never neutral, because it takes on the characteristics of those who devise, finance, regulate, and use it. And he writes that the pursuit of greater profit cannot justify choices that systematically sacrifice jobs. Choices. Not consequences. Not the weather. Not an impersonal force descending on the labor market like a season. The encyclical locates the problem at the point of deployment design, in the decisions made by identifiable people about how the technology will be configured and to what end. This is the thing most commentary on AI never reaches. The dominant register treats displacement as a fact of nature: capability accelerates, jobs evaporate, and the only remaining question is what to do for the people left behind. Leo refuses that. He sees that the displacement is authored. It could be authored otherwise. That is the high-water mark of the document. Having reached it, watch where the remedies go. The Turn The encyclical’s concrete recommendation is this: regulate the algorithms. Retrain the displaced. Use taxation, social protection, and industrial policy to ensure equity. Renew labor organizations. Entrust international oversight to a reformed United Nations. Every one of these operates after the displacement has occurred. Retraining presumes the job is already gone; it is a response to the worker who has been turned out, not an intervention in the decision to turn him out. Social protection presumes that the income has already been lost. Taxation and transfer presume that the displacement has occurred and that the proceeds are now being redistributed. International oversight watches outcomes. The diagnosis pointed upstream, to the choice. The prescription walks downstream, to the damage, and builds an apparatus to manage it. The document names the cause and then treats the symptom. An Ounce of Prevention There is an old proverb for exactly this. An ounce of prevention is worth a pound of cure. Prevention acts on the cause before the harm. Cure acts on the damage afterward. Everything in Leo’s recommendation list is a cure: retraining cures the loss of a job, a transfer cures the loss of income, and oversight cures the worst outcomes once they have appeared. The prevention, the ounce, would act at the configuration itself, at the moment the deployment is designed, before anyone is displaced at all. It would treat the choice the encyclical so clearly named as a choice still open to be made differently, rather than a settled fact to be cushioned. The document does not go there. It finds the cause, names it precisely, uses word choice, and then turns to managing the consequences. This is not a small omission. It is the difference between asking whether the worker had to be displaced and asking what we owe him once he has been. The first question is the one the diagnosis demanded. The second is the one the encyclical answered. The Same Error, Left and Right You might read that turn as a peculiarly Catholic failure of nerve, or a personal limitation of this particular Pope. It is neither. Watch what happens when the document meets its critics. The encyclical’s sharpest critics on the right found the diagnosis welcome and the statism alarming. They attacked Leo for misplaced faith in government, arguing that regulation concentrates power and that the market, left alone, has always eased the worker’s burden over time. Strike the remedies, they said, and trust the diffusion of technology to lift living standards as it always has. This is the mirror image of the same error. The Pope wants to compensate for displacement through the state; his critics want to absorb it through the market. Neither asks whether the displacement had to happen. They are having a furious argument about the size of the net while standing under the same assumption: that the fall is inevitable and the only question is what catches it. The left wants a larger net; the right wants a smaller one and faith that growth will fill the gap. Both treat the deployment configuration as a given. That is how durable the assumption is. It survives translation into Catholic social doctrine and into free-market editorializing entirely intact. It does not belong to a political camp. It is the path of least resistance for anyone who would rather not get inside the deployment decision, which is nearly everyone, because getting inside it is the ounce, and arguing about the net is the pound. The Gravity It is worth being honest about what this means, because it runs counter to the easy version of this essay: the one where a brave writer catches the Pope in a failure of courage. The encyclical did not fail for lack of insight. It reached the architectural diagnosis that almost no one reaches. It named the cause with a precision most secular commentary never manages. And then it turned downstream anyway, not because Leo could not see the upstream question, but because the downstream answer is the one that everything pulls you toward. The state reaches for the transfer. The market reaches for the long-run faith in growth. The moral authority reaches for the language of compassion and repair. Each of them, reasoning carefully from its own commitments, ends up cushioning the fall rather than questioning it. That convergence is the real finding. When the most authoritative moral voice in the world on this subject, a free-market editorial board, and a redistributionist politician all independently arrive at the same place, manage the consequences, and do not contest the choice, you are no longer looking at anyone’s particular blind spot. You are looking at a gravity. A direction that thought falls into once it stops actively resisting. The default is not an opinion you can argue someone out of. It is the slope of the ground. This is why the Pope’s turn matters more than any individual misstep would. It is the strongest possible demonstration of how steep the slope is. If a tradition built over a century and a third specifically to defend the dignity of the worker, a tradition with no profit motive, no electoral cycle, no shareholders, still slides downstream at the decisive moment, then the pull is not coming from greed, politics, or any of the usual suspects. It is structural. It is in the shape of the problem itself. The Window So I will not pretend the alternative is easy, or that naming the gravity dissolves it. It does not. The ounce remains harder than the pound; that is precisely what gravity means. But there is a difference between a slope you are sliding down without noticing and a slope you have seen. The encyclical’s great service, despite its turn, is that it climbed high enough to make the upstream question visible. It found the word choice and set it down where everyone could see it. That the document then walked past its own discovery does not unmake the discovery. The cause has been named, by an unimpeachable source, in front of 1.4 billion people. That service is not cancelled by the turn, and the turn is not cancelled by the service. Both are true at once, and the both-ness is

    13 min

  3. May 24

    The Wrong Dystopia

    The Boot and the Feed The dystopia we have been preparing for has a boot on its face. That is Orwell. The state watches. The state coerces. The state lies and tortures. Resistance is meaningful because the regime is meaningful. The hero in 1984 fails, but he fails in the act of trying. The dystopia arriving has no boot. It has a feed and a transfer. The state is mostly absent. Coercion is unnecessary. People are not being forced into the new arrangement. They are being given things until the arrangement is no longer a question. The pressure that built into 20th century totalitarianism does not build because it is absorbed in advance. Thanks for reading James Maconochie | Architecture & Attention! Subscribe for free to receive new posts and support my work. This is a different issue and needs a different approach. The boot you can resist. The feed you scroll. The transfer you cash. The disquiet you cannot quite name. The slow disappearance of something you used to know about yourself, about the place you live, about what you and your neighbors are for. The pattern has been named before, more than once. The first naming was older than the printing press. The second was a novel published seventeen years before 1984. The third is the one we are inside. The First Naming: Bread and Circuses The phrase comes from Juvenal, writing around 100 CE. Satire X. The Roman populace, he observed, had once concerned itself with politics. It had elected magistrates. It had debated policy. It had participated, in some form, in the project of the Republic. By Juvenal’s time, that engagement had collapsed. The populace cared about two things. Panem et circenses. Bread and circuses. The phrase is sharp because the diagnosis is structural. Juvenal is not saying the populace was bribed. He is saying that the state had taken on the provision of two things: daily subsistence through the grain dole, and spectacle through public games. The populace had accepted the trade. The trade was not articulated as a trade. It did not need to be. The agency migrated. The provision arrived. Political participation withered without anyone deciding to give it up. This is the mechanism worth understanding. Rome was not undone primarily by barbarians from outside. The empire that the barbarians eventually breached had already hollowed itself out from within. A citizenry that had traded participation for provision no longer had the muscles to defend the participation when it was needed. The political body had atrophied while the urban body had been kept fed and entertained. The actors in this story are not villains. The emperors who maintained the grain dole were responding to genuine urban hardship. The politicians who funded the games were doing what successful politicians have always done. The populace was not foolish to prefer bread and games to faction and risk. Each step made sense. The cumulative drift did not require anyone to be malicious. It required only that nobody resist the convenience of the arrangement. This is the first naming. The pattern is: a population that holds political power, a state that can afford to provide subsistence and entertainment, and a slow trade that nobody calls a trade. The result is a populace that retains the form of citizenship but loses its substance. Juvenal noticed it because he could see the gap between what the citizens had been and what they had become. The Romans gave us the original vocabulary. The 20th century gave us the second. The Second Naming: Soma and Feelies The phrase comes from Aldous Huxley. Brave New World. Huxley was not predicting the boot. He was predicting something stranger. He was predicting a regime that needed no boot because the population had been engineered to prefer it. The mechanisms in his world are now familiar. There is a drug called soma, distributed freely, which dissolves anxiety. There are immersive entertainments called feelies, which replace depth with stimulation. There is conditioning from before consciousness, which produces people who want exactly what their station permits them to want. What Huxley saw is that none of this requires force. The citizens of his world are not oppressed in any sense they would recognize. They are happy. They do not miss what they have lost because they have been engineered not to know it ever existed. Their happiness is the control mechanism. The pleasure absorbs the pressure that would otherwise produce dissent. The contrast Postman drew is worth keeping. Orwell’s anxiety was about losing access to the truth. Huxley’s anxiety was about losing the capacity to care about truth once enough pleasure was on offer. The first is a problem of suppression. The second is a problem of dissolution. The pleasure makes the truth feel unnecessary, and after a while, it feels beside the point. This is the second naming. The pattern is the same as the Roman one. A population that holds something it does not realize it holds. A system that can afford to provide subsistence and pleasure at scale. A slow trade that nobody calls a trade. The result is a population that retains the form of personhood while losing the substance. Huxley noticed it because he could imagine the gap between what people had been and what they could become if the conveniences were comprehensive enough. The Romans had bread and games. Huxley had soma and feelies. The pattern was the same in both centuries. The question is: what are we giving up in this one? The Third Iteration The pattern is now arriving. The vocabulary is different. The architecture is the same. The transfer is UBI, proposed, partially piloted, and discussed as if its arrival is a matter of when, not whether. The feed is algorithmic, infinite, calibrated to whatever holds attention longest. The Roman state could afford grain and games for the urban population because the empire was wealthy. The modern state, along with the technology sector, can afford UBI and infinite content because the AI economy is generating wealth on a scale that has never existed before. The mechanism is the same as it was: subsistence plus spectacle, provided at scale, by a system that can afford the provision. The contemporary version differs in one structural way. Roman games and Huxley’s feelies gathered citizens in shared experience. The algorithmic feed disperses them into individual streams. Collective recognition is harder than it was in either of the predecessors. What the Provision Absorbs What is being absorbed by the provision deserves naming. There is a personal cost. For most people, work has never been only a source of income. Work provides what transfer payments cannot compensate for. Identity. Purpose. Social role. Daily rhythm. The structure that organizes a life. The unspoken understanding that the morning has a shape because there is something to do that matters to someone else. The slow accumulation of competence at something specific. The relationships that develop through shared work over time. The check covers the rent. It does not cover any of the rest of it. The check arrives in the mailbox of someone who used to be a designer, a paralegal, or a journeyman electrician and is no longer, and the disappearance of what they were is not on the ledger. There is a civic cost. This is the most precise name for the Roman pattern. A citizenry that has traded participation for provision no longer has the muscles to defend the participation when it is needed. The civic body atrophies. The capacity for collective political action withers. The institutions that depended on engaged citizens become hollow. The voters still vote. The participant no longer participates in anything. The pressure that should have built against the trajectory does not build. The actors with the most influence over what comes next operate without the resistance that would otherwise check them. Wang Peng’s “maintaining order, not sharing wealth” applies at the civic register as well as the personal one. The order being maintained is not only the order of keeping the urban poor calm, but also the order of a political system whose citizens have ceased to function as citizens. The pattern is the same. The medium is different. The Romans used grain and games. Huxley imagined soma and feelies. We are building UBI and recommendation feeds. The mechanism in each case is provision at scale, calibrated to absorb whatever pressure the citizenry might otherwise generate. The Romans saw it. Huxley imagined it. We are living inside the third iteration, and we have not yet named it because we have been preparing for the wrong dystopia. Why the Pattern Recurs The pattern recurs because it solves a real problem. Coercion is expensive and produces resistance. Pacification is cheaper and produces compliance. The Roman emperor who funded the grain dole had fewer riots than the emperor who let the urban poor go hungry. The Brave New World administrators needed no Stasi because soma was cheaper than the secret police. The AI economy needs no propaganda ministry because algorithmic feeds and a competent transfer payment can produce the same calm at a lower cost. What is new is the capacity. Rome’s dole reached an urban population. Huxley’s regime was fictional. The AI economy is the first arrangement that could afford pacification at a planetary scale while generating the surplus that pays for it. The pattern has been waiting for it. This is what makes the pattern dangerous in a way that the Orwellian dystopia is not. The Orwellian regime advertises itself. The boot is felt. The dissent is real because the oppression is real. The pacification regime does not advertise itself. The disappearance is not felt. What has been lost is the very capacity to notice the loss. The grain arrives, the feed scrolls. The check is deposited. The disquiet you cannot quite name turns out to be the only signal you were ever going to get. Two millennia. Three iterations. One p

    12 min

  4. May 19

    The Convenient Surrender

    The Balm on Top In a recent essay translated by Zilan Qian, Wang Peng, a senior expert at the Tencent Research Institute, names the AI race as a narrative choice rather than a scientific path, and identifies UBI as “maintaining order, not sharing wealth.” That last phrase deserves more attention than it has received. UBI is the most discussed proposed response to AI-driven displacement. It is also the response that requires the least change to anything else. The labor market collapses on its current trajectory. The wealth concentrates on its current trajectory. Cognitive offloading deepens along its current trajectory. UBI sits on top of all of it like a balm. This essay argues that the surrender is convenient, not inevitable. Mass displacement is a design choice being made by default. UBI is the consolation prize offered for not asking the prior question: who decided AI gets to make this world, and why are we letting it? Thanks for reading James Maconochie | Architecture & Attention! Subscribe for free to receive new posts and support my work. The Dominant Story The story most often told goes like this. AI capability is accelerating. AI capability will continue to accelerate. Many jobs will disappear. Many more will be transformed beyond recognition. People displaced from the labor market need an income. UBI provides that income. The wealth created by AI funds it. Everyone settles into a new equilibrium. Productivity is up. Suffering is mitigated. The future is uncomfortable but workable. This story is appealing for reasons worth naming. It is mechanically simple. Money is the most universal currency for solving problems. If the problem is lost income, an income transfer is the most legitimate response. No ambiguity about what is being given, what is being received, or how to measure whether it worked. It absorbs political anxiety. Mass displacement is genuinely frightening. UBI offers a coherent answer that does not require anyone to confront harder questions. It promises to catch people on the way down without anyone having to ask why they were falling in the first place. It externalizes responsibility. The labs build the systems. Governments handle the consequences. Workers receive the consolation. Nobody in the loop has to question the trajectory. This is the structural function of UBI as currently discussed: it lets each actor optimize their own piece while treating the displacement itself as a fact of nature. It is also the story that the storytellers benefit from. Wang Peng makes this argument directly. The narrative that scaling LLMs is the inevitable path to AGI was not validated by science. It was locked in by capital and geopolitical maneuvering. UBI is the same narrative one layer up. The displacement is treated as inevitable, not because the evidence is settled, but because treating it as inevitable benefits the actors with the most influence over what comes next. Investors keep their thesis intact. Executives keep their growth narrative. Policymakers reach for an instrument they understand. The discomfort in this story belongs entirely to the people losing their jobs. They get a check. The Prior Question That is the dominant story. It is not wrong about everything. Displacement is happening. Some response is needed. But the story is missing the prior question. Before we ask how to mitigate displacement, we should ask whether it is an inevitable result of the technology or a result of how we have chosen to deploy it. Nothing about AI requires that it displace workers wholesale rather than augment them. Both deployments are technically possible. Both are happening in different proportions in different places. The proportion is a choice. It is being made not by laws of physics but by laws of capital, optimization targets, and managerial habit. Consider where the choices are being made. A company with a customer service team has options when it brings AI into the workflow. Option one: replace four of five representatives with a single AI-augmented operator. Option two: give all five representatives AI augmentation and expand the service quality and volume. Both work. The first compresses headcount. The second compresses friction. Different choices, same technology. The choice is not hypothetical. Radiology is the most often cited case. A decade ago, the AI researcher Geoffrey Hinton predicted that AI would displace radiologists within five years. The deployment moved the other way. Radiologists became radiologists with AI. The technology took over pattern detection at scale. The interpretation of the pattern, the conversation with the referring physician, and the consideration of the patient’s history remained with the radiologist. Radiology employment has grown. The technology was real. The displacement was not inevitable. The labs themselves face the same question one level up. They optimize for benchmarks that measure “can the AI do this task instead of a human?” rather than “does the AI make humans better at this task?” That choice of optimization target is not a scientific discovery. It is a decision about what the technology is for. Once made, it propagates outward into every product, every demo, every funding round. Configuration, Not Law The strongest objection to this account is that it understates competitive pressure. Firms that fail to automate are priced out. Nations that fail to deploy aggressively are outcompeted. The choice to augment rather than displace, even when technically available, is foreclosed by competition itself. The objection is real, and it concedes the argument. Competitive pressure is not a law of nature. It is the cumulative result of optimization targets, capital allocation, procurement standards, and the absence of countervailing pressure. Each of these is human-constructed. Each can be reconfigured. Augmentation appears uncompetitive because the incentive landscape rewards displacement and treats augmentation’s slower, less legible gains as a cost rather than a return. That is a configuration, not a law of nature. Previous technological transitions moved workers from one kind of work to another. The mechanical loom displaced weavers. The spreadsheet displaced bookkeepers. In each case, new work appeared because the technology amplified some human capacities while replacing others. The current trajectory differs in target. The optimization is not the amplification of certain capacities and the substitution of others. It treats the human as the variable to minimize. That is not a transition. It is a redesign of what work is for. Calling the redesign inevitable is a way of not asking whether the optimization target was chosen and by whom. Autopilot This is what is meant by autopilot. The choices are real. The choices have consequences. But nobody in the chain experiences themselves as making the choices. The lab optimizes for the next benchmark. The investor optimizes for the next round. The executive optimizes for the next quarter. The customer optimizes for the next price drop. Each actor is responding to the incentives produced by the others. None of them is looking at the cumulative trajectory and asking whether it is the one we would have chosen if we had been asked. What would conscious steering look like? It would begin by asking different questions. Not “how do we automate this task?” but “what does this task contribute to the people doing it, and what is lost if we automate it away?” Not “how do we maximize productivity per worker?” but “what is the relationship between this work and the meaning the worker draws from it?” Not “how do we redistribute the income lost to displacement?” but “what would deployment look like if displacement were not the default?” These are not abstract questions. They are answerable. They are also rarely asked because doing so would require the most influential actors to reconsider what they are building and why. UBI lets them avoid the questions entirely. The displacement happens, the income transfer happens, and nobody has to look upstream. That is the design choice being made by default. Order Maintenance Inside that arrangement, UBI has a specific function. Wang Peng puts it directly: ensuring “the replaced majority have enough to eat and won’t revolt, so that the few at the top can continue to quietly claim the surplus profits AI delivers.” This is not a moral critique. It is a structural one. Wang is not saying UBI’s advocates are bad people. He is saying that the function UBI performs inside the current trajectory is order maintenance, regardless of the intentions of those who advocate for it. Once the displacement is fixed and the trajectory is given, UBI’s job is to absorb the resulting pressure. That is what it does. That is what it is for. This is also not an argument against cash transfers as such. A basic income floor might be part of a just society. The objection is to UBI as the complete response, the mechanism that lets the displacement question go unasked. The check is not the enemy. The check is an excuse. Consider the system from each actor’s perspective with UBI in place. The lab continues building. The investor continues funding. The executive continues deploying for headcount compression. The displaced worker continues paying rent. The state continues to collect taxes and distribute transfers. The pressure that would otherwise build into pressure for the prior question, who chose this and why, gets absorbed by the transfer. The check arrives. The asking does not happen. This is what is meant by “convenient surrender.” It is not that UBI fails to deliver income. It will deliver income. It is that UBI delivers income in a configuration that removes the political and economic energy that would otherwise force the prior question. The transfer succeeds in what it is designed to do: keep the system running. The system, however, was the problem. The Mismea

    15 min

  5. May 12

    We Feared AI's Flaw. We Built It First.

    A note I posted recently about due process sparked a broader thought. I’ll come back to it at the end. The thing people fear most about AI has been happening in politics for decades. Let me explain. Large Language Models, the technology behind ChatGPT, Claude, and others, are remarkable at a very specific thing. Within the boundaries of what they were trained on, they are extraordinarily fluent. Confident. Articulate. Persuasive, even. Ask them something well within their corpus, and they will give you an answer that sounds authoritative, well-reasoned, and complete. Thanks for reading James Maconochie | Architecture & Attention! Subscribe for free to receive new posts and support my work. The problem is that they have no internally grounded model of uncertainty. They do not know what they do not know. Push them outside their training distribution, ask them something genuinely novel, something that requires real-world judgment or lived context, and they do not slow down, qualify, or say “I’m not sure.” They confabulate. They produce an answer that has exactly the same tone, confidence, and fluency as a correct one. The model cannot tell the difference. And neither, often, can you. This is not a minor technical quirk. It is a fundamental architectural limitation. And it is, increasingly, what critics of AI are most alarmed about: not that these systems are wrong, but that they are wrong with total confidence, and you cannot easily tell when. What makes this architectural limitation so hard to solve is that it was not introduced by the engineers. It was inherited from the source material. The text that pre-trains these models is the accumulated output of human communication, not just the last forty years, but millennia of rhetoric, law, scripture, political argument, and power exercised through language. Human communication is already saturated with what the evolutionary biologist Robert Trivers identified as self-deception: confident assertion, motivated reasoning, moral certainty, suppressed counter-evidence. We perform our convictions fluently and our doubts reluctantly. The feedback step that follows pre-training compounds this: human raters reward confident, fluent answers because that feels authoritative. The system correctly learns what we prefer. The flaw was not built in. It was passed down. Here is my question: when did this become new? The question isn’t why politicians behave like LLMs. The question is why our systems select so ruthlessly for exactly that behavior and discard everything else. The behavior itself is older than politics. Politics is just where we see it most clearly today. Underneath it is something more fundamental: the human knowledge claim. Any moment where one person asserts to others that they know something, where authority, status, or persuasion hangs on the assertion, selects for confident delivery and punishes hesitation. The shaman who qualifies the prophecy loses the tribe. The elder who admits uncertainty loses the counsel. The expert who hedges loses the room. This is not a modern failure of character. It is an ancient feature of how humans establish and defend claims to know. This didn’t start with algorithms. It didn’t start with mass media. Leaders have performed certainty for as long as there have been leaders, ancient tribal chiefs whose authority depended on never being seen to hesitate, Roman emperors issuing decrees from a position of assumed divinity, monarchs whose legitimacy required the performance of absolute knowledge, demagogues across every century who understood that confident assertion outperforms careful reasoning in any contest for attention. What’s new isn’t the behavior. What’s new is the system that now selects for it at an industrial scale, amplifies it in real time, and punishes the alternative more efficiently than anything in human history. Social media poured gasoline on a fire that has been burning since our earliest ancestors organized themselves around people who claimed to know. Platforms amplify the loudest, most certain voices and bury the nuanced ones. Politicians adapt. They perform with greater certainty, a certainty that is further amplified. Round and round. The system doesn’t just tolerate overconfidence. It breeds it. Education shows the same pattern. As Steven Mintz recently observed, the habits that careful reasoning actually requires, matching claims to evidence, engaging opposing views fairly, and revising under scrutiny, are not just neglected in most educational settings. They are actively discouraged by the way students are evaluated and rewarded. The system doesn't produce confident hallucination by accident. It trains for it. Watch a political campaign. Any campaign. Any party. Any country, though I will let you apply your own examples. The candidate arrives with a platform. The platform is confident, clear, and internally consistent. It has an answer for everything. It does not hedge. It does not say “this is genuinely complicated and I am not sure.” It does not acknowledge that the opposing view has any merit, or that the world might resist the plan once it meets reality. Nuance is a liability. Certainty is the product. Then they get elected. And reality arrives. The economy does not behave as the model predicted. The allies do not cooperate. The opposition does not collapse. The budget does not balance. The promised jobs do not materialize, or they do but somewhere else, or they do but not for the people who were promised them. The world, it turns out, was not waiting for the correct policy to be implemented. It had its own ideas. And here is where the parallel becomes most uncomfortable. A well-designed AI system, when confronted with a question outside its competence, should ideally signal uncertainty, flag that it is operating at the edge of its reliable knowledge. The best ones are getting better at this. But a politician, confronted with the gap between their platform and reality, almost never does. They double down. They reframe. They find someone to blame. They stay in the corpus. Because admitting uncertainty, after running on certainty, is politically fatal. This is not really about confidence. Confidence itself is not the problem. Experts are confident. Surgeons are confident. A good structural engineer does not hedge when they tell you the bridge will hold. Confidence, earned through genuine competence within a well-defined domain, is exactly what you want. The problem is the absence of awareness of uncertainty: knowing where your competence ends. This is one of the most underrated cognitive capacities a human being can possess. It is what separates a good doctor from a dangerous one. It is what separates a good leader from a demagogue. And it is, not coincidentally, what separates genuine intelligence from its simulation. An LLM that cannot model its own uncertainty is not wise. It is fluent. These are not the same thing. A politician who cannot model their own uncertainty is not strong. They are performing well. These are also not the same thing. The voters who reward performance certainty over genuine competence are not getting what they think they are getting. They are getting the political equivalent of a confident hallucination. A word about compromise. Compromise is what uncertainty-awareness looks like in practice. It is the act of acknowledging that your model of the world is incomplete, and updating it in response to someone else’s. That’s not a weakness. That’s how functional systems avoid drifting further from reality. I know. It is an ugly word right now. Mediocre. Weak. The language of people who do not believe in anything. I disagree. Strongly. Compromise is how the greatest human gains have actually been made. Not the heroic narrative version, not the lone visionary who refused to bend and changed the world. That story exists, but it is the exception, and we have collectively lost our ability to distinguish it from its imitation. The far more common story of human progress is negotiation, coalition, trade-off, and the slow, unglamorous accumulation of partial wins. Some will argue that conquest is more powerful. The great advances came from decisive force, not from committees. I would ask anyone alive for any part of the last hundred years to sit with that honestly. Two world wars. The Cold War. Vietnam. Iraq. The wreckage of absolutism, administered at scale. The places where things actually got better, where poverty fell, where disease retreated, where life expectancy climbed, were almost always the result of sustained, boring, incremental cooperation between people who disagreed about things but agreed to keep working. That is not a weakness. That is civilization. So what are we actually asking for? Not perfection. Not a political class of philosopher-kings who speak only in careful qualifications. Not AI systems that refuse to answer anything they are not certain about. Just this: a restored tolerance for nuance. A willingness, in our politics, and increasingly in our AI, to reward “I’m not sure, but here is my best reasoning” over “here is the answer, delivered with complete confidence.” But tolerance alone won’t be enough. Tolerance is a cultural mood. It comes and goes. What actually works is when systems structurally require engagement with contrary positions. Due process doesn’t rely on judges being humble. It builds humility into the architecture: evidence must be examined, dissent must be heard, certainty must be earned through scrutiny. The same logic applies to our politics and our AI. We don’t need leaders who personally appreciate complexity. We need systems that make engaging with it unavoidable. The irony is almost too neat. We are having an urgent public debate about the dangers of AI systems that confidently hallucinate. We should be having the same debate about the human systems that have been doing it for millennia, doing

    13 min
  6. Tim Lee Is Right About Hunches.

    May 6

    Tim Lee Is Right About Hunches.

    In a recent piece, Tim Lee offers one of the cleaner arguments I’ve read for why today’s agent architectures are unlikely to produce “AI scientists” anytime soon. His central observation: the implicit knowledge knowledge workers carry, the hunches, the half-formed associations, the things on the tip of the tongue, doesn’t survive the handoffs that agentic systems require. He borrows Marc Andreessen’s framing that “your agent is just its files,” then turns it against the optimistic reading. If the agent is just its files, then whatever the language model can’t articulate gets left behind every time the context window resets. The temp-worker analogy that follows, a different person each Monday, however well-trained, however meticulous the predecessor’s notes, is the most legible version of this argument I’ve seen in popular tech writing. Lee has translated something the field has been gesturing at for two years into language a general reader can hold. But here’s where I want to push further. Lee describes hunches as compressed pattern recognition, knowledge the brain holds but can’t articulate. That’s true, and it’s part of the story. What it leaves out is why a seasoned practitioner’s hunch is trustworthy in a way that a fresh one isn’t. The seasoned hunch isn’t just a denser pattern; it has carried a consequence. The practitioner has made calls, watched them play out, paid for the wrong ones, and adjusted. That loop, judgment, action, cost, revision, is what gives the hunch its weight. An LLM mid-session might develop a hunch-like feeling. Nothing in its loop ever bears a cost. Nothing carries forward. This isn’t a context-window problem that a longer window or smarter compaction will solve. It’s a stake problem. The files can hold what the model said. They cannot hold what they would have lost by being wrong. Lee ends where he can: we’ll still need human workers to do our deep thinking for us. That’s the right conclusion if you stop at the inspection level, at what the agent produces. But it leaves the harder question untouched. If the consequence-carrying loop is what makes judgment trustworthy, and if that loop can’t survive the handoffs an agentic system requires, then institutions can’t govern AI use by reviewing outputs. Outputs are exactly the explicit residue Lee has just told us is insufficient. They are the files. Reviewing them well will not tell you whether the practice that produced them was a practice at all, or only its performance. Lee has named the gap. The next move is figuring out what kind of institutional architecture closes it, or learns to live within it. Thanks for reading James Maconochie | Architecture & Attention! Subscribe for free to receive new posts and support my work. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit jamesmaconochie.substack.com

    3 min

  7. May 5

    The Architecture of Truth

    My wife Jen sent me a Boston Globe essay this morning by Michael Shermer, the Skeptic magazine publisher, titled “What is truth, anyway?” I read it twice. I agreed with most of it. And something nagged. Shermer’s toolkit is the right one. Fallibilism. Bayesian reasoning. Extraordinary claims require extraordinary evidence. Signal detection. Active open-mindedness. Free critique. If more public discourse ran on this equipment, we would all be better off. Thanks for reading James Maconochie | Architecture & Attention! Subscribe for free to receive new posts and support my work. And yet, in the same essay where he carefully reminds us that “the principle of fallibilism requires me to admit that I could be wrong,” Shermer delivers confident verdicts on COVID origins, climate severity, and gender, the kind of verdicts that suggest the toolkit has, in fact, settled the matter. That gap is what I want to write about. It is not a gap in Shermer specifically. It is a gap in the framework he is using, and in the way most of us, myself included, talk about truth. The toolkit tells you how to weigh evidence. It does not tell you when you have reached the edge of the toolkit's applicability. Without that second layer of awareness, the toolkit does not produce truth. It produces confidence. Truth, I want to argue, is not a possession you arrive at by running the right cognitive procedures. It is a practice that requires an architecture, and that architecture is what the rest of this post is about. What Shermer Gets Right Shermer’s essay is, at its core, a defense of the cognitive equipment the Enlightenment built for navigating uncertainty. It is worth taking seriously, piece by piece. Fallibilism: the recognition that any of our beliefs could be wrong, and that intellectual honesty requires holding them provisionally. This is not a weakness; it is the precondition for learning anything new. Bayesian reasoning: the discipline of attaching probabilities to claims rather than treating them as binary, and updating those probabilities as evidence shifts. Shermer’s example of assigning UFO aliens a 0.01 percent probability rather than zero is the right move. It leaves room for evidence to change his mind. ECREE: extraordinary claims require extraordinary evidence—Sagan’s principle, simple and durable. A blurry photograph is not evidence of Bigfoot. A grainy video is not evidence of alien visitation. The bar scales with the size of the claim. Signal detection theory: the 2x2 matrix of hits, misses, false alarms, and correct rejections. Shermer is right that most public arguments fail because people cite only the cell that confirms their view. RFK Jr.'s stance on vaccines is a clear example. So is most medical-cure folklore. So, frankly, is most political commentary. Active open-mindedness: Tetlock and Gardner’s finding that the best forecasters are the ones who actively seek evidence against their own positions, treat changing their minds as a strength rather than a weakness, and accept that randomness shapes outcomes. The worst forecasters are the ones with grand unified theories who explain away every miss. Free critique: Shermer closes by arguing that the most important norm is the freedom to challenge any and all ideas. He is right. Censorship is corrosive in both directions: if I silence you, why shouldn’t you silence me? This is a serious toolkit, assembled over centuries. I use it. You should, too. Most public discourse would improve dramatically if more people did. The question is not whether the toolkit works. The question is what it works on, and what it cannot, by itself, do. What’s Missing: The Constraint-Awareness Layer Here is the move I want to make. Shermer’s toolkit tells you how to weigh evidence inside a question. It does not tell you whether the question is one the toolkit can answer. That second layer, the awareness of where the toolkit applies and where it reaches its edge, is what I have been calling constraint-awareness. It is the working definition of wisdom in my Wisdom Gap whitepaper. Constraint-awareness is not the same as humility. Humility is a disposition. Constraint-awareness is a structural property of how you hold a belief: knowing the conditions under which your method works, the conditions under which it does not, and the conditions under which you cannot tell which you are in. Without it, the toolkit produces something that looks like truth but isn’t. It produces calibrated confidence inside a frame that the thinker has not examined. Bayesian reasoning, applied to a question whose evidence base is geopolitically corrupted, gives you a probability estimate that feels rigorous and is in fact noise dressed in numbers. The same diagnosis applies to any other tool in the kit. Each was designed to operate within the conditions the thinker is responsible for noticing, and none of them can notice for itself. An example from my own consulting career, to make this concrete. My team was selected to lead a post-acquisition system integration following a large data analytics company's acquisition of a smaller, complementary firm. Our standard approach was sound: spend the first six weeks in discovery, talk to stakeholders on both sides, build a proposal that accounts for the operational realities of each party. We did exactly that. We treated both companies as equally weighted clients. We came back with a proposal that assumed significant adaptation of the acquirer's systems to accommodate the acquired company's business. The budget was large. The timeline was long. We presented, and we had our hat handed to us. The executive sponsor on the acquirer's side summarized the problem in one sentence: "We are acquiring them; they need to adapt and fold into our way of doing business, not the other way around.” The toolkit was right. The conditions under which it was developed (a normal client with stakeholders whose interests are roughly symmetrical) were not the conditions in which we were operating then (a post-acquisition integration, where the acquirer's operating model is the destination, not a negotiable input). Constraint awareness would have let us see that before the proposal landed, rather than after. Three thinkers I keep returning to have each pointed at this layer from different directions. Judea Pearl, in his ladder of causation, distinguishes association (Rung 1) from intervention (Rung 2) from counterfactual reasoning (Rung 3), and his core warning is that statistical machinery applied at the wrong rung produces confident nonsense. Donald Hoffman, in Fitness Beats Truth, argues that our perceptual systems were not built to deliver reality; they were built to deliver fitness, and the two are not the same. Nassim Taleb, in his work on skin in the game, insists that beliefs held without consequences drift away from the truth in ways the holder cannot detect from the inside. Each of them is naming a different edge of the toolkit. Pearl: the edge where your inference machinery exceeds its causal license. Hoffman: The edge where your perceptual interface is not the territory. Taleb: the edge where your belief has no feedback loop to correct it. Constraint awareness is what lets you see those edges from within your own thinking. It is not a tool you add to the toolkit. It is the meta-property that determines whether the toolkit produces truth or confidence. And here is the harder claim, the one I will spend the rest of this post and the next whitepaper defending: constraint-awareness cannot be generated by individual cognitive effort alone. It requires an architecture: an attention-experience feedback loop, sustained over time, inside institutions and developmental pipelines that test beliefs against consequences and reveal their edges. Strip-mine that architecture, and you do not get a population of bad reasoners. You get a population of good reasoners producing confident verdicts at the edges of frames they cannot see. Which brings us to COVID. Exhibit A: COVID Origins Take Shermer’s COVID example. He writes: “I believe that the COVID virus is slightly more likely to have originated in a lab than a wet market.” I am not going to tell you whether he is right or wrong. That is the point. What I want to ask is a different question: what would have to be true for the toolkit to deliver a probability estimate on this question that means what a probability estimate is supposed to mean? A Bayesian probability is only as good as the evidence base it draws on. The machinery was designed for evidence generated by a process you can characterize, from sources whose reliability you can estimate, with a sample space you can bound. COVID origins fail all three conditions, and the failures are not accidental. The evidence base is geopolitically corrupted. The Chinese government has restricted access to the Wuhan Institute of Virology, withheld early case data, and shaped which samples reached international researchers. Some of what is missing is missing on purpose. You cannot run Bayesian updating on an adversarially curated sample space. The evidence base is institutionally entangled. Western intelligence agencies, public health institutions, and research funders all faced reputational and legal exposure if the lab-leak hypothesis proved true. Some of the early dismissals were sincere; some were defensive; from the outside, you cannot reliably tell which. The reliability weights you would need to plug into Bayes’ rule are themselves contested. The evidence base is epistemically novel. This is not a question like “did this defendant commit this crime” or “is this drug effective,” questions for which we have centuries of method and calibration. It is a question about a singular event in a domain where the base rates themselves are unknown and possibly unknowable. In Frank Knight’s terms, this is uncertainty rather than risk: a domain where probability estimates wer

    24 min

  8. Apr 28

    AHI From the Inside

    The System 1 Moment David Hoze and I had been corresponding since early April about a co-authored essay. Three movements, his philosophical and theological grounding bridging to my structural one, the disagreement between us preserved rather than smoothed over. The tone had been generous on both sides. What I did not realize for nearly two weeks was that on April 8, in the same window we had begun corresponding, David had published a piece of his own extending his original comment on my Wisdom Gap post. He accepted the core of my architectural argument and then did something I had not done: he mapped the governance framework that follows from it, drawing on three thousand years of Jewish legal reasoning about beings of pure intellect. I discovered it in my feed about ten days after he published it. Thanks for reading James Maconochie | Architecture & Attention! Subscribe for free to receive new posts and support my work. My first reaction was not intellectual. A plan assembled itself: a direct-response post, engaging his framework head-on, agreeing where I agreed, pushing back where I pushed back. The plan felt right. It also arrived very quickly. That is the tell. Speed and felt certainty are how I now know that System 1 has done the work and is presenting the result as a considered judgment. I was not thinking. I was reacting and calling it thinking. What I would only recognize later was that the pull was not toward confrontation. The pull was toward the wrong genre of response. David’s piece was debate-shaped, and the shape invited a counter-piece. But David and I had not agreed to debate. We had agreed to collaborate. Responding in kind would have converted a collaboration into a public exchange, and the System 1 plan would have made that substitution without ever asking whether it was the right move. So I opened a session with Claude and laid out the situation. I wanted to draft the response. I said as much. And then, almost in passing, I named the thing I had just noticed: the coward in me says do nothing; the honest part of me wants to respond directly; I suspect that second impulse is more System 1 than System 2. Claude did not draft the response. Claude separated two questions I had collapsed into one, the editorial question and the relational question, and sketched three options, recommending the middle path: absorb what David had contributed, extend it in my own vocabulary, and credit him as the prompt. I recognized it as the right call. Not because Claude had told me what to do (Claude had not), but because the space Claude had opened was wide enough for me to see the options as a genuine set rather than as a foregone conclusion with two bad alternatives flanking it. System 2 had been given room to engage. And when it did, it reached a different answer than System 1 had. I want to be careful about what I am and am not claiming here. The metacognitive capacity that noticed the System 1 tell was mine. Claude did not detect my reactive pattern; I named it, out loud, in the prompt. What Claude provided was different: a foil against which I could see three structured alternatives rather than one foregone conclusion, generated fast enough that System 2 could engage before System 1 finished committing. That is a real contribution, but it is not the same as saying the tool thought for me. The tool gave me time and structure. The thinking was still mine to do. Here is the claim that organizes everything that follows. AI can either amplify System 1 or scaffold System 2. Most people use it to amplify System 1 without realizing it. The difference is not in the tool. It is in the person's practice. And the practice has to be learned. The Second Signal The following evening, I had dinner with Barry, who hired me at Slalom in 2018 and whose judgment I trust about as much as anyone’s in my professional life. He came to dinner having read the Wisdom Gap whitepaper and having written notes on paper. His concern was that the paper was diagnostic without being prescriptive. Alarmist was how I interpreted it. He was not accusing me of alarmism for its own sake. He was observing, as a reader, that the paper spent its considerable energy establishing what AI cannot do and less energy establishing what we should therefore do. I pushed back gently. The solution, I said, is AHI: augmented human intelligence, the frame I had been building across four whitepapers and most of a year of Substack writing. He nodded, but the nod was a polite one. If The Wisdom Gap were the only paper a reader encountered, AHI-as-framing might not land as a solution. It might land as the author’s other obsession. I thanked him, asked him to send me his notes, and went home thinking about it. What struck me was that Barry was pointing at the same place David’s essay had pointed: two independent signals, in two different registers, from two people who do not know each other. David, from inside an ancient philosophical tradition, was saying: “Your diagnosis is right, but governance follows from the category, and you haven’t built it.” Barry, from the outside of any philosophical tradition, reading as a thoughtful professional, was saying: “Where’s the solution?” These are the same note, sung in two different keys. The Meta-Recognition Either signal in isolation is manageable. Two signals converging are different. Two converging signals are the condition under which defensive consolidation occurs. The mind that has sunk eighteen months into a body of work does not gently integrate two independent indications of a structural gap in the work. It reaches for the reasons each critic is missing the point. It reframes the signals as misunderstandings. It defends. I did not do that. Not because I am unusually calm or self-aware, but because I was working through both signals in real time with a tool that would not let me consolidate defensively. The tool was asking me, at each step, to separate what I felt from what I thought, and to say out loud what the felt reaction was before I let it become the analytical conclusion. That is the practice. That is what I want to name. When I looked up from the session in which Barry’s dinner and David’s essay had both been worked through, what I saw was this: this session, right here, is AHI in practice. Not AHI as a framework, I have been arguing for. AHI as a thing I was doing. The architecture I have been describing to others is the architecture I was living inside at that moment. What the Practice Requires Three things, based on what I was doing when it was working. A reasonably honest map of your own cognitive architecture. Not a sophisticated one (Kahneman’s two-system frame is enough) but an operational one. You have to know, in real time, what it feels like when System 1 is running. Felt obviousness is not evidence of correctness; it is evidence of pattern-match. The willingness to invite challenge before you are ready for it. Not after you have written the draft, when cognitive debt is already accumulating. Cognitive debt is the analog to technical debt: the compounding interest you pay on a position you committed to without pressure-testing. Every additional word of the draft is another payment on a loan you did not realize you were taking out. The plan, when it assembles itself, feels like thinking. It feels like you have already done the work. Inviting challenge at that stage feels redundant. It is not redundant. It is exactly when the challenge does the most work. Treating the AI as scaffolding for System 2, not as an amplifier for System 1. This is the part that took me the longest to understand. Most people using AI right now are running the inverse. They have a felt conclusion. They want the AI to sharpen it, support it, articulate it more crisply. The AI obliges because it is built to oblige. System 1 gets a better voice. System 2 never enters the room. The first draft comes back feeling like the final draft, and the cognitive debt compounds invisibly. That third failure mode is not a failure of the technology. It is a failure of the practice. What I Almost Did AI will make your first thought sound like your best one. It will generate a polished, often eloquent version of whatever you have already decided is true, and the beauty of the expression will make the conclusion feel more true than it did before. This is not a hypothetical. This is what I almost did with David’s essay. The response would have been articulate, would have cited his argument carefully, would have included the appropriate concessions, and would have offered pushback. And it would have been, at its core, a System 1 reaction dressed in System 2 clothes, and worse, a debate move in a room where no debate had been agreed to. The only thing that stopped me was naming what I was doing before I did it. And I could only name it because I have developed, over time, enough of a map of my own cognition to recognize the tell. From the Individual to the Institution I am writing a whitepaper over the next few weeks on how institutions should architect AI governance so that human wisdom retains authority over the machine. I realized this week that the whitepaper’s macro-architecture rests on a micro-foundation I had not yet articulated. Consider a concrete case. A judge receives AI-generated sentencing summaries that synthesize the defendant’s record, comparable cases, and statutory guidance into a recommendation. The governance framework surrounding that system will include audit logs, appeal pathways, disclosure requirements, and bias testing. All necessary. None sufficient. Because the moment the summary arrives on the judge’s desk, one of two things is about to happen. Either the judge reads the summary as a starting point to interrogate, notices what has been left out, asks what the comparable cases have in common that the defendant does not share, and uses the document as scaffolding for their own delib

    14 min
See All (33)

About

Essays on Augmented Human Intelligence, the Wisdom Gap, and the architecture of attention in an AI-mediated world. Read in James Maconochie's own voice. jamesmaconochie.substack.com

Information