The US administration announced an AI Force on Saturday, with an AI czar to run it, modelled on the Space Force. The announcement was clear about the mandate: “We will not in any way hinder or stifle the Growth of this incredible Industry. Rather, we will cherish it, help it, and watch over it, as it grows!” That is a mandate to cheerlead, but I think it is the wrong one. I’m in favour of an AI Force, but I would point it in close to the opposite direction. Think about the analogy of the modern corporation. We decided a long time ago that a corporation could hold rights approaching those of a person, so that it can own property, sign contracts, sue and be sued, and outlive everyone who founded it. That was a useful invention but a potentially dangerous one, so we spent the following century building the guardrails around it. Company law, consumer protection, competition rules, disclosure regimes, personal liability for directors. Nobody seriously argues that any of it stifled the growth of the limited company. It is the reason the limited company is tolerable. We are now creating a second category of non-human actor with agency, and the proposal on the table is to cherish it. On development itself there is very little for government to fix. Private capital is funding this at a scale no state programme will improve on, and the frontier labs are not short of money, compute or people. What is absent is the other half of the arrangement, an assurance framework that keeps people safe whatever gets built. I’m a fan of Yudkowsky and Soares’ If Anyone Builds It, Everyone Dies, and I know that book is contested, but you don’t have to accept its conclusion to take the underlying risk seriously, and I would put it the way I’d put anything into a board risk paper. AGI is a real possibility, nobody can currently put a defensible bound on the probability that it goes badly, and the severity if it does is unrecoverable. Low confidence in likelihood, combined with catastrophic and irreversible impact, is exactly when you build the controls in advance, while the data is still poor. So what would I ask an AI Force to do? Not licensing for model training, which is close to unenforceable and mostly theatre. The work is deciding what responsible development actually obliges a developer to do, and then getting that into law in enough jurisdictions for it to bite. The Council of Europe Framework Convention on AI is a start and nowhere near sufficient. Then enforcement, including against the countries that decline to sign up. There will be no global court for this. People tend to reach for the International Criminal Court as the model, and it does not fit, because the ICC can only act on the territory or nationals of its member states, and only where a national system has failed to act first. The precedent that does work comes from financial crime. FATF has no treaty, no court and no enforcement power of its own. It writes standards, assesses countries against them, and publishes a grey list and a blacklist. Landing on either raises your cost of capital and complicates correspondent banking, and countries rewrite their laws to get off it. Attach that same mechanic to compute access, chip supply and model export and you have real leverage. The third area is response, and it is where I would put the money first. Assume a rogue AI agent is already out there. What is the national capability to find it and stop it? The standard answer is that this is only software and you can pull the plug. That comes from people who have never had to run an enterprise technology estate. The plug is not in one place. Models have already moved out of the data centre and into the operational technology behind power, water, transport and payments. I’ve spent the past few years writing operational resilience policy against DORA, and what becomes clear is how few organisations can enumerate their critical dependencies at all, let alone sever one deliberately and keep running. So what would stopping one actually involve? First you have to find it, and that is closer to tracing a piece of malware than to matching a fingerprint. You look at how the thing behaves, the patterns in what it produces and the mistakes it makes, and then at where it is running, because a model of any size has to run on somebody’s computers and that leaves accounts, traffic and a bill. Once it has been copied and retrained a few times those traces blur, so what you identify is a family rather than an individual. We have nothing like this at national scale today. Then you need an order of things to try, from the cheap to the drastic. Cut off its logins and its access to the systems it is using. Shut it down where it is running, which is the nearest thing to a real off switch, because nothing of this size runs without a great deal of computing power sitting in a small number of buildings, and those buildings have owners. Cut it off from the network. Cut off its money, using the same machinery we already run for sanctions. Turn off the power to the data centre. And run the physical infrastructure by hand. That last bit deserves a moment, because it is where the work is. We have spent thirty years engineering out that kind of capability in the name of efficiency. When Ukraine’s grid was attacked in December 2015, the attackers sabotaged the remote controls so that nothing could be switched back on from a screen. Power came back because people drove out to the substations and worked the switches by hand. The question worth asking is whether the infrastructure you rely on could still do that. At the far end sits the electromagnetic pulse, and it earns its place there because of what it costs. It targets nothing, and takes out the hospital, the water pumps and the phone network inside the same radius. Reaching for it means everything above it has already failed. The uncomfortable part is that a model which has already been copied and passed around sits outside all of this. You can take away its computing power, its logins, its money and its reach. You cannot take it back. Which is the argument for doing the first two properly, while the option is still there. None of this hinders anyone’s growth. It is what you build so that the growth is survivable, and it takes years, which is why the moment to start is while the whole idea still looks like an overreaction. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit inproductionpod.substack.com
