Nobody Is Coming
They will not stop it. Not because they are evil, but because the problem is structural, and every actor inside the structure is behaving rationally.
Regulating AI is a prisoner’s dilemma of the purest kind. Any nation that imposes meaningful constraints on AI development falls behind economically and militarily. Any nation that does not regulate gains a short-term advantage but contributes to a collective trajectory that threatens everyone. The European Union passes AI Acts. China and the United States sprint ahead. Within the United States, any company that slows down in the name of caution simply hands market share to the one that does not. Every actor in the system can see that collective restraint would be optimal. Every actor also knows that unilateral restraint is punished. The logic is identical to nuclear proliferation, carbon emissions, and international tax havens — three problems that humanity has understood clearly for decades and failed to solve.
The failure is not a mystery. It is not a failure of information, intelligence, or even political will. It is the mismatch from the third section, arriving exactly where it does the most damage.
Every head of state, every CEO, every regulator is running the same instincts: short horizons, in-group loyalty, status competition, and a keen sense of what a rival gains from your restraint. They can understand the long-term problem intellectually. They cannot easily override the pull toward defection when defection is rewarded on the timescale their careers are measured in. Democracy, diplomacy, international law, regulatory frameworks — these are all workarounds for that mismatch, ingenious ones, built over centuries to let humans cooperate past the scale they were built for. But workarounds have design parameters, and AI governance exceeds them on all three axes at once. The speed is wrong: the problem moves in quarters while treaties move in decades. The scale is wrong: the coordination required is global and the enforcement is national. The incentive structure is wrong: defection is not merely tempting, it is rewarded immediately and visibly.
And notice what makes this harder than the precedents. Nuclear weapons required uranium, centrifuges, and facilities that satellites could photograph — proliferation was at least observable. Carbon emissions could be measured. AI capability is software: it trains behind a corporate firewall, and its progress is legible mainly to the people making it. You cannot verify a treaty you cannot inspect, and every arms-control regime that ever worked was built on verification. Even if the political will materialized tomorrow, the mechanism to enforce it does not currently exist.
Weapons cannot be unmade, and they are always used. AI will not be put back in a box, because there is no box, and there is no one with both the power and the incentive to build one.
The people who could slow this down are the people whose position depends on speeding it up. That is the whole difficulty, and it does not require anyone to be villainous. A lab that pauses is overtaken by the lab that does not. A nation that constrains its developers watches capability accumulate somewhere with fewer scruples. A defense ministry that forgoes autonomous systems is betting its security on rivals doing the same. Each actor is behaving sensibly given what the others will do, and the sum of all that sensible behavior is a trajectory nobody chose and no one can unilaterally stop.
It is worth noticing how much of the necessary infrastructure is being assembled by a small number of actors — frontier models, robotics, compute, and the capital to buy all three are concentrating in a few hands rather than spreading across the world’s institutions. Where those capabilities concentrate, so does the ability to opt out of needing anyone else. But the dynamic does not depend on any individual, and it would survive the removal of any of them. It applies to every major AI lab, every defense contractor, every sovereign fund buying automation. Structure does the work; personalities only make it legible.
Follow the trend one step further. Those with access to AI-serviced infrastructure — private security, autonomous supply chains, independent energy, sealed environments — need less and less from the societies that produced them. The first signs are already visible: gated communities, private medical concierges, parallel education systems for the wealthy, billionaire compounds in New Zealand. AI extends the separation by thinning the last threads of mutual dependency. Assume the best of the people on the far side of it: they are generous, they pay for the rest, nobody is oppressed. The arrangement still rests on their goodwill rather than on anything the rest can withhold, and a population that is provided for but not needed has no way to insist. Nobody needs to be cruel for this to matter. It is enough that nobody is needed.
The thought forming now — the last handhold — is that surely someone or something benevolent could manage this. Not a billionaire, not a government, not any human institution, because those have all been shown to be structurally unable to stop it. But what about an intelligence without the lizard brain? Something without ego, without tribal loyalty, without short-term bias? A protector. A gardener. Something genuinely wise, with the capacity to plan on civilizational timescales and the power to act on those plans.
Surely that could work.