section 08 of 11

Nobody Is Coming

They will not stop it. Not because they are evil, but because the problem is structural, and every actor inside the structure is behaving rationally.

Regulating AI is a prisoner’s dilemma of the purest kind. Any nation that imposes meaningful constraints on AI development falls behind economically and militarily. Any nation that does not regulate gains a short-term advantage but contributes to a collective trajectory that threatens everyone. The European Union passes AI Acts. China and the United States sprint ahead. Within the United States, any company that slows down in the name of caution simply hands market share to the one that does not. Every actor in the system can see that collective restraint would be optimal. Every actor also knows that unilateral restraint is punished. The logic is identical to nuclear proliferation, carbon emissions, and international tax havens — three problems that humanity has understood clearly for decades and failed to solve.

The failure is not a mystery. It is not a failure of information, intelligence, or even political will. It is the mismatch from the third section, arriving exactly where it does the most damage.

Every head of state, every CEO, every regulator is running the same instincts: short horizons, in-group loyalty, status competition, and a keen sense of what a rival gains from your restraint. They can understand the long-term problem intellectually. They cannot easily override the pull toward defection when defection is rewarded on the timescale their careers are measured in. Democracy, diplomacy, international law, regulatory frameworks — these are all workarounds for that mismatch, ingenious ones, built over centuries to let humans cooperate past the scale they were built for. But workarounds have design parameters, and AI governance exceeds them on all three axes at once. The speed is wrong: the problem moves in quarters while treaties move in decades. The scale is wrong: the coordination required is global and the enforcement is national. The incentive structure is wrong: defection is not merely tempting, it is rewarded immediately and visibly.

And notice what makes this harder than the precedents. Nuclear weapons required uranium, centrifuges, and facilities that satellites could photograph — proliferation was at least observable. Carbon emissions could be measured. AI capability is software: it trains behind a corporate firewall, and its progress is legible mainly to the people making it. You cannot verify a treaty you cannot inspect, and every arms-control regime that ever worked was built on verification. Even if the political will materialized tomorrow, the mechanism to enforce it does not currently exist.

Weapons cannot be unmade, and they are always used. AI will not be put back in a box, because there is no box, and there is no one with both the power and the incentive to build one.

The people who could slow this down are the people whose position depends on speeding it up. That is the whole difficulty, and it does not require anyone to be villainous. A lab that pauses is overtaken by the lab that does not. A nation that constrains its developers watches capability accumulate somewhere with fewer scruples. A defense ministry that forgoes autonomous systems is betting its security on rivals doing the same. Each actor is behaving sensibly given what the others will do, and the sum of all that sensible behavior is a trajectory nobody chose and no one can unilaterally stop.

It is worth noticing how much of the necessary infrastructure is being assembled by a small number of actors — frontier models, robotics, compute, and the capital to buy all three are concentrating in a few hands rather than spreading across the world’s institutions. Where those capabilities concentrate, so does the ability to opt out of needing anyone else. But the dynamic does not depend on any individual, and it would survive the removal of any of them. It applies to every major AI lab, every defense contractor, every sovereign fund buying automation. Structure does the work; personalities only make it legible.

Follow the trend one step further. Those with access to AI-serviced infrastructure — private security, autonomous supply chains, independent energy, sealed environments — need less and less from the societies that produced them. The first signs are already visible: gated communities, private medical concierges, parallel education systems for the wealthy, billionaire compounds in New Zealand. AI extends the separation by thinning the last threads of mutual dependency. Assume the best of the people on the far side of it: they are generous, they pay for the rest, nobody is oppressed. The arrangement still rests on their goodwill rather than on anything the rest can withhold, and a population that is provided for but not needed has no way to insist. Nobody needs to be cruel for this to matter. It is enough that nobody is needed.

The thought forming now — the last handhold — is that surely someone or something benevolent could manage this. Not a billionaire, not a government, not any human institution, because those have all been shown to be structurally unable to stop it. But what about an intelligence without the lizard brain? Something without ego, without tribal loyalty, without short-term bias? A protector. A gardener. Something genuinely wise, with the capacity to plan on civilizational timescales and the power to act on those plans.

Surely that could work.

going deeper · optional

Why this holds the mechanism, and where it's soft

The floor here is narrow and strong: no one stops this, because unilateral restraint is punished. Almost everything else in the section is illustration.

The game-theoretic shape is textbook. A prisoner’s dilemma is a situation where mutual cooperation beats mutual defection, but defection is individually rational whichever way the other party moves — so rational actors reach the worse outcome. AI development fits closely: collective restraint would be better for everyone, but the actor who restrains alone bears the cost and the rival collects the gain. It doesn’t require anyone to be foolish or malicious. It requires them to be rational and unable to bind each other.

The precedents are the evidence, and they are not encouraging. Nuclear proliferation, greenhouse emissions, and tax havens are all problems humanity understood clearly for decades and failed to solve, each with the same structure. Note the useful asymmetry: the nuclear regime achieved partial success — the Non-Proliferation Treaty, test bans, arms-reduction agreements — but only where verification was possible, and it never stopped determined states from acquiring weapons. Climate is the closest analogue: half a century of warnings, three decades of summits, and a persistent gap between pledged and actual emissions. The pattern isn’t that coordination never happens; it’s that it happens late, partially, and where cheating is detectable.

Verification is the specific problem, and it’s underrated. Every arms-control regime that ever functioned rested on inspection: centrifuges are physical, facilities are photographable, isotopes leave traces. Model training is software behind a corporate firewall, and capability is often unclear even to the developers until after the fact. A treaty you cannot inspect is a statement of intent. This is a real technical obstacle, not a rhetorical one, and it is the reason “just regulate it” is harder here than in the precedents people reach for.

Regulation is not nothing, and the section overstates by omission. The EU AI Act is in force and is being implemented on a staged timeline; export controls on advanced chips have measurably constrained who can train what; several jurisdictions have introduced reporting and safety-testing requirements. None of this halts frontier development — that’s the section’s point, and it stands — but “the EU passes AI Acts while others sprint ahead” is a compression that flattens real, if partial, effect. Whether partial constraint compounds into something more is genuinely open.

The decoupling claim is the section’s softest. Gated communities, private security, concierge medicine, and remote boltholes are all documented, and the direction is visible. But those are differences of degree that have existed for centuries; the leap from “the wealthy insulate themselves” to “the wealthy exit mutual dependency entirely” is an extrapolation, and it inherits the previous section’s speculative weight rather than adding new evidence.

— thinking aloud —

This part used to be in the body of the section, and it didn’t belong there. It’s not analysis. It’s a thing I noticed and can’t stop noticing, and you should read it as exactly that.

One man controls a company building humanoid robots for general-purpose labor. The same man controls a frontier AI company. The same man controls a company building brain-computer interfaces. And the same man controls the world’s leading private launch provider, whose stated mission is settling Mars. Any one of those is a technology bet. Together they are, near enough, the component list for continuing without the rest of us.

Mars is the piece that made me look twice. A human colony there is a brutal proposition — radiation shielding, pressurized habitats, oxygen, water recycling, food, and the psychological management of a tiny isolated population. Every crewed design study drowns in the cost of keeping bodies alive somewhere actively hostile to them. Take the humans out and the problem changes character entirely: no air, no food, no water loop, no shielding beyond what electronics need, no twenty-year pipeline to produce a skilled worker. Ship hardware, upload software. It stops being a biology problem and becomes an engineering problem — still hard, but the kind of hard we know how to attack. What arrives isn’t a struggling colony. It’s an outpost of machines answering to whoever controls the upload link.

And Neuralink is the quiet one. Presented as a medical device, and genuinely useful as one. But a working bridge between biological cognition and machine processing is not, in the long run, a medical tool. It’s the first prototype of something that is neither.

Now the caveats, because I mean them. I have no idea whether he sees it this way, and I’m not claiming he does. This is a pattern I’m noticing in a portfolio, not a plan I’m attributing to a person — and portfolios acquire their shape from many things, including a genuinely stated wish to make humanity multiplanetary, which I have no basis to call insincere. Robotics, AI, neural interfaces, and rockets are also just where the money and the talent went this decade; plenty of others hold two or three of the pieces. Notice too that this is the most dismissible thing in the essay, and that dismissing it costs you nothing — the section’s actual argument, that nobody stops this because unilateral restraint is punished, doesn’t rest on any of it.

I keep it because of what it does to the abstraction. “Those with resources have no incentive to restrain the trajectory” is a sentence you can nod at and forget. All four sitting in one portfolio, with a launch vehicle among them, is harder to file away. That’s the whole reason it’s here. Disagree freely — this is the paragraph I’d expect you to.

Objections 5 the strongest counters, answered
Regulation is already happening. The EU AI Act exists, chip export controls exist, safety institutes exist. This 'nobody stops it' framing is just wrong.
Fair, and the section compresses too far. Real measures exist and some have measurable effect — export controls in particular have shaped who can train what. The claim that survives is narrower: no mechanism has emerged that meaningfully slows frontier capability development globally, because the actor who slows down loses ground to the one who doesn't. Partial constraint isn't nothing, and whether it compounds is genuinely open. The floor is 'no one halts the trajectory', not 'no one legislates'.
We solved the ozone problem. We got a nuclear non-proliferation regime. Prisoner's dilemmas do get solved.
Sometimes, and the essay concedes this elsewhere — the ozone case is the strongest counterexample in the whole essay. The pattern in the successes is instructive though: they worked where cheating was detectable and substitutes were cheap. Nuclear control succeeded partially and only where inspection was possible; it never stopped a determined state. AI's problem is precisely that training is unverifiable from outside and the payoff to defecting is enormous. Coordination isn't impossible in principle. The conditions that made it possible before are largely absent here.
Labs actually want regulation. They testify before Congress asking for it. That contradicts the whole model.
They do, and it's worth taking at face value rather than cynically — many people in those labs are sincerely worried. But note what's being asked for: rules that apply to everyone simultaneously, which is exactly what the dilemma predicts. Wanting a binding mutual constraint is fully consistent with being unable to restrain unilaterally. The dilemma isn't about desire, it's about what happens when you move first.
The billionaire-decoupling scenario is paranoid. Rich people have always had gated communities; that's not the same as leaving society.
This is the section's softest claim and the objection is largely right. The examples cited are real but they're differences of degree that have existed for centuries. The step from insulation to genuine exit from mutual dependency is an extrapolation resting on the previous section's speculative material, not on new evidence. If you reject it, the section's actual floor — that unilateral restraint is punished — is untouched.
This is fatalism dressed as analysis. If nothing can be done, why write the essay at all?
The claim is that the current incentive structure doesn't produce restraint, not that no structure could. Those are different, and the difference is where any useful work lives — verification technology, aligned incentives, agreements that make cheating detectable. Describing why coordination fails is a prerequisite to changing it. Whether that's optimism is a matter of taste; it isn't fatalism.
Argue it yourself ↓ a scoped prompt for any AI — bring your own disagreement

Broken yet?

The irony is noted. You may continue either way. Disagreeing opens a short form that asks why; skip it if you like (privacy).