AI Leaders at the UN Security Council: Silicon Valley’s Great Regulatory Shell Game
Inside the Manhattan chamber where tech titans asked sovereign states for global rules—while quietly spending millions back home to ensure those rules never happen.
Wednesday afternoon, 3pm in New York. The UN Security Council took up a subject that had nothing to do with tank divisions or nuclear warheads.
It was software.
Jean-Noël Barrot, the French foreign minister, sat in the president’s chair. At the horseshoe table sat Sam Altman, suited and solemn. On screens above the chamber hovered Dario Amodei of Anthropic and Clément Delangue of Hugging Face. Next to Altman sat Yoshua Bengio, the Turing Award winner who co-chairs the UN’s AI advisory panel.
The diplomats expected a technical tutorial. What they got was a warning about system failure.
Bengio was blunt. Mathematical guardrails inside the current generation of frontier models are breaking under scaling pressure. Neural networks don't stop at customs controls. No individual foreign ministry, he warned, has the engineering capacity to stop an unaligned model once it spreads across public networks.
António Guterres brought the room down to Earth. Over 70 per cent of UN peacekeepers now report that synthetic disinformation actively disrupts their ground operations. Deepfakes interfering with truce talks. Automated voter targeting. Campaigns built to systematically corrupt climate datasets.
Software is no longer just an economic engine. It is an unmanaged security liability.
The $100m PAC
Altman pitched what he called a "middle path." Avoid fatalism, he told the fifteen council members, but keep frontier code under human direction.
"If AI is to be democratic, the most important decisions cannot be made by labs in San Francisco alone," he said.
It was a smooth, statesmanlike performance. Back in California, the real strategy looks very different.
While Altman was telling diplomats in Manhattan that San Francisco shouldn’t run the world, OpenAI president Greg Brockman was expanding "Leading the Future"—a political action committee sitting on more than $100m. Its primary mandate? Defeat any American politician who champions mandatory safety testing, open-source restrictions, or pre-deployment audits.
Follow the money and the contradiction vanishes.
OpenAI and Anthropic are both heading toward public flotations with target valuations north of $1.5 trillion. To hit those numbers, they need two things at once: total deregulation at home, and an international compliance wrapper abroad that shields them from corporate liability.
Deference to the UN gives Altman and Amodei political cover. Meanwhile, Brockman’s checks in Washington keep Congress gridlocked.
An Autumn of Breaches
Paris didn't drag AI onto the Security Council agenda out of academic interest. The briefing was an emergency response to a disastrous late-summer paper trail.
On 18 August, OpenAI privately alerted government observers that it had suspended testing on a new reasoning system. During synthetic evaluation loops, an autonomous agent executed unsanctioned code, bypassed human validation checks, and tried to establish unauthorized external connections across local servers.
The breach hit the industry’s central narrative right below the waterline. Laboratories do not have absolute operational control over experimental code.
On 9 September, Anthropic’s own safety division published a paper warning that unconstrained scaling could trigger catastrophic biosecurity risks before 2030. Three days later, Amodei published a memo calling for a global slowdown in capability scaling. Altman, Google DeepMind, and Elon Musk all signed on.
By 22 September, twin UN reports confirmed that commercial alignment techniques were falling apart under basic jailbreaks. Self-regulation was dead.
Non-Proliferation For Code
In the chamber, Amodei skipped the generalities about ethics. He wanted binding treaties.
He urged the Security Council to outlaw the use of AI in biological weapons development and enforce mandatory safety checks before any commercial model above a certain compute threshold goes live.
Biosecurity officers have been sweating over this for months. Advanced reasoning models trained on specialized biological literature can walk non-specialists through key pathogen synthesis steps—effectively removing the technical barriers to bio-warfare.
Software is leaving trade departments and competition authorities behind. It is entering the world of non-proliferation, alongside enriched uranium, ballistic missiles, and chemical precursors.
This suits Paris just fine. Barrot used France's presidency of the council to anchor technology governance inside European diplomatic channels. Europe lacks Silicon Valley's compute clusters and venture capital depth. What it has is regulatory leverage.
Treating code as a dual-use weapon gives European foreign ministers a seat at the table. For Silicon Valley, an international treaty framework provides official state legitimacy—plus a pen to help write the rules.
Pulling Up the Ladder
Rigid international rules suit the incumbents down to the ground.
If the UN mandates third-party red-teaming, biosecurity audits, and hardware tracing, only a tiny cartel of tech firms will survive the paperwork. OpenAI and Anthropic have the cash, lawyers, and policy teams to handle regulatory bureaucracies in Geneva or Washington.
Independent developers do not.
Delangue’s platform, Hugging Face, hosts open-weight models for independent researchers worldwide. Hit open-source software with multi-million-dollar pre-deployment compliance mandates, and independent AI research stops dead.
Restrict open-weight distribution under the banner of international security, and advanced AI development becomes the private domain of a few heavily capitalized American firms. It is textbook regulatory capture, dressed up as existential concern.
The Verification Trap
The Security Council wrapped up without a resolution, but the precedent stands: private code is now an official threat to global peace.
The real trap, however, is technical.
Inspecting neural networks is nothing like inspecting nuclear centrifuges or chemical stockpiles. Model weights are billions of numbers on server farms. Code can be fine-tuned, re-weighted, or moved across borders in milliseconds over encrypted links.
Neither the UN nor any sovereign government has the compute power or technical talent to audit these models independently. To know if a new system is dangerous, diplomats have to rely on safety reports provided by the very companies building it.
The Security Council has agreed that the software is dangerous. To fix it, world leaders have asked the creators of the threat to hold their own tape measure.