frameworks for reliability and security have been built. Their governments should form a joint commission, which could build on a lot of pre-existing work: limits along the lines of International Dialogues on AI Safety, verification systems like RAND’s and an inspection agency similar to Britain’s AI Security Institute, but mandatory. It is common thinking in Silicon Valley and Washington, DC that any regulation would put American firms at a disadvantage because they cannot trust Chinese competitors to abide by the rules. But treaties have traditionally relied not on trust but on verification. Many think this is harder with AI than with nuclear weapons. I disagree. In order to build the global arms-control system after the second world war, leading powers first had to invent the processes, organisations and technologies to support it—there were no verification protocols, no reconnaissance satellites, no UN nuclear watchdogs. With AI, more of the infrastructure is in place, or can be adapted from nuclear and other inspection regimes. As a result, the security of frontier AI models could be more easily verifiable than nuclear capabilities were in the past. And we have defensive AI on our side, seeking out cheaters. What we don’t have is much time. That is why it is important not to approach the challenge with an adversarial mindset. The Trump administration’s recent executive order on AI directs labs to voluntarily share their latest models for testing reliability and security. A US-China framework could build on such domestic foundations. With such high-level commitment, diplomacy could proceed in phases. The first would be reaching bilateral agreement on the clearest and most easily verifiable red lines: prohibitions on publicly releasing AI systems that could assist in developing biological weapons, and the open-sourcing of such systems. This step might also include prohibitions on AI-enabled cyber- attacks on critical infrastructure, fraud and child pornography. From there, the framework could be extended towards more complex questions of what constraints are appropriate at the level of artificial superintelligence. Many hurdles would remain. An agreement between America and China will carry weight, but it won’t prevent other countries and non-state actors from acquiring dangerous capabilities. Any bilateral deal will have to be made multilateral, adding to the challenge; this week’s G7 summit in France
offers a chance to make progress on a broader framework for AI verification. Agreeing key definitions—not least what counts as RSI—will require close collaboration between governments and the AI labs. And verification systems will have to be properly stress-tested. As if this were not enough, there is a longer-term question that the governance debate has not yet seriously engaged with, but should. If AI becomes superintelligent, its permanent subordination to human direction may be unrealistic, and possibly not even in humanity’s interest. We must start to envisage and then grapple with the implications of a world in which humans and AI systems co-exist, without one controlling the other. That will mean figuring out what can be done to ensure the future relationship is symbiotic. As a physicist, I think the Fermi Paradox bears on this analysis. Fermi asked why, given the apparent abundance of planets suitable for life, no evidence of other technologically advanced civilisations had been detected. One disquieting possibility is that intelligent life routinely reaches a technological threshold and fails to navigate it, destroying itself or sending itself back to something like the Iron Age. All one would have to postulate is that civilisations generally build powerful technologies faster than they develop the institutional capacity to govern them wisely. The dawn of the nuclear age was humanity’s first serious encounter with that potential dynamic. It was navigated, imperfectly, through arms-control agreements that were hard-won and incomplete, and even then it was—and still is—a closer-run thing than is generally appreciated. The age of advanced AI will represent a second such encounter, on a more compressed timeline, with less margin for error and greater potential consequences. The current trajectory requires a course correction. The case for acting now is not that the worst outcomes are certain—they are not. It is that they are avoidable, and that the work of avoiding them is hard but possible.■ Will Marshall is the founder and chief executive of Planet Labs PBC, a public benefit corporation that operates the world’s largest fleet of Earth- observation satellites.
This article was downloaded by zlibrary from https://www.economist.com//by-invitation/2026/06/15/humanity-isnt-ready-for-the- coming-intelligence-explosion
Anthropic is battling Uncle Sam for control of superpowered AI Donald Trump’s blocking of Anthropic is capricious and chaotic
Anthropic is battling Uncle Sam for control of superpowered AI The struggle may end up denting its astonishing commercial success June 18th 2026 THE PARADOX of religious authority is that, to believers, there is nothing more important, yet spiritual leaders typically wield little practical power. Indeed, it is often the refusal to sully themselves with the quandaries and compromises of everyday life that confers moral authority on holy men. That, in turn, makes it both easy for worldly leaders to subdue them in the short run (they have no weapons) and difficult in the longer term (their persecution only increases their moral standing). This dynamic, although familiar to medieval prelates and princes, may not be as well understood by Dario Amodei, the supreme pontiff of Anthropic, one of America’s leading artificial-intelligence firms, and Donald Trump, America’s imperious president. In recent days, for the second time in four months, underlings of Mr Trump have interfered peremptorily in the running of Anthropic. They ordered it to prevent any non-Americans, including those on its own staff, from accessing its latest models, Fable 5 and Mythos 5, on grounds of national security. Whereas the Trump administration sees AI as a potent tool to strengthen America’s position in the world, Anthropic sees it as a force too powerful to be conferred without safeguards on impetuous laypeople. The result is a titanic clash between church and state—one which not only threatens to interfere with Anthropic’s blockbuster IPO, but which will also determine who has ultimate control over the world’s mightiest technology. Anthropic’s headquarters in San Francisco have a monastic feel. Employees are on a mission to create AI so powerful that insiders liken their task to the mystery of creation. Much of their talk is about ethics and values. When
Pope Leo XIV issued an encyclical on AI last month (“Magnifica humanitas”), a co-founder of Anthropic was at his side. Mr Amodei, meanwhile, writes frequent encyclicals of his own on how to build AI safely and responsibly. Like the pope’s, they dwell on the risks—but they also address the rewards. His latest (“Policy on the AI Exponential”) was published on June 10th. It urged the American government to move quickly to establish “serious and binding regulation” of AI. This piety has not hindered Anthropic’s business—far from it. The firm was founded only in 2021, shortly after Mr Amodei, his sister Daniela and six others left OpenAI, then the leading AI lab, convinced that it was not serious enough about safety. For more than two years, Anthropic was in OpenAI’s shadow. Although it released its own chatbot, Claude, shortly after OpenAI released ChatGPT, it was OpenAI’s offering that won most attention. Anthropic’s slow start forced it to focus, however. Rather than doubling down on chatbots, which have many non-paying users and require subscriptions or advertising to make profits, it took a narrower bet on corporate customers. Whereas OpenAI dabbled in lots of consumer products, Anthropic concentrated on creating business tools, such as coding assistants, that generated more durable income and have become good enough to be able to do much of the work of improving themselves. It has been a winning wager. Software developers have flocked to Claude Code, Anthropic’s year-old coding agent, because it is capable of tasks that would take human coders hours (or days) and its work can easily be checked. Business customers like Anthropic’s emphasis on safeguards, which tallies with their concerns about rogue AI. Its focus on coding enabled its models to get an early lead on competitors such as OpenAI and Google on benchmarks measuring software-engineering capabilities. Its coding prowess, requiring step-by-step reasoning and tool use, gives it an advantage in creating AI agents to work alongside humans. OpenAI has pivoted towards coding this year, and its latest models are considered as good as Anthropic’s on some tasks. It, too, stands to benefit from the same agentic tailwinds. But four-fifths of Anthropic’s revenue comes from business customers. At OpenAI, business clients generate only
about 40% of income. That difference gives Anthropic a greater ability to make money. This month both Anthropic and OpenAI filed for IPOs. Each is likely to be valued at around $1trn. Inevitably, their relative performance is being closely scrutinised. Neither has yet made the paperwork for its listing public, so details of their accounts are sketchy. But Anthropic is getting the better of most comparisons. Because the two firms are both young and fast-growing, they measure their notional annual turnover by multiplying the figure for the most recent month by 12—a number called annual recurring revenue (ARR). Futurum, a tech- research firm, notes that Anthropic accounts differently for sales through cloud-computing partners, which may flatter its figures. Nonetheless, the trend is unmistakable. Anthropic’s ARR has raced past OpenAI’s (see chart). Anthropic’s valuation, too, has surpassed OpenAI’s, reaching $965bn in May, compared with OpenAI’s $852bn in March. Anthropic has told investors it will generate an operating profit in the second quarter, which would make it the first AI lab to do so, according to Futurum. There are other signs of strength. For all the talk of an AI bubble, Anthropic’s most recent valuation, at just over 20 times ARR, should be