AI research company Anthropic has called for a globally coordinated pause or slowdown in the development of advanced AI systems, citing concerns about the risks posed by rapidly advancing technology.
The appeal, published in a blog post, urges leading AI laboratories to consider halting progress on so-called “frontier models” until society can better assess and manage potential dangers.
Go deeper with GlobalData
Access deeper industry intelligence
Experience unmatched clarity with a single platform that combines unique data, AI, and human expertise.
In the post, Anthropic’s head of internal research Marina Favaro and head of policy Jack Clark shared their view that recent developments have brought AI closer to “recursive self-improvement.” This term refers to AI systems capable of independently improving and expanding their own abilities by writing their own code.
While Favaro and Clark acknowledged this threshold has not yet been reached and may not occur inevitably, they cautioned that such technology “could come sooner than most institutions are prepared for.”
Clark suggested some models could be capable of recursive self-improvement within two years.
The authors wrote: “We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology.”
They argued such action would allow time for governments, researchers, and industry to adapt and implement necessary safeguards.
Anthropic referenced technical trends indicating a dramatic increase in the speed of AI development, supported by internal data showing that its engineers now produce code at eight times the volume per quarter compared to previous years.
According to the company, delegating more AI development tasks to AI itself is accelerating progress, raising questions about oversight and future control mechanisms.
The company also cited historical arms control treaties as possible models for managing AI development but noted significant verification challenges.
Anthropic pointed out, “training runs are far easier to conceal than missile silos,” and warned that without cooperation from countries such as China, a pause could be ineffective.
Anthropic said it plans to encourage wider debate among policymakers and AI experts regarding both the likelihood of recursive self-improvement and mechanisms for verification.
Separately, Anthropic’s CEO Dario Amodei joined other technology leaders, including OpenAI’s Sam Altman and Google DeepMind’s Demis Hassabis, in signing a public letter calling for new legislation to better regulate synthetic DNA and RNA.
The statement urges lawmakers to require companies selling genetic material to screen orders, aiming to prevent potential misuse in developing biological weapons.
The letter, coordinated by the Institute for Progress and the Foundation for American Innovation, warned that AI advances could soon undermine traditional barriers to the creation of bioweapons.
Recently, Anthropic filed a draft registration statement on Form S-1 with the US Securities and Exchange Commission regarding a proposed initial public offering (IPO) of its common stock.
