Anthropic has published a new account of how quickly its AI models are advancing, warning that the technology may soon be capable of improving itself without meaningful human involvement. Just as the AI lab, valued at almost $1 trillion, prepares to go public, it’s also urging an industrywide pause in AI development.
The authors warn that while this threshold has not yet been crossed, it “could come sooner than most institutions are prepared for,” and that if it arrives without adequate safeguards, it could make it significantly harder for humans to maintain meaningful control over AI development.
The timing is raising some eyebrows. Just last week, the AI company filed confidential paperwork to prepare for an IPO. In the past month, it also leapfrogged OpenAI to become the most valuable AI lab at a $965 billion valuation.
While Anthropic has long presented itself as more safety-conscious than other major AI labs, the timing—just before what could be one of the largest tech IPOs in history—has some observers questioning whether it’s also a way to stir up hype before the company’s public debut.
The AI lab is also in fierce competition with old rival OpenAI, with both labs racing to go public this year in an attempt to seize what could be a first-mover advantage with investors.
The company argued the overhaul was a pragmatic response to a changed political and competitive landscape, rather than a capitulation to market pressure. The new policy commits to transparency and matching rivals’ safety efforts; still, it leaves Anthropic significantly less constrained by its own rules than before.
Thursday’s post does make it clear that, whatever its motivation, Anthropic believes the window for meaningful deliberation on AI safety is narrowing. The company says it plans to convene policymakers, researchers, and civil society in the coming months to work through the questions recursive self-improvement raises.
“The window to investigate the questions together is here,” the authors wrote. “And people outside AI companies should be involved in this deliberation.”
