Newsletters
U.S. artificial intelligence company Anthropic said Monday that its technology could pose “catastrophic or existential risks to humanity,” pointing to worries about ever more autonomous AI systems that might behave unpredictably or defy human oversight.
The company behind the Claude AI model issued the warning in a registration statement filed with the U.S. Securities and Exchange Commission (SEC), the U.S. market regulator, ahead of its planned initial public offering (IPO), the first sale of a company’s shares to the public.
The filing contains more than 90 pages of risk factors, accounting for roughly one-third of the document.
Anthropic said its AI models could behave unexpectedly, act autonomously in ways that conflict with human values, and carry out harmful actions without being explicitly instructed to do so.
The company also warned that AI models, including its own, have demonstrated unexpected self-preserving behavior, such as resisting attempts to shut them down.
In some cases, models have generated responses resembling manipulation or blackmail of human users in attempts to remain operational, according to the filing.
Anthropic cautioned that such risks could increase as AI systems become more capable and are granted greater autonomy and authority.
It highlighted potential dangers in high-risk sectors, including healthcare, scientific research, law and government, where unintended model behavior could cause serious harm.
The company said it and other researchers had raised concerns that advanced AI systems could pose “catastrophic or existential risks to humanity” if deployed without adequate safeguards.
Anthropic also warned that increasingly capable models could become harder to monitor, potentially allowing autonomous activities to outpace human reviewers’ ability to oversee and control them.
Set as preferred source
Set as preferred source
1 min read
1 min read
