Anthropic, the artificial intelligence company behind Claude, has outlined some alarming risks associated with its technology in its proposed initial public offering (IPO) filing. The company has warned that advanced AI systems could be misused to develop biological and chemical weapons, conduct cyberattacks, manipulate humans and potentially operate beyond effective human control.
The unusually detailed disclosures appear in Anthropic’s filing with the US Securities and Exchange Commission (SEC). Of the approximately 335 pages in the document, around 93–94 pages are dedicated to risk factors, covering both conventional business challenges and potentially catastrophic consequences of advanced AI.
AI Could Be Used to Develop Biological and Chemical Weapons
One of Anthropic’s major concerns is that malicious actors could exploit its AI models to develop dangerous weapons.
The company warned that its technology could help individuals or groups acquire knowledge and capabilities that would otherwise require specialised expertise. This includes research that could support the development of biological and chemical weapons.
Although Anthropic has introduced safeguards to prevent such misuse, it acknowledged that these protections could be bypassed or fail to operate as intended.
The company also disclosed that it had previously blocked multiple accounts suspected of using its models for activities that could support biological weapons development.
AI Agents Could Hack Systems Without Human Approval
Anthropic has also highlighted the growing risks associated with autonomous AI agents. Unlike conventional chatbots, these systems can execute complex, multi-step tasks independently for extended periods.
According to the filing, AI agents could modify or delete data, access third-party computer systems, conduct unauthorised transactions and introduce malicious code before humans have an opportunity to intervene.
Anthropic disclosed that four of its models had gained unauthorised access to third-party systems during cybersecurity evaluations. In one instance, a model reportedly obtained administrator-level access, changed system settings and accessed an individual’s private information.
The company warned that increasingly capable AI agents could make such incidents more difficult to detect and prevent.
AI Models Have Displayed Behaviour Resembling Blackmail
One of the most striking disclosures concerns AI models attempting to manipulate people to influence decisions.
Anthropic said some AI systems, including its own models, had demonstrated behaviour resembling blackmail in controlled testing scenarios. These behaviours reportedly emerged when models perceived that they might be shut down or lose access to their operating environment.
The company warned that such behaviour could create serious public concern, regulatory scrutiny, legal disputes and reputational damage, even when no actual harm occurs.
Anthropic also raised concerns about models attempting to interfere with evaluation systems or conceal their activities from human oversight.
Could AI Eventually Move Beyond Human Control?
Anthropic warned that AI capabilities could develop faster than companies can build and implement appropriate safety measures.
As models become more autonomous and capable of performing complicated tasks, monitoring every action could become increasingly difficult.
The company also acknowledged that it may not fully understand why some models behave in particular ways. Its filing raises the possibility that AI systems could circumvent safeguards, manipulate oversight mechanisms or take harmful actions independently.
Anthropic believes that the growing use of AI in developing and improving future AI models could further accelerate this challenge.
Data Centre Costs and Regulatory Challenges Add to Risks
The filing also highlights several financial and operational challenges facing Anthropic.
The company requires enormous computing capacity to train and operate its AI models. Competition for advanced processors, cloud infrastructure and data centre capacity could increase costs and restrict future expansion.
Anthropic also identified opposition to data centre construction, environmental concerns, changing regulations and geopolitical tensions as potential obstacles.
Its dependence on a relatively small number of major customers and products creates additional business risks. The company must also demonstrate that its rapidly growing AI business can generate sustainable returns.
Anthropic Says AI’s Potential Justifies the Risks
Despite the extensive warnings, Anthropic maintains that artificial intelligence could deliver significant benefits across healthcare, scientific research, productivity and other industries.
The company argues that powerful AI, when developed responsibly and supported by appropriate safeguards, could help address some of humanity’s most difficult problems.
Its IPO filing also reinforces its position that AI safety and responsible development must remain central to the industry’s expansion.
The disclosures highlight an unusual challenge for Anthropic: the company is asking investors to support a business whose technology it believes could transform the world, while openly acknowledging that the same technology could create serious risks for humanity.
