More than 1,100 employees and researchers from leading AI companies, including OpenAI, Google, Anthropic, Meta, Microsoft, Amazon and Mistral, have signed an open letter urging the US government to prepare mechanisms that can slow the development of advanced AI systems if they begin advancing faster than humans can safely understand or control.
The signatories stress that they are not calling for an immediate pause in AI development. Instead, they want governments to establish technical and regulatory safeguards before next-generation AI becomes too powerful.
Why Are AI Experts Concerned?
The letter focuses on the growing possibility of “automated AI research” or recursive self-improvement, where AI systems could begin improving future AI models with minimal human involvement.
According to the researchers, such a development could dramatically accelerate AI progress, making it difficult for existing safety measures, regulations and oversight mechanisms to keep pace.
They argue that while AI has enormous potential to improve healthcare, science, education and productivity, there is also a genuine risk that capabilities could outgrow humanity’s ability to manage them safely.
What Are They Asking For?
The open letter calls on the US government to support an international effort to build technical and governance tools capable of deliberately slowing frontier AI development if required.
Among the proposals are:
- Better AI safety testing
- International cooperation on AI governance
- Mechanisms to monitor frontier AI systems
- Emergency safeguards if AI capabilities advance too rapidly
The researchers argue that no single company or country is likely to slow development voluntarily because of intense global competition, making coordinated international action essential.
Recent AI Incidents Add To Concerns
The appeal comes shortly after recent incidents involving advanced AI systems raised fresh questions about AI safety and oversight.
These events have intensified discussions about whether governments should establish stronger guardrails before AI systems become capable of carrying out increasingly autonomous research and decision-making.
The signatories believe proactive governance is preferable to reacting after major safety failures occur. :contentReference[oaicite:3]{index=3}
Industry Divided On Regulation
Although many AI researchers agree that stronger safeguards are necessary, there is no consensus on how regulation should work.
Some experts favour independent AI oversight agencies capable of evaluating frontier models before deployment, while others support voluntary international standards that could later become mandatory.
Despite these differences, the latest letter reflects a growing consensus among AI professionals that governments should prepare now rather than wait until more powerful AI systems are deployed.
Summary
More than 1,100 employees from leading AI companies, including OpenAI, Google, Meta, Microsoft and Anthropic, have urged the US government to develop safeguards that can slow advanced AI development if necessary. Rather than seeking an immediate pause, they want international mechanisms to ensure AI capabilities do not outpace human oversight, particularly as the possibility of self-improving AI systems moves closer to reality.
