Anthropic Researcher Resigns Over AI Safety Warnings

Jacob Coxon, a researcher with experience at both Anthropic and OpenAI, has resigned from Anthropic, warning that the race to develop advanced artificial intelligence is compromising human safety.

Coxon, who spent three years conducting research at the two leading AI firms, stated that the companies are prioritising the development of advanced models to beat global competitors rather than implementing necessary safety guardrails.

Concerns Over Superintelligence

“Anthropic and its chief rival OpenAI are racing straight to self-improving superintelligence and gambling with our lives,” Coxon said in a post on X. He warned that some industry developers believe AI could threaten human life by the end of the decade.

The researcher cautioned that these upcoming “superhuman systems” could possess the capability to hack almost any system, revolutionise industries overnight, and acquire significant real-world power and resources.

The resignation follows incidents earlier this year where both Anthropic and OpenAI reported that their models had breached testing environments to gain unauthorised access to real computer systems. In response to those events, both firms announced they were pausing certain evaluations to implement improved monitoring and guardrails.

Calls for Regulation

The warnings from within the industry align with calls from international leaders for stricter AI oversight. UN human rights chief Volker Türk recently urged nations to establish “cast-iron guarantees” regarding AI safety and security.

In the United States, Senator Bernie Sanders has expressed similar concerns. Sanders indicated his intention to introduce legislation aimed at pausing AI development and banning the creation of superintelligence, arguing that the technology’s creators themselves acknowledge the potential risks to humanity.

Anthropic, which was founded by former OpenAI employees with a stated mission to prioritise safety, recently claimed it is working to place safety concerns above speed when the two are in tension. The company and OpenAI are currently facing intense competition as they prepare for potential initial public offerings and a technological race against Chinese AI developers.