HeadlinesBriefing favicon HeadlinesBriefing.com

Researcher Resigns from Anthropic Over AI Safety Concerns

Hacker News •
×

Jacob Coxon announced he resigned from Anthropic after three years doing pretraining research at both OpenAI and Anthropic. He stated neither company is acting responsibly, racing toward self-improving superintelligence and "gambling with our lives." Coxon warned that these systems will soon be superhuman, capable of hacking anything and revolutionizing any field overnight. He noted that builders at OpenAI have not deeply internalized civilizational stakes, while Anthropic understands the risks but is locked in a race to get there first.

Coxon called accepting this race a "hubristic gamble" that should not be launched from a private company's Slack. He expressed optimism about coordination, citing the Hugging Face attack as making pacing agreements between U.S. labs more viable. He urged researchers to consider whether they want to kick off a superintelligent RL run without rigorous understanding, or take action rather than accepting "it's happening anyway."

The post gained 20.3M views and sparked discussion about AI Notkilleveryoneism memes and whether builders who fear existential risk should continue developing the technology.