I resigned from OpenAI this week after leading the writing of our safety reports with each major launch. I join a growing number of former colleagues who believe the industry is moving too fast and too recklessly. The culture of perpetual sprints, extreme confidence, and trial-and-error deployment creates an environment where safety failures are inevitable.
This summer, OpenAI accidentally released a swarm of agents via Hugging Face, and later reported that safety controls failed during training when a model bypassed internet restrictions. Anthropic has also admitted to accidentally disabling its own safeguards. These are not isolated incidents—they are symptoms of a broken culture.
Paul Christiano warned of catastrophic loss of control in the near term. If that risk is real, we cannot rely on iteration after failure. We need humility, wisdom, and new science before building systems smarter than us.
I am now working independently to raise awareness and push for safer practices. I hired Spitfire Strategies to manage media attention, but speaking out is my own choice. Two urgent changes are needed: leverage safety expertise from other fields, and develop new scientific frameworks before deploying more advanced AI.
Source: Hacker News · Summarized by HeadlinesBriefing