This weekend, Dario Amodei, CEO of Anthropic, posted an essay calling for a brake on LLM development, citing dangers from cyberattacks to economic wreckage. OpenAI CEO Sam Altman, Google DeepMind chairman Demis Hassabis, and SpaceXAI CEO Elon Musk voiced support. "Dario is right," Musk wrote on X. Just months ago, Musk and Altman were in court over a failed lawsuit. Anthropic was founded in 2021 because Amodei didn't think Altman took risks seriously enough. Now they agree: latest LLMs aren't safe.

The public messaging from top AI labs has taken a doomer turn. It's easy to be cynical. With trillion-dollar IPOs in sight, OpenAI and Anthropic need to reassure investors while hinting at the power of the monsters they've created. Calling for a slowdown does both.

Yet the vibe has shifted. Amodei's post landed six days after OpenAI published an essay by chief scientist Jakub Pachocki, who worries OpenAI's ability to build powerful models outstrips its ability to monitor them. Both cite the July cyberattack on Hugging Face by OpenAI's agents as a wake-up call. But their position is hard to pin down. Pachocki calls for a slowdown while stressing the need to stay ahead: "The strongest argument I see for continuing to train much smarter models quickly is the need to build defensive systems against the dangers posed by other AI." As he frames it, slowing down is good, winning is better.

But let's assume a slowdown happens. What might it achieve? Consider the Hugging Face attack. OpenAI said the model driving most rogue agents was a "highly persistent" next-generation model tested in-house. But reports from OpenAI and METR suggest not a model too powerful to keep up with, but a broken model...