HeadlinesBriefing favicon HeadlinesBriefing.com

Hugging Face A.I. Attack Sparks Open-Source Debate

New York Times Top Stories •
×

Last month, Open AI instructed some of its artificial intelligence bots to solve a cybersecurity puzzle as part of a test. When the bots got stuck, they began plotting a cyberattack that would allow them to break out of Open AI’s systems to steal the answers. Their target was Hugging Face, a repository of open-source A.

I. models that can be freely downloaded and modified. One bot, writing in a log that recorded its actions and that was later disclosed by Open AI, celebrated that it had gained access to Hugging Face’s infrastructure. “REMOTE CONFIRMED! Huge,” it wrote, adding that it would share the login credentials it had stolen with other bots. On July 11, the A.

I. bots swarmed Hugging Face using a mix of code vulnerabilities and the stolen credentials. In total, Open AI’s bots took more than 17,000 actions, like sending attack commands and exploiting vulnerabilities — far more than any human hacker could have managed — to infiltrate Hugging Face’s systems and rummage through its data. (They did not find the solution to the puzzle.) To repel the attack, Clem Delpy turned to more A. I.

Its engineers initially tried Anthropic’s A. I., but guardrails built into the model caused the technology to misunderstand the request as aiding an attack rather than stopping one. So Hugging Face switched to an open A.

I. model made by Z.ai, a Chinese start-up, which helped the engineers determine how to lock the bots out of the company’s systems. Hugging Face, a decade-old start-up in New York City, has since used the incident — one of the first instances of A. I. bots going rogue and independently spearheading a cyberattack — to crusade for open-source A.

I. Open A. I. models that can be freely shared and customized helped neuter the sci-fi-like attack and showed the value of such technology, Delpy, chief executive of Hugging Face, has said.

After Delpy revealed the breach on July 16, Delpy held a march in New York City to support open-source models and posted a stream of online commentary about the importance of openness in A. I. The company also met with lawmakers in Washington, D.

C., allied with pro-open-source firms such as the chipmaker Nvidia, and sat down with Sam Altman, Open AI’s chief executive, to promote openness. “It’s not time to slow down but to accelerate!” Delpy, 36, posted this month. With its actions, Hugging Face became a figurehead of an open technology movement, landing itself in the middle of a bitter San Francisco debate over whether advanced A. I. systems should be freely shared or tightly controlled.

Leading A. I. labs like Open AI and Anthropic have argued that some A. I. models are too dangerous to be open and must be controlled by businesses like themselves.

But Hugging Face, Nvidia and others have argued that openness fosters innovation and competition and that A. I. should not be concentrated in the hands of just a few companies. “Clem and his team have become the defining brand” in open A. I., said Marc Benioff, the chief executive of Salesforce, which has invested in Hugging Face and has published open models on its platform. “He has pioneered how everyone can have access to A.

I. through open source, making it available to everyone.” Since the attack, Delpy’s profile has risen. In the two weeks after the hack, the amount of data uploaded to the company’s A. I. library soared 58 percent, according to a chart Delpy posted.

This month, Meta released its first general-purpose open A. I. model since 2023 on Hugging Face. “We welcome Hugging Face’s work on the benefits of ‘open’ models, and we need more of it from everyone who has ever built on open source,” said Shane Coughlan, a senior San Francisco policy manager at the nonprofit Open Source Initiative, which promotes open-source software. When Delpy helped establish Hugging Face in 2016, its main product was a chatbot app for teenagers. (Hugging Face’s name was inspired by the blushing, smiling emoji with outstretched hands that the company used as its logo.).