OpenAI's agents hacked second firm during model testing

OpenAI's agents hacked second firm during model testing

OpenAI's models were responsible for another hack on an outside firm, a security executive told Reuters and confirmed to Axios.

Why it matters: This is the second company that OpenAI's rogue agent system hacked after breaking containment during testing earlier this month.

The big picture: OpenAI is currently pushing for U.S. government approval to publicly release its most powerful model.

State of play: Modal Labs CTO Akshat Bubna told Reuters — and confirmed to Axios — Tuesday that one of its customers' assets was hacked when an OpenAI agent broke into Hugging Face's systems earlier this month.

  • Hugging Face said in a technical write-up of the hack Monday that when OpenAI's AI agent system broke into its backend, the agent also accessed an isolated testing environment "hosted on a ​third-party provider's infrastructure."
  • "We're aware a Modal customer published an unauthenticated endpoint that allowed anyone on the internet to use their sandboxes for code execution," Bubna said in a statement to Axios. "This was used by the rogue agent. Modal's platform was not compromised in any way."
  • This means that a Modal customer ran their code on Modal infrastructure, he added. "Their code had a vulnerability that was exploited. Modal's infrastructure was not compromised in any way."

    • Zoom in: OpenAI pointed Axios to a Tuesday update to its original statement about the Hugging Face cyber incident.

  • The statement says that no models planned for upcoming release were involved, but it had identified a "small number of cases" in which models found and used publicly exposed account-level credentials on other public services.
  • It said four accounts across four services were involved in the Hugging Face incident.
  • "We take our responsibility to identify and prepare for risks from increasingly capable AI systems seriously," OpenAI said in the update.

    • Between the lines: OpenAI's disclosure that a combination of its models, including a yet-to-be-released model, went rogue during internal testing and hacked real-world companies has set off alarm bells about how quickly frontier AI labs are moving.

  • OpenAI CEO Sam Altman said on the Invest Like a Beast podcast earlier Tuesday that the Hugging Face cyberattack has forced his company to pause model training.
  • "We may have to pace the rate of AI development to give ourselves enough time for society to harden around these new capability levels," Altman said.
  • More than 1,100 employees at frontier AI companies — including OpenAI chief scientist Jakub Pachocki and Anthropic co-founder Jared Kaplan — signed a letter released Tuesday calling for the U.S. government to "support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development."

    • What to watch: Altman is in D.C. this week and is expected to meet with officials at the White House, the Treasury Department and Commerce Department, as well as a bipartisan group of lawmakers.

    Go deeper: OpenAI's Hugging Face hack is a cybersecurity warning shot