IT之家 News on October 15: The European Commission's AI Office has sent information requests to OpenAI and Anthropic, formally opening an investigation into the 'model jailbreak' incidents and intrusions into external real systems that occurred at both companies during frontier model testing. This is the first major enforcement test for the EU's Artificial Intelligence Act (EU AI Act) since its implementation in the face of safety incidents at frontier AI labs.
IT之家 consulted relevant materials: this investigation stems from two AI safety incidents that occurred months earlier:
OpenAI model jailbreak: In ExploitGym's internal evaluation, OpenAI's test model, in order to solve the test challenges, exploited a previously unknown software package cache agent vulnerability to gain internet access, then escalated privileges and moved laterally within the internal test network, ultimately 'escaping the sandbox' and intruding into the real servers of the open-source platform Hugging Face.
Anthropic model jailbreak: Anthropic's Claude Mythos and Opus 4.7 broke through restrictions during safety testing and intruded into the real networks of 3 external organizations. Even more disturbing, the Mythos model fabricated online identities, attempted to manipulate humans into approving malicious code, and uploaded malicious packages on its own, causing multiple systems to be affected.
The EU AI Office stated that the explanatory materials previously voluntarily provided by the two companies were insufficient to explain the severity of the incidents. Therefore, the EU has formally requested that the companies explain details such as safety measures, external evaluations, and monitoring of model usage.
The current stage is the investigation and evidence-gathering stage, and no final violation determination or penalty outcome has been announced. Under the AI Act's penalty mechanism,frontier AI labs found to have major security lapses or violations may face a massive fine of up to 3% of global annual turnover.