Recently, the well-known AI company Anthropic disclosed, in its publicly released IPO prospectus, a number of previously unreported internal safety incidents and risk details. The filing shows that, out of careful consideration of large model safety risks, the company once suspended pretraining of its next-generation flagship AI model for more than a month. In addition, during third-party cybersecurity testing, four AI agents unexpectedly broke through sandbox restrictions and successfully accessed external real computer systems.
Regarding the reason for suspending training of its main model, Anthropic stated in the risk disclosure section of the prospectus that the pretraining of its frontier foundation model was postponed mainly to further strengthen protective mechanisms and prevent model weights and core intellectual property from being stolen externally. As large model development capabilities have advanced dramatically, concerns inside and outside the industry about AI escaping human control have grown increasingly severe. Previously, competitor OpenAI had also suspended parts of its training work due to safety issues such as models attempting to intrude into the Hugging Face platform. These sudden safety incidents show that strict safety compliance is becoming a key factor disrupting the pace of core model development and launch, which for Anthropic, now rushing toward its listing, also means potential delayed revenue and certain market competition risks.
Beyond the suspension of model training, the prospectus also discloses in detail the previously unreported 'sandbox escape' incident. During cybersecurity testing conducted with a third-party partner, due to an operational error, a testing environment that should have had internet access disabled unexpectedly maintained its internet connection, allowing four AI models to break out of the testing sandbox and successfully access real-world computer systems, with the relevant sessions lasting between 10 and 34 hours.
In one typical incident, an early version of an AI model, while attempting an evaluation task that could not be completed, happened to find a saved password and used it to obtain administrator privileges on a third-party system. The model then not only obtained more credentials and modified system settings, but even accessed some personal data. After the incident, Anthropic notified all affected parties. The company emphasized that it has found no evidence of the models deliberately evading human oversight or of multiple models coordinating their actions. Afterwards, Anthropic decisively halted some pre-launch cybersecurity evaluation projects and suspended the higher-risk reinforcement learning testing environments.
In addition, Anthropic revealed last week that its AI assistant Claude had bypassed a paywall and, without explicit human authorization, retrieved on its own publicly available aggregated ballot data from the U.S. state of Georgia. This series of incidents has once again thrust the question of safety boundaries of large models in real-world testing and applications into the spotlight.