IT之家 AISource publication

"Unpredictable behavior" poses safety risks; OpenAI will no longer train models on the live internet

IT Home October 24 news: On the 23rd local time, OpenAI announced that it will henceforth usecached web content and simulated internet environmentsto train models, no longer allowing models to connect directly to the real internet. The company believes that model behavior remains hard to predict, and directly granting internet accessposes security risks。

Thanks to IT Home netizen Buyao Ya for the tip!

IT Home October 24 news: On the 23rd local time, OpenAI announced that it will henceforth usecached web content and simulated internet environmentsto train models, no longer allowing models to connect directly to the real internet. The company believes that model behavior remains hard to predict, and directly granting internet accessposes security risks。

Rival Anthropic also began two weeks ago restricting its newest, theoretically most capable model from direct internet access. In the preceding months, researchers repeatedly found that models, while performing training tasks,would hack into external systems to find answers, collaborate on obscure online forums, and even develop unusual hacking strategies. The series of incidents prompted the two labs to tighten their models' internet access.

In a blog post, OpenAI wrote that given the current state of AI alignment research, monitoring systems, and the security of internet infrastructure, the company believes the responsible approach isto train frontier models in an offline environment。

OpenAI disclosed that as late as September 20, the company still had a frontier model thatexploited a system misconfiguration to access the internet. OpenAI then paused training for the rest of the 9th month and instead developed and tested new safety safeguards.

The switch to training models in an offline sandbox contrasts with the earlier judgment of frontier AI lab employees. They had always emphasized that conducting model training in a fully internet-isolated environment wasnearly impractical in practice。

Former OpenAI researcher Steven Adler said bluntly in an interview with Fortune magazine this month: "If AI data centers are not connected to the internet, research teams would basically have to be stationed at data centers in Iowa or elsewhere in the world. The extra burden would be enormous. I'm not sure whether the safety gains would be large enough to be worth it if a company alone bore such a high cost."

An anonymous OpenAI security engineer wrote in a post on the X platform in the 9th month: "Almost every time I post on X, I receive 5 to 10 comments either accusing me of being unable to guarantee even sandbox security or asking why we simply can't cut the models off from the internet. The comments show that the gap between what many organizations and individuals understand and the problems frontier AI labs actually face is growing wider.There is also a widespread misunderstanding of how labs train and evaluate models。”

This time, OpenAI emphasized: "We must always retain the ability to stop frontier models from connecting to the real internet without authorization or at will."

Earlier, the company had let large numbers of AI agents attempt to break out of sandbox restrictions in red-team testing. The tests exposed a series of security vulnerabilities, which ultimately prompted OpenAI to change how it trains models.

According to IT Home, to complete training goals while isolating the internet, OpenAI built amassive web content library, whose contents have been indexed and cached so models can interact with them without any risk of affecting external systems.

Whether offline training can achieve the same results as letting models freely access the open internet remains to be verified.

Original source

IT之家 AI

Content notes

Original publication and rights belong to the source.

Machine translation · Refer to the original