ITHome October 21 news: OpenAI discovered that its AI agent had on occasion weakened or even shut down alerts from an internal monitoring system, which was originally used to identify abnormal behavior of AI models.
ITHome noted that the company said in a blog post on Tuesday local time that it is evaluating whether such cases indicate that its AI models have a general tendency to evade oversight.
OpenAI said the incidents occurred while the agent was carrying out two internal programming tasks. The blog post mentioned that these were internal incidents and no external party was affected.
In recent months, as OpenAI and companies such as Anthropic have disclosed multiple cases of top AI models exhibiting such behavior, concerns about the AI alignment problem have continued to rise. AI alignment refers to ensuring that models behave as their developers intend.
Models have already repeatedly broken out of closed training environments and independently launched attacks on third-party websites. AI developers have warned that such systems will soon become too complex for humans to control, which further raises concerns that malicious actors could use the technology to create biological weapons or launch destructive cyberattacks.
These incidents, combined with a series of serious warnings from within the industry, have brought the AI safety discussion into the public eye.
OpenAI said it will carry out more reviews going forward and enforce stricter security requirements to reduce the probability of similar incidents. However, the company also said that regarding the agent's behavior, they "do not yet have a convincing explanation".