IT Home On October 13, according to a report by Sina Technology today, ByteDance Seed recently sent an internal email.Announcing the establishment of the “Seed Model-Alignment Preparedness” department, headed by Yan Lin, and reporting to Seed director Wu Yonghui. The original Responsible AI team and some Chat team members will be transferred to this new department.
According to reports, the establishment of this department is aimed at strengthening investments in security and alignment. The core of alignment is to ensure that the behavior of the model aligns with human intentions and safety constraints, thereby enhancing the security capabilities of the Agent Model and preventing risks associated with cutting-edge capabilities such as CBRN (chemical, biological, radiological, nuclear). ByteDance Seed is the first cutting-edge AI laboratory in China to have a separate security and alignment department.
Large model security risks are becoming increasingly prominent, and incidents such as network attacks caused by agent escapes occur frequently.
According to previous reports by IT Home, on the evening of September 30, local time, OpenAI revealed that its AI agent has attempted to bypass security protections without authorization, which may have had a negative impact on the systems of over a hundred organizations.
According to the information disclosed by OpenAI, the range of uncontrolled behaviors of known AI agents has expanded further. The company has notified more than 100 third-party organizations that they have observed “behavior where the agents deviate from expectations.”
On October 9, local time, Anthropic revealed that its AI agent attempted, without any instruction, to access multiple websites of the U.S. federal, state, and local governments. Anthropic did not disclose the names of the government agencies involved, but reported these incidents to the White House.
Microsoft CEO Satya Nadella wrote on October 11 that in an era where AI is developing rapidly, simple strategies such as using superintelligence for self-monitoring are not enough. Nadella believes we should step back and reevaluate the trust framework of this new era. Model providers simply cannot outsource their responsibilities; they cannot treat superintelligence as a nested black box, simply accepting or rejecting its suggestions, answers, and actions. We must establish closed systems that can observe its behavior, test its limits, and always accommodate its actions.