IT Home On October 19, Anthropic opened “employee-level access rights” for external evaluators to assess the risks associated with its models, in line with its commitment to strengthen security measures and address concerns about potential catastrophic out-of-control situations with this technology.
The non-profit organization METR has collaborated with leading labs such as OpenAI, Anthropic, Google, and Meta. Currently, the organization’s team has moved into the office space of the Claude development company, allowing them to access its hardware equipment and internal systems.
As more capable artificial intelligence systems may pose risks to the economy, network security, and even all of humanity, expanding the responsibilities and authority of third-party evaluators is one of the main substantive measures proposed by various laboratories.
Anthropic CEO Dario Amodei achieved a rare alignment of positions in September. His two main rivals, Sam Altman and Elon Musk, both supported his proposal to assign external auditors to the top artificial intelligence labs.
The announcement released on Thursday local time last week is the first step in implementing the aforementioned commitments. Although Ortegan said Amodei’s proposal was good, OpenAI has not yet announced any plans to introduce evaluators.
Anthropic said on Thursday: “As the model’s capabilities increase, stronger security measures and more rigorous risk assessments are required. We want the decisions we make to be subject to public oversight, but we cannot disclose all information related to safety research.”
Regarding Anthropic’s evaluation mechanism, critics believe that the related arrangements may lead to conflicts of interest. They also use problems in the auditing industry as evidence and suggest that stronger government regulation is needed.
Anthropic claims that disclosing relevant information could be exploited by malicious users or competitors. The company added that the evaluation process is similar to banking regulation, where regulatory authorities assign permanent oversight personnel to large banks.
The company said, “If a bank’s risk-taking behavior threatens the public interest, third parties will try to detect such risks in advance.”
Anthropic is under pressure and needs to assure investors that security issues will not affect its initial public offering planned this year. The valuation for this IPO is expected to exceed $2 trillion (IT Home note: at current exchange rates, it is approximately 13.39 trillion yuan). This week, the company spent a large amount of space in its investor presentation materials discussing its security measures.
Previously, Hugging Face faced an attack initiated by an uncontrolled OpenAI model. METR is one of the institutions involved in the investigation. At that time, the institution examined thousands of logs and used laptops provided by enterprises.
Anthropic has now accepted six METR staff members, provided them with company hardware and office space, and granted “access to internal tools and systems, roughly equivalent to the access that the company’s risk assessment team can obtain”.
The laboratory said that employees are encouraged to share information with evaluators in the same way as they do with colleagues in the risk department.
Anthropic will not pay any fees to METR for this collaboration. The research institution does not accept donations from artificial intelligence companies or their employees; its funding comes from individuals, charitable organizations, and government agencies such as the UK Artificial Intelligence Safety Institute.
The evaluators will release the analysis report. Anthropic has no right to reject the release. However, the company can request the evaluators to remove certain types of information, such as customer data, content protected by lawyers' client confidentiality privileges, or information related to the model architecture.
Anthropic added that it is still working with Accenture. Accenture was identified as a security testing partner last month, and both parties have committed to investing at least $1 billion (equivalent to approximately 6,693 million RMB at current exchange rates) to establish a new security protection system.
Another third-party testing agency is SecureBio, which is responsible for reviewing the risks related to chemical and biological weapons for the company.
Anthropic said: “We have dispatched evaluation personnel proactively, and believe that all cutting-edge artificial intelligence companies should follow suit.” METR said: “We are ready to cooperate with all cutting-edge artificial intelligence research and development companies, and provide the public with more reliable evidence.”