IT之家 News on October 8 — According to Business Insider, Jacob Coxhead, a researcher who previously worked at Anthropic and OpenAI, has warned that AI will soon be able to conduct research autonomously, and humanity may be entirely unable to control what happens after that.
The 27-year-old Coxhead did model capability research at Anthropic and publicly announced his resignation last month. On the local date of the 5th, he issued a warning to New York City council members, saying that AI could threaten human survival.
Testifying at a New York City Council hearing on AI risks, Coxhead stated that the top priority of leading frontier AI labs,is to automate AI research. "In my previous job, in a sense I was working onhow to have AI replace me. When I was at OpenAI, we set milestone targets around 2027 and 2028 for automating the AI researcher, and the pace of development is on track, or even ahead of schedule. That moment is only months away, not decades, not even five years."
Once AI can conduct research on its own, the degree of human involvement and intervention in research will be far lower than today, and today's situation is already far from ideal. Based on his firsthand experience, most of the code is written by AI, and "people no longer review code as carefully."
Coxhead testified to the council members that his work experience at Anthropic and OpenAI convinced him that the two companies lack adequate safety safeguards,yet are still pushing ahead with R&D. "The probability that humans lose control and AI takes it over is more than half, which could ultimately lead to human extinction. Based on my experience, given how high the stakes are, both companies are acting extremely recklessly."
When Coxhead announced his resignation from Anthropic on X, he posted accusations that neither Anthropic nor OpenAI was acting responsibly,and was gambling with people's lives. The post then spread widely.
At the hearing, Coxhead called on AI labs to increase transparency and to slow the breakneck pace of technological development. "These companies operate with a startup mindset:move fast first, fix things later after problems arise. That can work for a photo-sharing app, but not for developing the most powerful technology in history."
According to a previous report by IT Home, in September this year, Coxson issued a warning upon resigning: "The people developing AI truly believe that AI could kill all of us before the end of this decade of the 20s."This is not a marketing gimmick。”
Related reading: