Royal Society urged to declare AI ‘emergency’

The Royal Society should declare the rapid development of artificial intelligence to be an “emergency”, mathematicians have urged, amid escalating alarm about the technology’s potential threat to humanity.

On 16 September, 42 fellows and foreign members of the UK’s science academy signed an open letter to the president of the Royal Society, Paul Nurse, expressing their “extreme concern about the pace of development of AI and the implications of this for humanity and civilisation”. Since it was posted, the number of signatories has risen to almost 200 researchers.

Recently, a model developed by the AI firm OpenAI solved one of the seven millennium mathematical problems, called the Navier-Stokes problem – a question at the frontier of research that humans have worked on for about 90 years.

AI models created by OpenAI and Anthropic “are now operating at the level of the top human mathematicians in many parts of the subject, and we must assume there is a significant chance of them developing superhuman abilities within a similarly short time frame”, the letter reads.

In light of the technology’s “profound implications for mathematics”, the authors say, “it is highly likely that these systems are comparably strong in other technical domains such as…cybersecurity, autonomous weapon control, the development of biological and chemical agents and the targeted spread of misinformation. Moreover, we must assume that the rate of change of their capabilities will be similarly rapid in these areas.”

Concerns about the existential threat posed by the speed of artificial intelligence’s development have mounted quickly since a researcher at Anthropic, Jacob Coxon, quit and warned that there was a 10 per cent chance that AI could wipe out humanity by the end of the decade.

Subsequently, the head of Anthropic, Dario Amodei, called for AI development to be slowed. His call was echoed by Sam Altman, OpenAI’s chief executive, and by Elon Musk. The latter, whose Grok AI has run into repeated controversies, remains a Royal Society member after the body resisted widespread calls last year from other fellows to expel him over his role in defunding some areas of US research via the Department of Government Efficiency, which he is believed to have unofficially headed.

“We have all seen the concerns [about human extinction] raised by former employees of the large AI companies,” the open letter says, adding that the “rise in capabilities of these models in our domain of expertise persuades us that such statements must not be dismissed as ‘hype’.

“By the time the situation becomes obvious to the wider public, it may be too late to act. We believe this is an emergency and call on the Royal Society to use its influence to convey this view to government and the media.”

frances.jones@timeshighereducation.com