United Nations rights chief Volker Turk on Monday called on governments and leading artificial intelligence companies to take “urgent action” to rein in increasingly powerful AI systems, warning that the technology could create unprecedented risks for humanity.

In a letter sent to governments and AI developers and shared with the media, Turk said the world was approaching a potentially irreversible transformation driven by the rapid development of advanced AI.

He warned that the consequences would extend beyond the present generation, saying humanity was “on the cusp of irreversible change, affecting not just us but generations of humanity to come.”

UN rights chief warns of AI’s 'unprecedented risks'

Turk said the accelerating competition to develop increasingly powerful AI represented a major escalation in the risks posed by the technology.

“The current race towards ever more powerful AI is a step change towards greater existential risks to every aspect of our lives,” he said.

His warning comes as governments, technology companies and researchers face growing questions over how advanced AI systems should be developed and controlled.

Turk argued that the issue cannot be left solely to individual technology companies, particularly as a small number of firms control much of the development of the most advanced systems.

“In today's world, we are almost entirely reliant on a small number of powerful companies to make the right choices on Al development for all of humanity,” Turk said.

His comments place human rights and international governance at the center of a debate that has increasingly focused on the ability of governments and companies to control increasingly autonomous AI systems.

Read more: AI hits UK graduate jobs as coding roles decline

Frontier AI race raises human-control concerns

Turk's intervention comes amid renewed concern over so-called superintelligent AI systems and whether sufficiently advanced models could eventually operate beyond meaningful human control.

The debate intensified after a researcher publicly resigned from Anthropic, one of the leading AI companies, over concerns that future AI systems could escape human control.

Another Anthropic employee who remained at the company also issued a stark warning, saying, “we really do earnestly believe AI could kill all humans.”

The employee estimated that the probability of such an outcome was greater than “10 percent within the next decade.”

While such assessments remain contested, they illustrate the growing divide over how quickly frontier AI should advance and how much risk developers should accept in pursuing increasingly capable systems.

AI companies face calls to slow development

The warnings have also prompted calls from within the technology industry for greater coordination over the pace of AI development.

Anthropic CEO Dario Amodei called Saturday for AI companies to “pace the frontier,” advocating coordination among leading developers to slow the development of increasingly powerful systems.

His position received public support from competitors including xAI CEO Elon Musk and OpenAI CEO Sam Altman.

The backing reflects growing recognition among some leading AI executives that competition between companies could create incentives to develop increasingly capable systems faster than safety mechanisms and regulatory frameworks can keep pace.

However, the discussion remains largely centered on voluntary action by companies, highlighting the governance gap facing governments seeking to regulate rapidly evolving technology.

UN urges international rules for advanced AI

Turk called on governments and companies to address the issue through multilateral institutions rather than leaving decisions over frontier AI exclusively to private firms.

“I call on States and companies to mobilise urgently through multilateral fora to chart a path for action to govern advanced, frontier AI models, grounded in international human rights law,” he said.

The call places AI development within a broader international rights framework, emphasizing the potential consequences of advanced systems for individuals and societies.

For the UN, the challenge extends beyond technical safety. Increasingly powerful AI can affect privacy, employment, access to information, discrimination and other fundamental rights, making international oversight a broader governance issue.

Read more: China to lead BRICS open-source AI push as Xi calls for global rules