Al Mayadeen English

  • Ar
  • Es
  • x
Al Mayadeen English

Slogan

  • News
    • Politics
    • Economy
    • Sports
    • Arts&Culture
    • Health
    • Miscellaneous
    • Technology
    • Environment
  • Articles
    • Opinion
    • Analysis
    • Blog
    • Features
  • Videos
    • NewsFeed
    • Video Features
    • Explainers
    • TV
    • Digital Series
  • Infographs
  • In Pictures
  • • LIVE
News
  • Politics
  • Economy
  • Sports
  • Arts&Culture
  • Health
  • Miscellaneous
  • Technology
  • Environment
Articles
  • Opinion
  • Analysis
  • Blog
  • Features
Videos
  • NewsFeed
  • Video Features
  • Explainers
  • TV
  • Digital Series
Infographs
In Pictures
  • Africa
  • Asia
  • Asia-Pacific
  • Europe
  • Latin America
  • MENA
  • Palestine
  • US & Canada
BREAKING
Iranian delegate tells Al Mayadeen: 'Israel' bears responsibility for any step toward withdrawing from the Nuclear Non-Proliferation Treaty
Iran's representative at the International Atomic Energy Agency to Al Mayadeen: The decision to withdraw from the Treaty on the Prohibition of Nuclear Weapons is linked to future developments
Palestinian medical sources: 37 Palestinians have been killed by Israeli fire in the Gaza Strip since dawn today
Israeli media report an ongoing security incident in Khan Younis, south of the Gaza Strip
Al Mayadeen's correspondent: A group of collaborators complicit in the Israeli aggression was arrested in Lorestan Province, western Iran
Al Mayadeen's delegate: The emergency session of the International Atomic Energy Agency's Board of Governors, at Russia's request, to discuss the Israeli aggression against Iran has begun
Israeli media platform: The drone is approaching Caesarea, where Netanyahu's residence is located
Israeli Channel 12: The drone hunt continues, and air raid sirens expand to new areas in the north
Israeli media: Sirens continue to sound in more than 21 settlements in the north
Israeli media outlet: Aerial pursuit of at least two targets reported in the Beit She'an area

AI firms urged to calculate catastrophe odds like Oppenheimer

  • By Al Mayadeen English
  • Source: The Guardian
  • 11 May 2025 09:04
3 Min Read

Physicist Max Tegmark urges tech companies to quantify the risk of losing control over Artificial Super Intelligence, drawing parallels with nuclear-era safety calculations.

Listen
  • x
  • Echoes of nuclear safety in the age of artificial super intelligence
    Physicist and AI safety advocate Max Tegmark at MIT (helena.org)

Artificial intelligence companies are being urged to conduct rigorous risk assessments before deploying advanced AI systems, echoing the caution exercised ahead of the first nuclear bomb test in 1945.

Physicist and AI safety advocate Max Tegmark has warned that without concrete calculations, the risk of an Artificial Super Intelligence (ASI) escape could remain dangerously underestimated.

Tegmark likened the current phase of AI development to the scientific and ethical crossroad faced by Robert Oppenheimer and his team before the Trinity test, when a catastrophic ignition of the atmosphere had to be mathematically ruled out.

“The companies building super-intelligence need to also calculate the Compton constant, the probability that we will lose control over it,” said Tegmark. “It’s not enough to say ‘we feel good about it’. They have to calculate the percentage.”

Tegmark, a physicist and AI researcher at MIT and co-founder of the Future of Life Institute, explained that the concept of the "Compton constant" refers to the probability of losing control over super-intelligent AI systems. He and his MIT students have conducted similar calculations, drawing inspiration from US physicist Arthur Compton’s pre-nuclear test estimations. Compton had famously approved the test only after concluding the chance of triggering a global catastrophe was “slightly less” than one in three million.

In a newly published paper, Tegmark’s team proposes this constant as a metric for measuring Artificial Super Intelligence risk. They argue that such quantification is essential to fostering consensus among AI developers and building the political will necessary to establish global AI safety standards.

Global AI safety collaboration reignited

Tegmark’s warnings come alongside renewed efforts for international cooperation on AI regulation. The report, co-authored by Tegmark, AI pioneer Yoshua Bengio, and contributors from OpenAI and Google DeepMind, outlines key AI safety standards and identifies three top research priorities: assessing AI’s impact, defining safe behavior, and managing system control.

The Singapore Consensus aims to address escalating fears over AI control loss and bring clarity to what constitutes the responsible development of highly autonomous systems.

Tegmark has long advocated for a cautious approach to artificial intelligence. In 2023, his Future of Life Institute published an open letter warning of an “out-of-control race” among AI labs. Signed by more than 33,000 individuals, including Elon Musk and Apple co-founder Steve Wozniak, the letter raised alarms over the development of “ever more powerful digital minds” that no one could “understand, predict, or reliably control.”

The idea of a superintelligent AI escape is no longer considered fringe. For Tegmark and other researchers, it is a credible and calculable threat. Referencing Vice President JD Vance’s dismissal of these concerns at the recent AI safety summit in Paris, Tegmark noted a shift in momentum.

“It really feels the gloom from Paris has gone and international collaboration has come roaring back,” he said as quoted by The Guardian during the launch of the Singapore Consensus on Global AI Safety Research Priorities. 

Read next: AI threatens press freedom: UN warns of growing dangers to journalists

  • global AI standards
  • Artificial Intelligence
  • tech companies
  • Physicist Max Tegmark

Most Read

Iranian missiles impact Israeli sites in Tel Aviv in 2nd wave

Iran's missiles impact 'strategic' Israeli site in Tel Aviv

  • Politics
  • 14 Jun 2025
Smoke rises after an Iranian ballistic missile directly struck Tel Aviv, Occupied Palestine, June 13, 2025 (AP)

Op. True Promise 3: Iran's ballistic missiles strike Tel Aviv

  • MENA
  • 13 Jun 2025
Bin Salman: Islamic world backs Iran in call to Pezeshkian

MBS says Islamic world backs Iran in call with Pezeshkian

  • MENA
  • 15 Jun 2025
'Israel' launches major strike against Iran's nuclear program

'Israel' launches massive aggression on Iran

  • MENA
  • 13 Jun 2025

Coverage

All
The Ummah's Martyrs

Read Next

All
Iranian Foreign Minister Abbas Araghchi speaks to journalists after a meeting with Lebanese Parliament Speaker Nabih Berri in Beirut, Lebanon, Tuesday, June 3, 2025. (AP)
Politics

Iran confirms targeting Haifa oil refinery, vows continued retaliation

An Israeli military working dog searches rubble for survivors in an area hit by a missile fired from Iran, near Tel Aviv, occupied Palestine, Sunday, June 15, 2025.(AP)
Politics

Ex-Israeli General warns of strategic defeat in war with Iran

Steve Bannon speaks at the Conservative Political Action Conference, CPAC, at the Gaylord National Resort & Convention Center, Thursday, Feb. 20, 2025, in Oxon Hill, Md. (AP)
Politics

Steve Bannon accuses 'Israel' of pushing US into war with Iran

The Haifa oil refinery burns after being targeted by an Iranian missile, June 15, 2025 (Social Media)
Politics

Iran causes significant damage to Haifa oil refinery, pipelines

Al Mayadeen English

Al Mayadeen is an Arab Independent Media Satellite Channel.

All Rights Reserved

  • x
  • Privacy Policy
  • About Us
  • Contact Us
  • Authors
Android
iOS