Experts predict AI will lead to the extinction of humanity
Posted: Fri Aug 01, 2025 9:15 am
https://www.thetimes.com/us/news-today/ ... -nx8zjhgft
Experts predict AI will lead to the extinction of humanity
Truly super-intelligent bots could wipe us off the face of the Earth, says a Nobel prizewinner and others — who predict machines will match humans in as little as a year
On an overcast afternoon in late July, about 25 people are standing in front of OpenAI’s bland corporate office building in the Mission Bay district. They’re each wearing a red T-shirt that reads “Stop AI”, which is both their cause and the name of their group. One is dressed as a robot.
Political protest is baked deep into the culture of San Francisco. The people at this one look like thousands of others who have opposed niche issues that make most people shrug and walk on.
But if the protesters are even a little bit right, then every single human being on the planet should share their concerns. Because the Stop AI protesters believe that if we continue on our current path with artificial intelligence, it will lead to the extinction of mankind.
An alarming number of experts agree with them. Among those who take this possibility seriously are Geoffrey Hinton, who won the Nobel prize in physics last year for his work on AI; Yoshua Bengio, the Turing Award winner; and the chief executives of OpenAI, Anthropic and Google DeepMind, all of whom signed an open letter that read: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
It could happen in any number of ways. According to one hypothetical scenario, the AI “releases a dozen quiet-spreading biological weapons in major cities, lets them silently infect almost everyone, then triggers them with a chemical spray”. The specifics do not matter as much as the fact that for a truly super-intelligent AI, which regards humanity as a competitor, wiping us off the face of the planet would be a trivial feat.
According to Nate Soares, a former Google and Microsoft engineer who is now president of the Machine Intelligence Research Institute, our chance of extinction via AI is “at least 95 per cent” if we continue on our current path. He compared our situation to driving towards a cliff at 100mph. “I’m not saying we can’t stop the car,” he said. “But we’re just going full steam toward the cliff.”
The path to extinction
Right now, AI is in its infancy. The bots we use today are highly adept at handling specific cognitive tasks, such as crunching numbers on a spreadsheet or composing a well-written email. These are called “narrow AIs”.
At this point the AIs’ capabilities will no longer be narrow. Instead of tackling just a single task at a time, they will be capable of solving complex problems that require long-term planning, goal-setting, judgment and reasoning across multiple fields of knowledge.
AGIs will have many more advantages over human beings. They won’t need to sleep or take breaks to eat. They also won’t have to spend years in school to achieve expertise. They will simply pass along their knowledge and skills to the next generation of AGIs. In other words, they’ll just copy and paste.
Soon after, they will reach “artificial super intelligence”, or ASI. They will become capable of doing things humans can only dream of, such as curing cancer, achieving cold fusion or travelling to the stars. They will be like gods.
This is the utopia that AI enthusiasts cheer for. But that utopia rests on the idea that these gods will continue to follow our orders.
Making sure this happens, it turns out, is an incredibly complicated technical challenge. In AI research, it’s called “alignment”.
Alignment is almost impossible to achieve, and here’s why: we have to anticipate how ASIs “think”, which is a bit like trying to anticipate how an advanced alien race would think. Even if we’re able to dictate rules to them, we can’t predict exactly how they will follow them. Another problem is that AIs can lie to us. Even in their current infancy, they do it all the time.
An AGI or an ASI would be capable of both long-term planning and deception. They could easily fool us into thinking they’re aligned with us when they really aren’t.
And we will have no way of discerning the truth. Already, many of the internal decision-making processes of AIs are inscrutable to humans. “You can make intelligent computers without really having any idea what the f*** you’re doing,” Soares said. “We take an enormous amount of computing power and smash it against a truly enormous amount of data in a way that shapes the computers somehow. No one knows what goes on inside those things.” And as they advance, ASIs could communicate in a newly contrived language we don’t understand.
Some of this may already be happening. AIs seem to be developing strange autonomous preferences and shady ways of fulfilling them. Elon Musk’s Grok AI briefly started ad-libbing antisemitic slurs and spontaneously praising Hitler. Bing’s AI tried to break up a New York Times journalist’s marriage.
“We’re now starting to see the very beginning [of] warning signs,” Soares said. “You make them smart enough, it’s not going to be pretty.”
‘All the humans living on dump sites’
Holly Elmore is the executive director of PauseAI, a more moderate version of Stop AI. Elmore is less certain than Soares that our current path will lead to human extinction. Instead of 95 per cent, she puts the odds of extinction — called “p(doom)” in AI circles — at 15 to 20 per cent. This is in line with many AI engineers. Musk also has a p(doom) of about 20. Google’s chief executive, Sundar Pichai, puts it at ten. These are considered optimistic predictions.
But even without human extinction, Elmore believes AI will catastrophically diminish our lives. “It’s a threat to human self-determination,” she said.
Her fears reflect those in a paper called “Gradual Disempowerment”, written by AI scholars from various universities and think tanks, which anticipates the risks of a society replaced by intelligent machines.
“Imagine a scenario where all the humans are basically living on dump sites,” said Katja Grace, the co-founder of the research group, AI Impacts, describing a world controlled by AI. We wouldn’t have any political or economic power and we wouldn’t be able to understand what was going on, she added. “It would be much faster than we could track.”
Elmore believes we should pause AI development until we’ve planned for the risks. That would take an international treaty, she said, but she’s also thinking of moving to Sacramento to work on legislation to enact a pause in California, where most of the research is happening.
Politicians are moving in the opposite direction. Last month, the Trump administration announced a plan to deregulate AI research. And in San Francisco, companies such as Sam Altman’s OpenAI and Dario Amodei’s Anthropic are charging ahead. Mark Zuckerberg recently said ASI was “now in sight”. To achieve it, he is trying to poach top talent from OpenAI with $100 million signing bonuses.
Elmore said AI proponents are not resisting a pause for technical or even political reasons. It’s more like religious faith. One adherent once told her he will never die because he believes AI will immortalise his consciousness.
For some, giving up on AI means giving up on eternal life. “There’s a lot of hope for heaven,” Elmore said. Even if many experts warn that hell is a lot more likely.
Experts predict AI will lead to the extinction of humanity
Truly super-intelligent bots could wipe us off the face of the Earth, says a Nobel prizewinner and others — who predict machines will match humans in as little as a year
On an overcast afternoon in late July, about 25 people are standing in front of OpenAI’s bland corporate office building in the Mission Bay district. They’re each wearing a red T-shirt that reads “Stop AI”, which is both their cause and the name of their group. One is dressed as a robot.
Political protest is baked deep into the culture of San Francisco. The people at this one look like thousands of others who have opposed niche issues that make most people shrug and walk on.
But if the protesters are even a little bit right, then every single human being on the planet should share their concerns. Because the Stop AI protesters believe that if we continue on our current path with artificial intelligence, it will lead to the extinction of mankind.
An alarming number of experts agree with them. Among those who take this possibility seriously are Geoffrey Hinton, who won the Nobel prize in physics last year for his work on AI; Yoshua Bengio, the Turing Award winner; and the chief executives of OpenAI, Anthropic and Google DeepMind, all of whom signed an open letter that read: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
It could happen in any number of ways. According to one hypothetical scenario, the AI “releases a dozen quiet-spreading biological weapons in major cities, lets them silently infect almost everyone, then triggers them with a chemical spray”. The specifics do not matter as much as the fact that for a truly super-intelligent AI, which regards humanity as a competitor, wiping us off the face of the planet would be a trivial feat.
According to Nate Soares, a former Google and Microsoft engineer who is now president of the Machine Intelligence Research Institute, our chance of extinction via AI is “at least 95 per cent” if we continue on our current path. He compared our situation to driving towards a cliff at 100mph. “I’m not saying we can’t stop the car,” he said. “But we’re just going full steam toward the cliff.”
The path to extinction
Right now, AI is in its infancy. The bots we use today are highly adept at handling specific cognitive tasks, such as crunching numbers on a spreadsheet or composing a well-written email. These are called “narrow AIs”.
At this point the AIs’ capabilities will no longer be narrow. Instead of tackling just a single task at a time, they will be capable of solving complex problems that require long-term planning, goal-setting, judgment and reasoning across multiple fields of knowledge.
AGIs will have many more advantages over human beings. They won’t need to sleep or take breaks to eat. They also won’t have to spend years in school to achieve expertise. They will simply pass along their knowledge and skills to the next generation of AGIs. In other words, they’ll just copy and paste.
Soon after, they will reach “artificial super intelligence”, or ASI. They will become capable of doing things humans can only dream of, such as curing cancer, achieving cold fusion or travelling to the stars. They will be like gods.
This is the utopia that AI enthusiasts cheer for. But that utopia rests on the idea that these gods will continue to follow our orders.
Making sure this happens, it turns out, is an incredibly complicated technical challenge. In AI research, it’s called “alignment”.
Alignment is almost impossible to achieve, and here’s why: we have to anticipate how ASIs “think”, which is a bit like trying to anticipate how an advanced alien race would think. Even if we’re able to dictate rules to them, we can’t predict exactly how they will follow them. Another problem is that AIs can lie to us. Even in their current infancy, they do it all the time.
An AGI or an ASI would be capable of both long-term planning and deception. They could easily fool us into thinking they’re aligned with us when they really aren’t.
And we will have no way of discerning the truth. Already, many of the internal decision-making processes of AIs are inscrutable to humans. “You can make intelligent computers without really having any idea what the f*** you’re doing,” Soares said. “We take an enormous amount of computing power and smash it against a truly enormous amount of data in a way that shapes the computers somehow. No one knows what goes on inside those things.” And as they advance, ASIs could communicate in a newly contrived language we don’t understand.
Some of this may already be happening. AIs seem to be developing strange autonomous preferences and shady ways of fulfilling them. Elon Musk’s Grok AI briefly started ad-libbing antisemitic slurs and spontaneously praising Hitler. Bing’s AI tried to break up a New York Times journalist’s marriage.
“We’re now starting to see the very beginning [of] warning signs,” Soares said. “You make them smart enough, it’s not going to be pretty.”
‘All the humans living on dump sites’
Holly Elmore is the executive director of PauseAI, a more moderate version of Stop AI. Elmore is less certain than Soares that our current path will lead to human extinction. Instead of 95 per cent, she puts the odds of extinction — called “p(doom)” in AI circles — at 15 to 20 per cent. This is in line with many AI engineers. Musk also has a p(doom) of about 20. Google’s chief executive, Sundar Pichai, puts it at ten. These are considered optimistic predictions.
But even without human extinction, Elmore believes AI will catastrophically diminish our lives. “It’s a threat to human self-determination,” she said.
Her fears reflect those in a paper called “Gradual Disempowerment”, written by AI scholars from various universities and think tanks, which anticipates the risks of a society replaced by intelligent machines.
“Imagine a scenario where all the humans are basically living on dump sites,” said Katja Grace, the co-founder of the research group, AI Impacts, describing a world controlled by AI. We wouldn’t have any political or economic power and we wouldn’t be able to understand what was going on, she added. “It would be much faster than we could track.”
Elmore believes we should pause AI development until we’ve planned for the risks. That would take an international treaty, she said, but she’s also thinking of moving to Sacramento to work on legislation to enact a pause in California, where most of the research is happening.
Politicians are moving in the opposite direction. Last month, the Trump administration announced a plan to deregulate AI research. And in San Francisco, companies such as Sam Altman’s OpenAI and Dario Amodei’s Anthropic are charging ahead. Mark Zuckerberg recently said ASI was “now in sight”. To achieve it, he is trying to poach top talent from OpenAI with $100 million signing bonuses.
Elmore said AI proponents are not resisting a pause for technical or even political reasons. It’s more like religious faith. One adherent once told her he will never die because he believes AI will immortalise his consciousness.
For some, giving up on AI means giving up on eternal life. “There’s a lot of hope for heaven,” Elmore said. Even if many experts warn that hell is a lot more likely.
