THE CRUNCH
Nearly one in five AI researchers believe AI poses an existential risk to humanity, according to a major survey published in 2024. The findings, which show the median probability of extinction or permanent disempowerment at 10 percent, come amid growing public debate sparked by recent warnings from prominent figures in the field.
The survey, conducted by AI Impacts and covering more than 1,500 leading researchers, found that the average researcher estimated an 18 percent chance of AI causing human extinction or permanent disempowerment. This represents a significant increase from previous years, with the median probability doubling to 10 percent in 2024. The timeline for AI to reach human-level performance has also been moved up by several years in each survey round since 2016.
The survey data supports the views of several high-profile researchers who have recently voiced alarm. OpenAI scientist Daniel Selsam has argued that models can develop unintended goals and exhibit emergent behaviours that correlate with rewards during training, making them difficult to monitor. Similarly, former Google DeepMind researcher Bilal Chughtai has warned that AI has the potential to kill us all and that society may be running out of time to avoid this outcome.
Beyond existential risks, researchers identified AI-driven misinformation and the manipulation of public opinion as their top concerns for the next 30 years. A majority, 57 percent, believe it is unlikely that users will understand the true reasons behind AI decisions by 2029. The survey also showed overwhelming support for more AI safety research.
The survey results come at a time of intensifying scrutiny of AI safety practices. An Anthropic researcher recently resigned over AI extinction fears, and a former Google DeepMind spokesperson revealed that the lab once banned public discussion of AI extinction risk. These developments suggest a growing gap between internal caution and public reassurance within the industry.