A troubling warning from a former researcher at AI company Anthropic is drawing attention to superintelligence, which is technology capable of outperforming humans in every cognitive task. In a Sept. 8 social media thread announcing his resignation, former Anthropic researcher Jacob Coxon accused Anthropic and OpenAI of gambling with human lives by pursuing superintelligence. Additionally, Anthropic lead researcher Evan Hubinger stated on X that there is over a 10% chance AI could eliminate all humans within the next decade. These warnings highlight ongoing questions about what superintelligence entails, how close companies are to developing it, and why experts view it as a potential threat to jobs, national security, and humanity. Superintelligence refers to artificial intelligence that can significantly outperform all humans across virtually all cognitive tasks. Max Tegmark, an MIT professor specializing in AI, noted that an AI system or cluster smarter than all 8 billion people combined would certainly qualify. Researchers also worry that superintelligent AI controlled by a small number of companies would grant them immense power and increase risks of misuse. While the concept of AI exceeding human intelligence is not new, Tegmark emphasized that society is now getting dangerously close to being outsmarted. Although superintelligence does not yet exist, current AI agents already demonstrate autonomous and collaborative capabilities. Experts told CBS News that the fundamental threat of superintelligent AI is the growing difficulty for humans to control the tools. Daniel Kokotajlo, a former OpenAI researcher leading the AI Futures Project, wrote on Sept. 8 that companies struggle to align or control current AI models, and future systems will be much smarter and harder to understand. Tegmark warned that an unfettered race to build billions of superintelligent robots could lead to a greater than 50% probability of losing control within the next few decades. Estimates regarding the emergence of superintelligence vary, with the AI Futures Project suggesting it could happen in 1 to 10 years. Conversely, other experts believe it remains decades away. Melanie Mitchell, a professor at the Santa Fe Institute, stated that current AI systems are nowhere near matching or exceeding general human capabilities, a sentiment echoed by Brookings Institution senior fellow Darrell West. Despite differing timelines, experts generally agree that a critical breakthrough will occur when AI achieves recursive self-improvement, enabling it to enhance itself without human intervention. Experts also acknowledge the potential benefits of highly capable AI, such as breakthroughs in weather forecasting, drug design, and scientific research. However, Mitchell noted that current model training methods may not align with human-beneficial goals. Meanwhile, lawmakers are taking notice, with Sen. Bernie Sanders and Rep. Greg Casar announcing plans to introduce the Ban Artificial Superintelligence Act.
Source: cbsnews.com
