Jacob Coxon, a researcher at Anthropic, has resigned over concerns that self-improving artificial intelligence could lead to catastrophic outcomes. Coxon described the situation as 'gambling with our lives' and argued that the pace of development is too fast for safety measures to keep up. The researcher called for 'pacing agreements' between AI labs to ensure that safety research progresses at a rate comparable to capability research.

In a separate report, Ars Technica quotes Coxon as saying, 'We really do earnestly believe AI could kill all humans!' This stark warning underscores the intensity of the internal debate within the AI safety community regarding the potential for advanced models to become uncontrollable or hostile.