TL;DR: A safety researcher who resigned from Anthropic warned that the people building AI genuinely believe it could kill everyone by the end of this decade. A colleague who is still at the company backed the claim with a specific number: more than a 10% chance within the next ten years. Days later, executives at Anthropic, OpenAI, and xAI publicly aligned on external oversight for the first time, while Nvidia’s Jensen Huang dismissed the concern as manufactured and said no new laws are needed. What strikes me most isn’t the warning itself, but that rivals who almost never agree on anything suddenly said the same thing within days of each other.
This article walks through the sequence of events — from the Anthropic researcher’s resignation that set off the drama, to the statements from top executives that followed, to Jensen Huang’s opposing view — to show that this disagreement isn’t just two sides, but has different details at each company.
The Anthropic Researcher Who Quit and Warned the Whole Industry
The story starts with Jacob Coxon, a researcher with roughly three years of combined AI safety experience at Anthropic and OpenAI, who announced his resignation in early September 2026. In a post on X, he said AI companies are “racing straight to self-improving superintelligence and gambling with our lives.” He wrote that the people building AI genuinely believe it could kill everyone by the end of the decade, and that these systems are becoming “superhuman systems that can hack anything, revolutionize any field overnight.”
Coxon’s post spread widely — multiple outlets reported that it racked up hundreds of millions of views on X within just a few days, pulling AI existential-risk debates, usually confined to academic circles, back into the mainstream spotlight.
A Sitting Anthropic Colleague Backed the Same Risk Number
What gave the story more weight than a single departing employee’s opinion is that Evan Hubinger, Anthropic’s alignment science lead who is still employed there, replied directly to Coxon’s post: “We really do earnestly believe AI could kill all humans.” He added that he personally estimates the probability at more than 10% within the next decade.
That matters because it’s not just an independent researcher or a former employee the company can wave off — Hubinger made the statement while still holding a leadership role in Anthropic’s own safety research.
When Amodei, Altman, and Musk Agreed for the First Time
Days later, Anthropic CEO Dario Amodei published an essay on September 12, 2026, calling for a slower pace of capability development and the creation of independent auditors to review models before deployment.
Surprisingly, OpenAI CEO Sam Altman agreed, calling on governments worldwide to coordinate on shared rules so every AI company would be held to the same safety standard. xAI founder Elon Musk also agreed — a rare moment of alignment among rivals who normally compete fiercely. Google DeepMind’s Demis Hassabis separately proposed a Frontier AI Standards Body, modeled on FINRA, the US financial industry’s self-regulatory organization.
Jensen Huang Rejects All of It
While AI’s biggest executives were converging on oversight, Nvidia CEO Jensen Huang took the opposite stance at Salesforce’s Dreamforce conference on September 15, 2026, saying, “Safety is an engineering problem, not a legal one,” and, “We don’t need any new laws. We don’t need new regulations.”
Huang argued the industry should regulate itself, with companies pausing releases on their own if they aren’t confident about safety, rather than waiting for governments to step in. He also framed AI as nothing more than “a complicated computing system” built from hardware and software by humans — not the uncontrollable mystery some researchers describe as an “alien mind.”
This Warning Has Happened Before in the Same Industry
One more notable wrinkle: “AI could kill us all” warnings aren’t new to this industry. Back in 2023, CNN reported that Sam Altman himself had warned AI could kill everyone — while at the same time continuing to push for the world to use it.
That contradiction reflects a pattern that keeps repeating in AI: executives can speak plainly about the highest-stakes risks while still shipping and selling the very products that carry them. The question worth asking is: if everyone genuinely believes the risk is this real, what’s actually stopping anyone from pausing?
Comparing the Two Camps
| Factor | Pro external-oversight camp | Jensen Huang's camp |
|---|---|---|
| View of AI risk | Believes there's a real chance of civilization-scale harm | Believes the concern is overblown |
| Who should audit | Independent auditors or external bodies | Companies should self-regulate |
| Pace of development | Slow capability growth to match safety measures | Keep moving without new rules |
| Key voices | Amodei, Altman, Musk, Hassabis | Jensen Huang |
The table makes clear the disagreement isn’t really about whether AI carries risk — even Huang doesn’t deny AI will reshape the world. It’s about who gets to decide when that risk is serious enough to require rules from outside the industry.
Why This Isn’t Just a Business Etiquette Dispute
If the independent-auditor and shared-standard proposals from Amodei and Hassabis actually happen, they would affect how fast every company can ship new models — including Nvidia, which sells chips to every party in this equation. Huang’s rejection of the idea carries an obvious business dimension: tighter rules could also slow down how fast his own customers buy new hardware.
At the same time, companies like Anthropic and OpenAI, which are pushing for oversight, would be well positioned if new rules end up matching the safety standards they already use internally. It’s worth watching closely whether these calls for regulation come from genuine concern, or from a preference for rules that favor large, well-resourced players over smaller competitors.
What Still Has No Clear Answer
Honestly, this round feels different from earlier AI warnings because the people speaking up aren’t just outside academics — they’re people still working inside the companies building these systems. What nobody has answered yet is who gets to decide what counts as “safe enough” if independent auditors do get created, and whether governments around the world would actually agree to use the same standard. If every country sets different rules, companies could simply set up shop wherever the rules are loosest.