AI workers with apocalypse fears are facing a troubling, though not unprecedented, career dilemma.
Should they quit their jobs to avoid potentially helping make the doomsday scenario they envision a reality? Or, should they drive change for the better from within?
It’s a thorny predicament that came into focus this week when Jacob Coxon, a former OpenAI employee, said in an X post that he was resigning from Anthropic because the “people building AI earnestly believe that it could kill us all by the end of the decade.”
Soon after, Evan Hubinger, who leads Anthropic’s alignment stress-testing team, essentially backed Coxon, saying he believes there is a greater-than-10% chance AI could kill all humans within the next decade.
Why some workers stay
For AI workers worried that they’re contributing to mankind’s destruction, deciding whether to quit or stay put is more complicated than it may seem, said Jeremie Harris, CEO and cofounder of Gladstone AI, a Washington, DC-based safety and advisory company focused on national security risks and catastrophic threats from advanced AI. “I don’t particularly fault anyone for making either call,” he said.
Harris said his job involves regularly talking with staff at Anthropic and other top AI labs, and that in recent months he’s heard many express fears similar to those Coxon described. While quitting can be a moral response to building a dangerous technology, he said, Al labs could end up with few or no workers willing to press for change if resignations pile up.
“The people who are left are going to be the ones who are least concerned about safety,” Harris said, while “people who care arguably should be the ones in the seat.”
In response to Coxon’s announcement, Anthropic employee Anna Wang said on X that she works at the company because she thinks she “can do better at reducing risks from the inside.” She added that she respects and endorses others “who think that it’s better to do so from the outside.”
A spokesperson for Anthropic declined to comment after confirming Wang’s employment.
Coxon’s X post, which also sparked online debate over its impact on Anthropic’s image, the company’s coming IPO, and the possibility that this is all a strange publicity stunt, echoes concerns of a pending AI calamity previously made by Geoffrey Hinton, the computer scientist widely known as the “Godfather of AI,” xAI founder Elon Musk, and others.
The general public is apprehensive about AI, too. Some 52% of Americans are more concerned than excited about the use of the technology in daily life, according to a June study from the Pew Research Center. That’s up from 37% who said the same in 2021.
A dilemma with a history
For AI workers who truly believe their employer is creating an existential threat, Alex Wellerstein, a professor at Stevens Institute of Technology, said internal voices of reason have a poor track record of “overcoming preexisting institutional momentum.”
“If you stick around, you’re participating,” he said.
That tension is not new. Technology historian Matthew H. Hersch said the Manhattan Project, a World War II-era program to develop nuclear weapons, is a close parallel.
Ralph Austin Bard, then undersecretary of the US Navy, privately urged officials to warn Japan before using the atomic bomb and resigned weeks before Hiroshima, likely in protest of the decision not to do so, said Hersch, also a professor at Harvard University and a member of the School of Historical Studies at the Institute for Advanced Study.
Other Manhattan Project scientists also unsuccessfully pushed for a demonstration of the bomb before it was used in combat, some of whom were effectively barred from future defense work, he added.
“Whistleblowers can do tremendous good,” Hersch told Business Insider, “but are apt to be discredited or harassed into silence, and become unemployable in their industry.”
