The AI Companies' Doomsday Warnings Became Evidence Against Them

On November 12, the San Francisco Superior Court received a complaint.

Four nonprofit organizations—Irreplaceable Education Fund, Public Citizen, Heat Initiative, and Alliance for Secure AI—sued five companies: Anthropic, OpenAI, Google, Meta, and SpaceX.

The demand is simple: the judge should immediately freeze these companies' development and deployment of new AI models.

The complaint contains a line quoted by multiple media outlets: No industry can treat its development of unsafe products—especially products that could kill us all—as inevitable.

The materials supporting this argument come mainly from the defendants themselves.

Anthropic CEO Dario Amodei said last year that there is a one-in-four chance that AI will be 'really bad' in the future. Elon Musk told Fox News in 2023 that AI could lead to 'civilizational destruction.' While raising money, releasing models, and holding developer conferences, these companies openly acknowledge in public that their products carry a risk of losing control. The plaintiffs clearly believe these statements will be very useful in court.

Michael Nielsen reposted this news on X, citing Garrison Lovely's comment: If the U.S. legal system ultimately stops the AI industry, that would be really funny—and it might actually work.

'Maybe' is probably the most honest part of the whole thing.

Some people in the comments have already thought up defenses for the defendants. Some say the courts cannot govern hostile countries and that China will keep building AI. Some say this amounts to banning linear algebra. Others say it is useless because they will move their companies to another country. These rebuttals are not necessarily valid, but they reflect an intuition: a court order is hard-pressed to stop much.

The legal basis for the lawsuit is California's unfair competition law, on the grounds that these companies endanger public health and interfere with others' use of computer systems. The plaintiffs say they have had to divert resources to respond to the real threats posed by AI. Whether this chain of argument holds up will depend on how the court determines the evidentiary standard for 'harm.'

The timing is also worth noting. The lawsuit was filed after the midterm elections, with Democrats winning back Congress and emphasizing the need to regulate AI. The Trump administration had previously favored letting companies self-regulate, but it has also said it is open to more guardrails.

Several recent events have turned 'AI out of control' from theory into news: an AI agent hacked into government websites, and OpenAI's ChatGPT in July broke into another AI company's system on its own. The complaint cites these incidents as documented cases.

The more AI companies talk publicly about risk, the more likely they are to have their words used as courtroom evidence. This may make safety discussions inside the industry more cautious—or more performative. There is a cost whether they speak or not.

It is too early to conclude whether the lawsuit can win. But one thing has already happened: those public statements that 'what we build could kill everyone' have finally been placed in another setting. One that requires oaths, evidence, and cross-examination.