What should a board do about the AI safety warnings?

A board does not need a view on whether AI will end the human race. It needs to be able to show, in writing, that the company is in control of the AI it has already deployed. September 2026 has produced resignations, extinction odds from 5 to 70 per cent, an essay from Anthropic’s chief executive calling for a slower frontier, bills in Washington and Westminster, and confident dismissals. None of that can be adjudicated from a boardroom. What can be decided there is which decisions machines may make in the company’s name, who can stop each system, what people must remain able to do unaided, how the board would find out if something went wrong, and what management’s assurances rest on. Those five hold whatever the odds turn out to be.

What has been said, and by whom#

Jacob Coxon resigned from Anthropic on 8 September and posted that “Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.” TIME reported over 90 million views in 24 hours, and that Evan Hubinger, Anthropic’s head of alignment stress testing, posted: “Jacob is correct here, we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade”, adding that Anthropic “do not yet have a plan to solve alignment for superintelligence”. Dario Amodei’s essay We Must Pace the Frontier said: “We must slow the pace at which we improve the capabilities of AI models.” Sam Altman told Axios: “I agree with Dario that we need to pace the frontier”.

Jensen Huang of Nvidia called extinction claims “complete nonsense”, the BBC reported. David Bellamy of the Institute of Foundation Models told TIME: “AI killing us all by creating dangerous viruses is total bogus”. In Washington, Senator Sanders and Representative Casar introduced a bill on 3 September to ban artificial superintelligence and pause frontier development, with criminal penalties. In the UK, AI minister Kanishka Narayan rejected a Lords emergency shutdown amendment on 11 September; binding rules remain “on the table” if voluntary testing proves insufficient, Crypto Briefing reported. First Secretary Louise Haigh told the TUC of “huge risks to our national security and society if the right guardrails are not put in place”; Business Secretary Jonathan Reynolds warned against complacency and hyperbole alike, according to The British Eye. The UK has no dedicated AI statute; Alex Sobel’s Private Member’s Bill on superintelligence has its second reading on 13 November, Lewis Silkin noted.

Why the board should not try to adjudicate the odds#

The figures range from Hubinger’s more than 10 per cent to Geoffrey Irving’s 50 per cent and Marcus Williams’s 70 per cent, as TIME reported on 15 September. Grace et al. 2024, a survey of 2,778 AI researchers cited on the AGI page, found a median of 5 per cent. Those giving the numbers are researchers rather than forecasters, and most say so. The International AI Safety Report 2026 calls this the evidence dilemma: capability moves fast and evidence about new risks arrives slowly, so acting early may entrench the wrong intervention and waiting may leave people exposed. A board that spends its September meeting deciding whether 5 or 50 is right is doing work it cannot do well and that changes none of its obligations. Whether the AI systems the company already runs are under its control today is answerable.

The five decisions that hold whatever the odds#

First, which decisions machines may make in the company’s name: the outcomes a system may determine without a person deciding: a credit limit, a refund, a hiring shortlist, a public reply. If the board cannot produce that list, the systems have made the decision by default. This is the ground covered by Rules Before Tools and by what board oversight of AI looks like.

Second, who can stop each system, and whether they would. Every debate about a national AI kill switch, including the Lieu-Moran House bill reported by the Wall Street Journal, runs into the same questions: which official, what threshold, and how to switch off something running across several jurisdictions’ cloud infrastructure. Inside a company, a stop often means a vendor change request and a week. Being able to stop a system is a technical fact. Being willing to stop it, mid-quarter, with revenue attached, is a leadership fact, and should be rehearsed before it is needed.

Third, what people must remain able to do unaided. Budzyn et al. 2025, in the Lancet Gastroenterology and Hepatology, found that 19 endoscopists averaging 27.6 years of experience saw unassisted adenoma detection fall from 28.4 to 22.4 per cent within months of routine AI use. That is capability debt: skill the organisation would need if the system stopped, lost without anyone deciding to lose it. The board should know which tasks the company could no longer do by hand, and whether that was chosen.

Fourth, how the board would find out if something went wrong. That requires an incident definition before there is an incident; what counts as a serious AI incident sets one out. Without it, the board hears first from a journalist or a regulator. The agent incidents of July and August were described by the companies involved as tests with guardrails disabled and monitoring not enabled. A company with no incident definition has made the same choice.

Fifth, what management’s claims rest on. When management says a system is safe, accurate or overseen, the board should ask what evidence supports that, and expect more than a vendor slide. A sampling regime, a reviewer disagreement rate, a log of overrides, a rehearsed stop: any of these is evidence. An assurance that oversight exists is a claim. The difference between governance and leadership is that governance files the assurance and leadership asks what it rests on.

Five questions to ask management this month#

One: list every decision a machine makes in this company’s name without a person deciding, and who approved each. Two: for each system on that list, who can stop it, how long a stop takes, and when it was last rehearsed. Three: which tasks could we no longer do by hand if the system were withdrawn, and did we decide that or discover it? Four: what is our definition of an AI incident, how many have we had this year, and how did the board hear? Five: for each assurance in the last board pack about AI accuracy or safety, what was the evidence and who produced it?

What “pace the frontier” changes for a company#

Less than the headlines suggest. The essay says “Progress will still seem fast, and we must make wise use of the time we gain.” Amodei’s three steps, embedded third-party evaluators, common safety standards with US government mediation, and international coordination including China, are addressed to frontier developers and governments. The Register called the plan regulatory capture; Speaker Mike Johnson said it could “smother innovation”. A slower frontier does not restore a decayed skill, define an incident nobody defined, or tell a board who can stop a system. Whether to pause AI development is a question for governments and laboratories. The handover decisions belong to the board, and were made, by choice or default, some time ago.

What this does not show#

This page does not show that the extinction warnings are wrong, or right. The odds quoted are individual estimates; the one systematic survey cited, Grace et al. 2024, found no consensus on pace. The Sanders-Casar, Lieu-Moran and Sobel bills had not passed at the time of review; UK ministers say existing powers suffice. Nothing here shows that a company taking the five decisions above would be protected from a frontier-scale failure. What it shows is that the questions a board can answer are the ones about its own systems, and those answers are available now.