Should AI development be paused or slowed?

When a studio audience or a select committee asks whether AI should be paused until the risks are understood, three different questions are being asked as one. Whether the frontier labs should stop training larger models: that was asked for in March 2023, in writing, by tens of thousands of people, and did not happen. Whether anyone knows enough to say when it would be safe to resume: the expert evidence says no, and says it will not soon. And whether an organisation or a person should pause before handing a decision to a machine: that one is in your gift, it is the only one with measurable evidence behind it, and almost nobody asks it.

The pause that was asked for#

On 22 March 2023 the Future of Life Institute published an open letter asking all AI labs to pause, for at least six months, the training of any system more powerful than GPT-4, and for governments to impose a moratorium if the labs would not. It gathered tens of thousands of signatures, among them Yoshua Bengio, Steve Wozniak and Elon Musk. No lab paused. Within the six months the letter asked for, more capable models had been released by several of the companies it was addressed to, and one signatory had founded a new lab. The letter's lasting effect was on the conversation: the words pause and extinction risk entered the mainstream, a UK safety summit followed in November 2023, and an international expert report was commissioned. The schedule did not move.

That is the first thing to hold on to. A pause on development is a decision that sits with perhaps five companies and two governments, and the one occasion on which it was asked for with maximum public force produced none. Anyone in a British boardroom or a radio studio arguing for or against it is arguing about a lever they do not hold.

What the experts agree on, which is less than either side claims#

The largest survey of the people building these systems is Grace and colleagues, published January 2024: 2,778 researchers who had published in the previous year at the main machine learning venues, a 15 per cent response rate. The number everybody quotes is the median 5 per cent chance given to future AI causing human extinction or similarly permanent disempowerment. The number fewer people quote is that the same population gave 10 per cent when the question was worded around an inability to control advanced AI, and that between 41 and 51 per cent gave more than a 10 per cent chance depending on how they were asked. The authors write that their respondents are experts in AI and not, to their knowledge, skilled forecasters, and cite work showing that framing alone has moved lay estimates of existential risk by nearly six orders of magnitude.

On pace, the survey found no agreement: asked whether faster or slower progress would be better for the future, the researchers split rather than converged. So neither "the experts want a pause" nor "the experts think it is safe" summarises the survey. The people closest to the work put a non-trivial number on catastrophe, disagree with each other about it, and disagree about whether slowing down would help.

Why "until we understand the risks" is not a threshold#

The 2026 International AI Safety Report, written by over a hundred experts on a panel nominated by more than thirty countries, makes no policy recommendation by design, but it names the problem that any pause proposal runs into. It calls it the evidence dilemma: the capability landscape changes fast and evidence about new risks emerges slowly, so a decision-maker who acts early risks entrenching the wrong intervention, and one who waits for evidence may leave people exposed while it arrives. Understanding is not a line that gets crossed once. It lags the capability, and the gap is structural.

The same report's finding on loss of control tends to get loosened in both directions, so here it is as written. Current systems show at most early signs; expert views on the future vary widely; and the disagreement is between experts rather than between experts and the public. Neither side can claim that as support. It describes a situation in which the confident positions on either side are running ahead of the evidence.

The risk that is measured, and the pause nobody proposes#

While the argument about pausing the technology goes on, the evidence that does exist points somewhere else: at what happens to the people once the machine is in the loop. The strongest single study is Budzyn and colleagues in The Lancet Gastroenterology and Hepatology, 2025: nineteen endoscopists averaging 27.6 years of experience, across four Polish centres. After a few months of routine AI-assisted colonoscopy, their adenoma detection rate when working without the AI fell from 28.4 to 22.4 per cent, six percentage points, in the most experienced clinicians in the building. It is observational, one procedure in one country, and detection is a proxy for skill rather than skill itself. It remains the clearest measured harm in the whole debate, and the model's power had nothing to do with it: the handover was made without anyone deciding what the humans had to stay able to do.

That pattern repeats across the evidence base: capability debt accruing where work moved to a machine and the practice that built the judgement stopped, a human in the loop who cannot in practice intervene, and oversight that nobody counted as work. None of it requires a frontier model. All of it follows from a decision that was not made.

The pause that is in your gift#

So the useful answer to "should AI be paused" is: not the part you cannot pause, and yes to the part you can. The pause an organisation controls is the one between buying a tool and letting it decide. It takes a page, not a moratorium. Which decisions may the machine inform, which may it recommend, which may it make, and which stay with a person. Who can stop each one, and whether they would in practice. What the organisation's own people must remain able to do unaided, and how anyone would know if the machine went wrong. Written down, with names against it, before the next licence is signed. That is what Rules Before Tools sets out and what meaningful human oversight means when it is more than a phrase.

A pause of that kind has a test. Ask three questions of anything a machine helped decide: who verified it, against what, and who carries the consequences if it is wrong. If all three can be answered without hesitation, the handover was designed. If any one cannot, it drifted, and no moratorium in California would have prevented it.

September 2026: the labs asked to be slowed#

The question moved in the week this page was written. On 3 September Senator Bernie Sanders and Representative Greg Casar introduced a bill to ban artificial superintelligence and pause frontier development. On 8 September Jacob Coxon resigned from Anthropic saying the companies were “racing straight to self-improving superintelligence”. On 12 September Dario Amodei, Anthropic’s chief executive, published We Must Pace the Frontier: “We must slow the pace at which we improve the capabilities of AI models”, with the qualification that “progress will still seem fast”, and a three-step plan of embedded evaluators, common standards among developers and international coordination. Sam Altman agreed, Axios reported. In the UK, Alex Sobel’s Private Member’s Bill on superintelligence has its second reading on 13 November. None of this changes the analysis above. A call to slow from inside the industry is new; a mechanism that binds the callers is not yet on the table, the plan’s later steps depend on parties who have not agreed to them, and every proposal still runs into the evidence dilemma. It does change who is asking. When the people with the lever say it should be pulled more slowly, the organisational pause described below stops being a consolation and becomes the part of the answer that is already available.

16 September 2026: the European Commission joins the pacing call#

Four days after the Amodei essay, the call reached a government with a lever. In her State of the European Union address in Strasbourg on 16 September, Ursula von der Leyen said that “CEOs of the most advanced companies tell us that it is time to slow down on the self-recursive models. To pace the frontier”, and that she would “invite the main frontier labs for a discussion on how we can support ongoing industry efforts to pace the frontier”, Reuters reported. She described self-improving models as systems able to build more capable versions of themselves and called the AI Act the guardrail already in place, Help Net Security reported; no date for the meeting was given. The same day OpenAI published a framework for disclosing its own models’ misalignment, and wrote in it: “We do not believe that the AI industry has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer.” In the UK, Louise Haigh told the TUC Congress that AI poses huge risks without proper protections and that government should heed the warnings of those building it, AI Insider reported, while Lewis Silkin recorded on 14 September that the government had rejected a statutory kill switch on the ground that it would need international coordination to work.

What has changed since the section above is that two of the largest developers have now said in their own words that the current pace is not safe to sustain, and the executive of the world’s largest regulated market has said it will support their slowing rather than order it. What has not changed is the shape of the problem. Support for a voluntary pace is not a mechanism that binds anyone, the labs that have not spoken are not covered by it, and the evidence dilemma applies to the Commission as much as to anyone else. For an organisation, the position is as it was, with one addition: when both the builders and the regulator say the frontier should move more slowly, the pause before the handover is no longer a consolation prize but the only part of the answer that anyone in the room can act on this quarter.

What this does not show#

It does not show that a development pause would be harmless, or harmful; the estate holds no evidence either way and the largest expert survey found none. It does not show that the extinction estimates are right, only that they exist, that they move with the wording, and that the people giving them say so. It does not show that the endoscopy result generalises beyond one procedure, though the mechanism it demonstrates is the one this whole site is about. And it does not show that the handover pause is sufficient: it is the part of the risk an organisation can measure and own, which is a different claim from the whole of the risk.