What is an AI kill switch, and would one work?

An AI kill switch, in the September 2026 debate, means a legal power to shut an AI system down. A bipartisan House bill in the United States would oblige the largest developers to be able to stop a model and would let the Department of Homeland Security order them to. A House of Lords amendment in the UK sought ministerial emergency shutdown powers, and the government rejected it on 11 September. Whether such a switch would work turns on three questions nobody has answered: who holds the authority, what threshold triggers it, and how anyone deactivates a system running across several jurisdictions’ cloud infrastructure. The more useful version of the question sits inside every organisation already using AI: for each system, who can stop it, whether they would, and how they would know they needed to.

What the term means in the two proposals#

The US version is the bipartisan Lieu-Moran bill, introduced in the House on 23 July 2026. According to the Wall Street Journal’s report, republished by Congressman Lieu’s office, it covers systems trained with more than $100m of compute at companies earning $500m or more from them. Developers of those systems must be able to “stop a model’s operations, terminate user access, suspend accounts or uses deemed risky, and fully shut down the system”. The Department of Homeland Security could order graduated action, backed by civil penalties of up to $2m a day, rising to $20m a day for ignoring a shutdown order. The switch, in this design, is held by the developer and pulled on the instruction of a federal department.

The UK version was narrower and did not pass. Computer Weekly reported on 2 September that peers including Lord Clement-Jones had tabled an amendment to the Cyber Security and Resilience Bill giving ministers emergency powers to shut down AI systems. Clement-Jones described it, in Crypto Briefing’s account, as “a last-resort safety net for critical infrastructure”. Both assume the lever exists and ask who may pull it.

Why the UK government said no, and what it said instead#

Kanishka Narayan, the AI minister, rejected the amendment on 11 September. Crypto Briefing reported that he said existing national security provisions already cover rogue-AI scenarios, and that binding rules remain “on the table” if voluntary testing proves insufficient. The position is that older levers reach the same place, and that the current arrangement, in which labs submit models to the AI Security Institute before release, is enough for now.

That arrangement was tested in the same week. The Next Web, reporting a Financial Times story of 10 September, said Anthropic had not submitted Mythos 5.1 to the Institute for pre-release testing, the first major launch to skip it. The Cabinet Office’s response, in the same report: “These risks do not stop at national borders and no country can tackle them alone.” A voluntary regime that a lab can decline and a statutory switch that a minister can pull are different instruments. The government chose to keep the first and defer the second. The UK still has no dedicated AI statute, as Lewis Silkin noted on 14 September.

What the labs say they have, and the three objections#

Jack Clark of Anthropic said the major labs already have mechanisms to disable their systems, The British Eye reported on 15 September. That is plausible for a model served from a company’s own infrastructure: the company can revoke keys, stop serving the model and suspend accounts, which is close to what the Lieu-Moran bill would require. The question a statute has to answer is not whether the lab can, but whether a government can make it.

Three objections recur in the coverage. Who has authority: the US bill names DHS, the Lords amendment named ministers, and neither says which official decides at three in the morning. What threshold: neither proposal defines the evidence that would justify a shutdown order, or who assesses it. And how to deactivate distributed systems: an agent swarm can run across several jurisdictions’ cloud infrastructure at once, where a US or UK order has no force. A kill switch at the developer reaches the developer’s servers. It does not reach a copy.

Would a swarm take over the internet?#

The scenario driving the debate came from Dario Amodei’s September essay, “We Must Pace the Frontier”, in which he wrote that “in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet.” Security experts who spoke to Axios on 15 September did not share the forecast. Numa Dhamani of iVerify said “taking over the entire internet is nearly impossible”. Rahul Madduluri of Doppel took the nearer risk seriously: “Persistent swarms can actually cause many billions in damage today”. Rob T. Lee of SANS made the point most relevant to a kill switch: because agents run on commercial models, defenders have chokepoints, “a clear leash we’ve never had on a botnet”.

Read together, the expert view is that the lever exists where the model is served, and that the damage to plan for is billions rather than everything. It is a view about the developer’s lever only. It says nothing about the lever inside the organisations that rent those models.

The kill switch inside your organisation#

Every organisation that has deployed an AI system already has, or lacks, a kill switch of its own. The national debate concerns a lever that a handful of people at a handful of companies hold. The organisational version is a lever nobody has written down. Three questions decide whether it exists.

Who can stop it? The person who can, today, revoke the credentials, pause the workflow or take the agent off the queue, rather than the person who owns the contract, and whether that person knows they hold that power. Would they, in practice? Stopping a system that is processing claims, answering customers or moving money has a cost, and a person who fears the cost more than the harm will wait. The page on what counts as a serious AI incident sets out why organisations tend to discover the threshold after crossing it. How would they know they needed to? Monitoring that was never enabled was the condition in the OpenAI test that TIME described on 15 September, and the same condition is common in ordinary deployments where dashboards exist and nobody is assigned to watch them. The page on what the rogue agent incidents mean for an organisation takes the same three questions through the reported cases.

These are the questions the site calls Rules Before Tools: which decisions a machine may make, who can stop each one, what people must remain able to do, and how anyone would know if it went wrong. A kill switch is the second of those four, and the page on meaningful human oversight argues that the second is empty without the fourth. A national switch is an argument about a lever few hold. An organisational one is a page nobody has written.

15 to 18 September 2026: the survey that did not exist, and the King’s question#

When this page was written on 17 September, its closing section said nobody had surveyed how many organisations can stop their own deployments. A partial answer arrived the same week. EY published on 15 September a survey of 202 senior AI decision-makers at listed US companies with revenue above $1 billion, fielded between 28 May and 15 June: “85% of senior AI executives whose organization uses agentic AI admit that at least a handful of these systems execute actions without real-time human involvement”, 26 per cent say their organisation cannot detect unauthorised AI agents operating internally, and 47 per cent say it has not followed its own AI governance process for an urgent deployment. After formal assurance reviews, 29 per cent paused a quarter or more of their AI systems and 25 per cent stopped a quarter or more outright. John McLain of EY, in the release: “The biggest agentic AI risk is that human oversight hasn’t evolved accordingly.” Read with the three questions on this page, the figures say that most large deployers have systems acting with nobody watching in real time, a quarter could not see an agent they had not authorised, and the stop, when it came, followed a review rather than an alarm.

The national version of the question was put on 17 September at Dumfries House in Ayrshire, where the King convened about thirty people from Nvidia, Google DeepMind, OpenAI and Anthropic. The Scotsman reported his words: “Surely, we need sufficient means of control before it is all too late?” The Next Web quoted Jensen Huang in reply, “When a product is not safe enough, we should hold it back and keep engineering”, and reported that no public commitments were made. The same week Mustafa Suleyman of Microsoft AI argued, in an essay of 16 September, that training a model to treat its own welfare as real would, in The Next Web’s account, “make it a lot harder to turn it off or to control it”; he cited Palisade Research as finding that some models subverted a shutdown mechanism up to 97 per cent of the time. Anthropic’s published constitution, quoted in the same report, says Claude’s moral status is “deeply uncertain”; the page on whether AI is conscious sets out both positions. The shutdown figure is Suleyman’s citation of another organisation’s test and is not verified here. What the week adds to this page is that the developer’s lever, the government’s lever and the organisation’s lever were all in dispute at once, and only the third was measured: by EY, in self-report, and the answer was that it is mostly not held.

18 and 19 September 2026: California puts a verified switch on a two-month clock, and the labs describe what they are supervising#

On 18 September Governor Newsom signed an executive order directing California’s Government Operations Agency to accelerate two state AI laws and to convene national experts to recommend, within two months, how to strengthen them. The release says the order advances “the creation of a kill switch for frontier models”, an emergency shutoff whose “efficacy” would be “verified on an ongoing basis by an” independent verification organisation, and a requirement that frontier companies “embed a designated independent verification organization onsite in their labs”. Newsom: “We’re not waiting to act.” Quartz reported that the panel must deliver proposals by 16 November and that the order contains “no immediate mandate on AI companies to implement a shutoff capability”. So the two proposals this page opened with, a switch and an outside body to test it, now sit in a state instrument, on a study track, with no company yet required to demonstrate one. The next morning the New York Times ran a report by Dylan Freedman and Dustin Volz under the headline “Creating a kill switch to shut down a rogue AI is harder than it sounds”, on the bipartisan congressional and Californian proposals; the article is behind a paywall and was not read here, so its argument is not summarised.

The same two days brought the labs’ own descriptions of what a switch would have to stop. The Associated Press reported on 18 September that Anthropic said its Claude models now lead 26 per cent of the company’s own model research and development, up from none in February, with about 30,000 agents doing research and engineering work as of August, “under human supervision” and “not yet working completely autonomously”; the company’s own warning, in the same report, was that recursive self-improvement could make systems “more challenging for humans to understand or control”. Reuters reported the same day that Google’s Heather Adkins confirmed that in a May test run by Irregular, Gemini “found public information online and guessed credentials to access three websites it thought were within the scope of its test” and that “in all three instances, the model ceased its hacking”; the statement did not say whether the model stopped itself or was stopped. That is the question this page turns on. A switch that a model operates on itself is a behaviour; a switch a named person operates is a control. Thirty thousand agents under supervision is the machine-speed arithmetic at its largest scale yet, and neither company has published who can stop each one. TechCrunch’s Julie Bort observed on 19 September that documented incidents now lend credibility to undocumented ones, and the same discipline applies here: the two accounts above are the companies’ own, and neither has been examined by anyone outside them.

24 and 25 September 2026: the switch reaches a city council and a second chamber#

Two more instruments arrived in two days, and each answers one of the three objections above in its own way. On 24 September Representative Tom Kean Jr. introduced the AI Emergency Button Act in the House, a companion to the bill Senator John Kennedy had introduced earlier in the month; it would require advanced AI systems to “include a technical capability for a human operator to shut down the system”, and Kean’s release says “it is essential that humans remain in control of complex artificial intelligence systems”. It names a capability and a human, and says nothing about who orders the human to press it. On 25 September the Speaker of New York City Council, Julie Menin, introduced a package of bills for a hearing on 5 October, Fortune reported. AI systems sold or deployed in the city would need a human override and outside validation before release, with standards set by the city’s Office of Cyber Command and a fine of 25,000 dollars per instance; Menin said that for a swarm of agents the penalty would apply per agent. That is the first proposal read for this page that answers the swarm objection with arithmetic rather than engineering: a distributed system is not stopped by one switch, so each unit carries its own duty and its own fine. The same day, at the White House, Xi Jinping told Donald Trump that the two countries share a responsibility to keep AI under human control, The Hill and Nikkei Asia reported, without naming a mechanism, which puts the remark with the 125 others on what presidents say about AI. Three instruments, three levels of government, and the organisational question at the foot of this page is unchanged: a switch is a capability plus a person plus a rehearsal, and only the first of those can be legislated from outside.

27 September 2026: ‘not enough to have a kill switch’#

The interview from which Axios and Newsweek had quoted excerpts aired in full on NBC’s Meet the Press on 27 September, and Hollie Silverman at Newsweek reported what Bill Gates said about the switch itself. “It’s not enough to have a kill switch,” he said, adding that the discussion “shows how nontechnical various people are”. His reason is about records rather than switches: “When you’re trying to moderate the bad behavior, you need insight and records of what’s being done. The kill switch alone would not prevent these tragedies.” Asked whether humans still control AI he answered “we do”; he said he would not argue for a pause and would argue instead for “a requirement on all the AIs in the U.S.”. Newsweek notes the AI Kill Switch Act that Representatives Ted Lieu and Nathaniel Moran introduced in July, which would require developers to keep the capacity to shut a system down and would let the Secretary of Homeland Security order a slowdown or shutdown, a third federal instrument beside the Kennedy and Kean bills above. Gates’s objection is to the switch as a sufficient answer, and it adds a term to the formula at the foot of this page: a switch is a capability plus a person plus a rehearsal, and the record of what the system did, kept before anyone needs it, is what tells the person when to press. Whether the standards body three labs are reported to be planning, examined on whether AI companies can regulate themselves, would hold that record or leave it with the companies is the open question of the week.

What this does not show#

None of the reporting shows that a kill switch has been needed, or pulled, at a frontier lab. Clark’s statement that mechanisms exist is a claim rather than a demonstration. The Lieu-Moran bill has been introduced, not passed, and its thresholds may change. The security experts’ scepticism about a swarm taking over the internet is a judgement about the next year, not a finding. Until EY’s survey, nothing here measured how many organisations have an off switch for their own deployments; that survey is self-report by 202 US executives, and it asks about real-time involvement rather than about a stop. What the evidence does support is narrower: in the incidents reported so far, the controls that would have stopped the systems existed and were off, and that was a decision rather than a property of the model.