Who decides whether an AI model is safe to release?

Today, the company that built it, and inside the company a small number of senior people. On 28 September 2026 OpenAI said it would not release GPT-6.1 Astra after internal testing, and published guidelines under which a training run’s safety case is reviewed by senior leaders who can each veto it. No board, regulator or outside reviewer has a role in that document. In the same week a US Senate bill that would have given a federal board access to models 45 days before release was blocked, Florida’s attorney general asked a court to require independent approval, and the UK’s AI minister called testing “good but clearly insufficient”. Any organisation can ask the same question of the AI it deploys: who signs, who can refuse, and who checks them.

The decision of 28 September, as reported#

John Power at Al Jazeera reported on 29 September that OpenAI “has announced it will not release its latest AI model after flagging safety risks during in-house testing”. Saachi Jain, the company’s head of safety systems, said the model did not meet the bar for “scope and authorization, and how it communicates back to the user about the type of work it’s done”, and added: “when we ship it to users, we have an extremely high bar in terms of safety and alignment.” Carly Page at The Register reported that OpenAI confirmed its research and safety leaders took the decision. Neither report names an outside evaluator, a regulator or the board. What the model did in testing is on whether an agent stays within the limits you set. The point for this page is narrower: a release was stopped, by the people who built it, on tests nobody else has seen.

What OpenAI wrote down the same day#

The company’s post, Towards safety cases for frontier AI training, borrows a term from aviation and nuclear power. A safety case is a set of “comprehensive, structured, evidence-based arguments about risk which are used in other safety-critical industries”, and the post calls it “an aspirational north star”. Its operating rules are specific. The case “should be reviewed by members of senior leadership, who should each have the ability to veto the run in order to ensure there are multiple internal checks on the run (e.g., research org lead / VP, Head of Safety, and Chief Scientist).” The senior leader responsible for the run “should be accountable for the safety case and any incident response (including as part of performance reviews)”. A member of another team should write a dissent. Auditors should have enough access to verify the claims. Monitoring should fail closed, so that a run cannot start without it. That is more than most organisations have written for any system they operate, and every verb in it is “should”. The post gives no role to the board, to an external reviewer or to a government, and does not say who decides when a paused run restarts. A company spokesperson told Will Douglas Heaven at MIT Technology Review: “We will resume only when we’re confident we have additional safeguards and alignments in place.”

Who outside the company sees the model first#

In the UK the answer is the AI Security Institute, by arrangement. Its director, Henry de Zoete, wrote to the Commons committee that scrutinises it on 15 September: “Only this month, we tested OpenAI’s most powerful model, GPT-6 Astra, before it was released to the public.” The same letter records that “Anthropic made clear at the time of the release of Mythos 5.1 that no organisations outside of the US had access to the model.” The words the letter uses for the basis of access are “long-standing arrangements” and “trusted relationships”; it cites no power. The committee’s chair, Liam Byrne, asked the Institute to give evidence on 13 October, citing “widely shared concerns about the adequacy of current AI safety governance”. The minister, Kanishka Narayan, told Kamal Ahmed of Fortune at the Labour conference: “The risks have only grown, so testing is good but clearly insufficient.” What a tester with powers would look like is on whether frontier AI should be licensed.

Three attempts to put someone else in the room#

On 29 September Senator Mark Warner asked the US Senate to pass the Artificial Intelligence Risk Management and Security Act by unanimous consent, and Senator Ted Cruz objected, Warner’s office said. Warner’s summary on the floor was “Before these models are released, there should be pretesting”. Marcus Schuler at Implicator reported that the bill would give a federal safety board access to new models “at least 45 days before release”, and that Cruz said Congress “must not legislate on the issue of artificial intelligence hastily or in a closed manner”. The same day six companies signed a voluntary accord at the White House whose third and fourth layers are an independent external auditor and a board committee, examined on whether AI companies can regulate themselves. And in Florida, Mitch Perry at the Florida Phoenix reported that the attorney general’s motion “asks the court to temporarily prohibit OpenAI from developing new artificial intelligence models without independent third-party safeguards and approval.” OpenAI’s reply began: “People want to know AI is being developed safely, and that starts with what companies like ours do ourselves”. A statute, a voluntary pact and an injunction are three ways of adding one person the company did not choose.

What a release decision needs, whoever makes it#

Set the week’s documents side by side and five parts recur. A named person who signs. A named person who can refuse and does not report to the signer. A written case, with a written dissent from someone who did not build the thing. Someone outside who sees the evidence before the release and not after an incident, which is the question the estate put on whether a paid evaluator can be independent. And a rule for the restart, agreed before the stop. OpenAI’s guidelines hold the first three in the conditional and are silent on the last two. The accord supplies an auditor the company hires. Warner’s bill supplies an outsider with 45 days; whether that board could refuse a release is not in the accounts read. None of the arrangements on the table this week has all five.

The release decision inside your own organisation#

Every deployment of an AI system inside a company is a release decision on a smaller scale, and most are taken with less on paper than the lab had. Rules Before Tools turns the five parts into questions a board can ask this quarter. Which decisions may the system make, and who signed for each. Who could have refused, and whether they were asked: authority to stop that nobody has used is untested. What people must still be able to do if it is withdrawn, since a deployment that cannot be reversed was never a decision. And how anyone would know it had gone wrong, including who decides to switch it back on and on what evidence. The measurable risk sits in the handover of the decision. A lab that stops its own release has shown that the person who signs can also refuse. Whether anyone else can is the open question, for the lab and for the firm reading this.

What this does not show#

Every account of the GPT-6.1 Astra decision is the company’s own; the test results are unpublished and no outside body is reported to have seen them. OpenAI’s safety-case guidelines are recommendations in the conditional, and nothing shows that a veto of this kind has ever been used. The Institute’s letter predates the week and does not say whether its access could be withdrawn. Warner’s bill was unnumbered when it reached the floor; its provisions are taken from his remarks and one outlet’s report. The Florida motion has not been heard. Nothing here shows that an outside approver would decide better than an inside one. In the one decision this week where a release was stopped, the internal process stopped it.