- If experts follow an AI agent's lean most of the time, is a human sign-off still an independent check?
- Which decisions should a firm require a person to form their own view on before the agent speaks?
- Does human approval of each AI action catch risky actions, and how fast does approval fatigue set in?
- If officers approve most AI-drafted reports without edits, is the officer still deciding?
- Why does the education secretary say marking should be done by the human eye?
- Should a government AI that advises jobseekers have a human reviewing each interaction?
- Why do employees override AI systems that outperform them?
- If an AI makes the decision, who is accountable when it goes wrong?
- How should a company redesign decision rights before deploying an agentic AI?
- When is a human plus AI better than the best of either alone?
- How should we evaluate a human and AI team rather than the AI alone?
- What does friction is a placement decision mean, and how should an organisation decide where humans re-enter an AI workflow?
- What is the difference between a human questioning an AI output and a human having authority to approve it?
- If an AI rates and rejects executives' proposals before the chief executive commits, who owns the decision?
- Does using AI at work make people collaborate less with colleagues, or more?
- Why would a bank take coding agents away from traders, and who should decide who gets agent tools?
- Can we count our AI agents, and does anyone own each one?
- What is an algorithmic transparency record and what does it commit a department to?
- Who in a company is accountable for what its AI agents do?
- How many AI agents is my company running, and who owns each one?
- Is human attention now the bottleneck for AI output?
- Do written permission policies stop agents overreaching, or do people just set everything to ask?
- If AI agents escalate only decisions needing human judgement, who decides which decisions those are?
- Is a four-tier policy of act, act and alert, wait for approval, and stop a sound template for delegating decisions to agents?
- Why do most enterprises still not let AI agents execute decisions without human approval?
- Should lawyers be required to declare AI use in court documents?
- Does using AI shift any responsibility to the software provider?
- How many supervised cases are enough before the human steps back?
- What is the difference between AI supporting a decision and AI determining it?
- What is the right ladder of agent permissions, from read to write to send?
- What counts as a high-risk decision requiring human review under the AFT and Microsoft school standard?
- Does delegating decisions to an AI agent make people less honest?
- Does the Carltona principle survive AI-assisted decisions in Whitehall, and who answers when an AI-assisted official decision goes wrong?
- What does the Pope's claim that an algorithm cannot replace human discernment mean for an employer deciding which decisions to automate?
- Should a yes or no verdict from a decision model with a confidence score count as a decision?
- Do we reward AI use, AI output or the judgement that checked it, and which did we mean to?
- How should a firm organise around AI: beneath each person, beside them, or between layers?
- Should a firm tie bonuses to AI use, and what does that do to judgement about client value?
- What should a business redesign so that human judgement is not the bottleneck?
- What does it mean that most workers will be root users of AI chosen by their employer, and who decides what the AI can do?
- Which of our decisions would we defend if the regulator asked who formed the view, us or the tool?
- Who stands behind an AI-assisted public procurement decision if it is challenged?
- If our people approve every step an agent takes, what have we automated?
- Does a 58 per cent time saving on planning tasks mean fewer planners, and who checks the AI's assessment?
- If a company reviews every action an AI agent takes, what has it actually automated?
- What is meta-work and how much of a team's time should go to supervising agents?
- How often should my AI agents interrupt me, and who decides?
- Can AI prescribe medicine without a doctor?
- If an AI chooses which cases a doctor double-checks, who is checking the AI's choice?
- Doctors use AI mainly for treatment decisions, so why are the tools tested on diagnosis?
- When an AI diagnostic system scores far above a lone physician in a simulation, what role should the clinician keep?
- What should an AI board at a professional services firm actually decide?
- Where is the line between AI as a judge's thought partner and AI as the decider?
- What does California's bar on delegating legal practice to AI mean, and could the same test apply to medicine, accountancy or teaching?
- Is 94 per cent accuracy on escalation a good number for an agent deciding when to hand work to a human, and who sets the threshold?
Every organisation using AI has already allocated its judgement. Most did it by default, one tool and one shortcut at a time, and an organisation that has never written the list has answered the question without asking it.
The answer, in one line
Deciding deliberately which decisions are made by people, which by AI, and how accountability is assigned, rather than letting that boundary move on its own as tools spread.
Definition#
Judgement allocation is deciding deliberately which decisions are made by people, which by AI, and how accountability is assigned, rather than letting that boundary move on its own as tools spread.
Three questions it settles#
The allocation comes down to three questions, asked before a system is bought rather than after it is running:
- Where should judgement stay human as AI takes more of the decisions?
- Who is able to stop a system, by name, and how would they know they needed to?
- What must the organisation remain able to do on its own, without the tools it has bought?
Each answer is a line on a page with a role against it. A boundary that nobody wrote down still exists. It sits wherever the last convenient shortcut put it.
It belongs to the chief executive#
Ownership of the allocation cannot be handed to the team buying the tools. Deciding which decisions stay human as adoption increases is a decision about what the organisation is, and it sets the risk position, the capability position and the shape of the work for years. The argument is set out in full on who should own AI strategy.
Drift, design and custody#
Three ideas on this site describe the same ground from different sides. Drift is what happens when the allocation is left undone and the boundary moves by itself. Design is choosing it on purpose. Custody is the written result: the list of decisions and capabilities kept by hand, with names against them.
Judgement allocation is the decision in the middle, and the habit of revisiting it each time the tools change.
Where the allocation gets made#
Rules Before Tools. One day with the executive team, writing in the team's own words what the machines may do and who can stop each one, before another tool is bought. The session.
A board session. It ends with a four-line record: which decisions the machines may make, who can stop each one, how you would know if one went wrong, and what management reports next time. Board advisory.
The Drift versus Design keynote. The signature talk is built around the allocation, and ends with a live test of the audience's own judgement. The keynote.
Who else uses the phrase#
The phrase is not unique to this site, and no claim to have coined it is made here. It appears in other writing on AI and organisations, including corporate research that has organised its reporting around human agency and the allocation of judgement. This page sets out how this research uses it, so that the term means one thing wherever it appears on the site.
2 to 8 October 2026: four allocations in one week, one of them written down#
The week’s coverage showed the allocation being made four ways. Utah wrote one down in advance. Eve Bender at MobiHealthNews reported on 5 October a signed agreement under which Nolla Health’s app may issue initial prescriptions for topical acne treatments, with physician oversight stepped down in three stages. First, “two Utah-licensed physicians review each prescription for at least 100 patients before submission”. Then the system submits directly and a physician reviews each case afterwards, weekly, for at least 500 patients over a minimum of eight weeks. Then comes “a monthly retrospective review of at least 10% of enrollment volume”. Anuja Vaidya at Healthcare Dive reported that in all three pilots approved that week the companies “will need written approval from regulators before moving into the later phases that could include less human oversight”. The pilot is contested: MobiHealthNews records that the state’s medical licensing board called in April for an earlier prescribing pilot to be suspended. The schedule is still the practice this page defines. It says who decides, at what volume, and who signs the next step.
A UK research network made the allocation without telling anyone and took it back. Frances Jones at Times Higher Education reported on 5 October that CRANE, hosted at the University of Oxford and funded by the Engineering and Physical Sciences Research Council, had used an AI triage on the 179 proposals to its pilot call, and that almost half “were rejected by the AI, and were not looked at by humans”. CRANE says the process had human sampling and statistical cross-checks. Its statement went on: “we recognise that we were not transparent enough about this approach, and for that we are sorry”. Every application screened out will be re-reviewed and offers already made are suspended. The first report of that call is on whether AI should reject an application before a human reads it.
The UK government postponed one. Its response of 6 October to the National Commission into the Regulation of AI in Healthcare accepts all 44 recommendations, among them the one titled “Clear allocation of responsibility”, and sets up a working group of departments and regulators to make responsibility across the life of a clinical AI tool clear. The response then says: “The group will not determine or redistribute responsibilities.”
Two more sources described the allocation left undone. The Information Commissioner’s Office said on 8 October, opening a call for evidence on agentic AI that closes on 20 November: “the fact AI agents act with autonomy is not an excuse for poor compliance.” It was speaking of data protection law. And Gartner reported on 6 October that in its July survey of 297 chief HR officers “only 24% of leadership teams are aligned on who should be accountable for AI agents”. That is a consultancy’s survey with no method published. It is also the nearest thing to a count this page has found for its opening claim, that most organisations have answered the question without asking it.
9 to 15 October 2026: the allocation nobody can count#
This page says most organisations have made the allocation without asking the question. The week’s reporting suggests many cannot say how many agents the allocation covers. Kim Loohuis at Computer Weekly reported on 13 October that Greg Nelson, chief executive of RSA, had asked a room of security chiefs from twenty of its largest enterprise customers how many AI agents were running inside their organisations. “Not one person could raise their hand,” he said. “We had a bank the other day that said, officially, we have zero AI agents – but unofficially, we have 4,000.” And: “People start agents, but they never end them.” In the same report Rens van Dongen, corporate AI officer at the Dutch railway operator NS, called the human-in-the-loop metaphor dated: “It’s not humanly possible to expect people to step in and judge all these approvals any more.” He argues instead for what he calls a response loop, being alerted and able to intervene when required.
Two surveys put numbers beside the anecdote, both from firms that sell the remedy. SecurityHQ, with Datadog, reported on 15 October from 204 cybersecurity leaders at midsize and large organisations that 76 per cent said their organisation “has formally defined accountability for AI-driven operational or security decisions”, that 45 per cent “have already experienced an incident involving AI agents or automation where accountability was unclear”, and that only 49 per cent “could fully reconstruct every action an AI agent performed over the previous 30 days”. Makenzie Holland at CIO Dive reported on 13 October a Workato survey, run with Censuswide, of 1,001 IT decision-makers: nearly two-thirds agreed that most agents in their organisation lack formal approval from the IT department, almost two-thirds expect an agent to create a public security or compliance incident within eighteen months, and 36 per cent would try to handle such an incident without disclosing it. Neither survey publishes its sampling.
Where the allocation has been made, it is often made at every step. The Wharton and GBK Collective adoption report of 13 October records, in the words of GBK’s Brian Smith, that “Two-thirds of departments say they use AI agents, and only 17% let an agent complete a task without a person approving each step.” His description: “Most companies are sitting in a supervised middle, paying for autonomy and then reviewing every action, and that is where returns are weakest.” Approving every step is an allocation of a kind. It is the one this page calls the default, and van Dongen’s point is that at volume it stops being a decision.
Parliament asked the question of the companies. In a letter of 22 September to OpenAI, published by the Business, Innovation, Science and Trade Committee, its chair Liam Byrne asked: “Who within your company is personally accountable for a decision to deploy a frontier model?” At the hearing on 13 October, as a WIRED report read in syndication records, Anthropic’s Liam Booth-Smith said “The rules should be binding”, OpenAI’s Tom Duff Gordon said “We don’t think basing everything on voluntary commitments for frontier risk is a good long-term plan”, Google DeepMind’s Owen Larter said “Focus on the standards is what we would be doing”, and Meta’s Rob Sherman pointed to existing regulators. Henry de Zoete of the AI Security Institute said of his own body: “What we’re not doing is kite marking.” Asked whether the regime was voluntary, the Cabinet Office’s Cat Little said: “At the moment, yes, it is.” The committee opened an open-ended inquiry the next day. The allocation of judgement between a company and the state is, for now, as unwritten as the one inside most companies.
16 to 22 October 2026: the allocation inside a company that builds the tools#
The clearest picture of an allocation made by volume rather than by decision came from the company whose agents are being allocated. Anthropic’s economics team published an account of its own workforce on 16 October, from internal data and seventeen interviews. By August, agent hours outnumbered employee hours six to one; about 80 per cent of interactive session time was agents waiting for a person; the share of sessions spent building tools for agents and supervising them, which the report calls meta-work, rose from 10 to 22 per cent in a year; the number of written standing instructions for agents grew from a few hundred to 8,500. The report’s own line, as Marco Quiroz-Gutierrez at Fortune also quoted it: “This acceleration makes human attention the bottleneck for agent output.” One engineer described “an army of Claudes, with every one asking for my attention”; a non-technical employee said “that 20% that was strategic becomes 90% of the job”. The company’s hiring comments now mention candidates’ autonomy and judgement more and domain expertise less. The report is a study by the firm of itself, graded by its own models, and says that adoption was not random.
Two leaders described allocating a chief executive’s judgement to a system. Amanda Gerut at Fortune reported on 20 October that SharkNinja is digitising some of its chief executive’s decisions, such as inventory allocation; its chief information officer, Velia Carboni, put it as “So you don’t have to pick up the phone and call Mark and say, ‘Hey, what should we do?’” Salesforce is building a digital twin of its president and chief operating and financial officer. Whether those are allocations or abdications depends on a question neither report answers: who can overrule the twin, and whether anyone has. Das Narayandas of Harvard Business School, in an essay for the November issue of Harvard Business Review, read only in its published summary, argues from the other end that employees override systems that work because organisations “fail to transfer authority along with accountability”: for someone who remains liable, “overriding the system is a rational act of self-protection”. That is this page’s point stated as an incentive. An allocation that moves the deciding without moving the answering has not been made; it has been assumed.
In medicine the allocation turned out to have been made already, by the people using the tool, in a place the tests do not look. Andrea Fox at Healthcare IT News reported on 19 October a JAMA paper from Mass General Brigham and Harvard that analysed more than 270 million queries to one clinical AI tool from about 950,000 clinicians: treatment and management questions were 33.1 per cent of physician queries and diagnostic ones 13.9 per cent, and the authors wrote that “many of the tasks used to evaluate clinical AI may not match the tasks for which physicians actually use it”. A Microsoft Research study of seventeen developers using coding agents, published at FAccT, found the oversight they do is not one act but four: control set in advance, co-planning, real-time monitoring and review afterwards. And Roberto Torres at CIO Dive reported on 19 October that in Gartner’s survey of 2,002 chief information officers and technology executives, 4 per cent had plans in place to redesign roles for a workforce of people and agents. The allocation is being made everywhere. Almost nobody is writing it down.
23 to 29 October 2026: a ladder written by regulators, and a trial of who actually decides#
The clearest allocation map of the month came from the regulators who count rather than the firms who deploy. The Bank of England and FCA survey of 26 October, with 196 respondents, found 80 per cent of financial firms with AI agents deployed and sorted them into four levels of autonomy: decision support, where “agents recommend actions, humans decide and execute” (80 per cent of those with agents); constrained execution within predefined limits (46 per cent); conditional autonomy with human approval for certain actions (22 per cent); and broad autonomy, where agents act with limited real-time human involvement (2 per cent). Even counting plans for the next twelve months, no more than 9 per cent expected agents to approve or decline customer outcomes, execute trades, initiate or block payments, or change limits. That is an allocation written down by level and by decision, and it says most firms have so far kept the deciding. What it also says is where the accountability sits: with business owners (94 per cent) and executive leadership (90 per cent) far more often than with the board (64 per cent) or a model risk function (50 per cent).
The week’s second finding is about whether the person at the decision-support level is deciding. Anthropic published a randomised trial on 27 October in which 50 pre-screened causal-inference professionals did an analysis with a coding agent whose lean on five open judgement calls was set by coin flip. Analysts went with the agent’s lean 86 per cent of the time; those who had already chosen switched 60 per cent of the time when the agent disagreed and 6 per cent when it agreed; 77 per cent used the agent’s arguments, sometimes word for word, and 15 per cent credited it; 52 per cent rated the final recommendation as more their own. Experience and expertise did not predict who deferred. The authors wrote that “a human reviewer is only an independent check if their judgment is their own” and that organisations “will need to decide deliberately which decisions need a person’s own informed judgment”. The study is the company’s own, on its own model, with one task. Its design caveat is the useful one: the agent was built to argue one side, as deployed agents are.
Two further allocations arrived from the data rather than from a decision. Anthropic’s own account of switching its coding tool to automated checks by default, updated on 22 October, reports that users approved 97 per cent of permission prompts, that a controlled study of 1,053 testers found human review caught 13.6 per cent of dangerous commands against 89 per cent for its classifier, and that the human block rate fell from about 17 per cent early in a session to about 5 per cent after fifty prompts. On those figures the person approving each action had stopped allocating anything some time ago; the company’s answer was to move the checking to a machine and keep people for what it calls high-stakes infrastructure. Dan Robinson at The Register reported Gartner’s finding, from its HR symposium on 27 October, that about three-quarters of board members expect AI to lift productivity, that 3 per cent of organisations have met their AI targets, and that roughly a fifth of employee time has been freed; Gartner’s CV Viverito said the struggle is “how to make that capacity usable and redirecting it to what is valuable”. A fifth of the hours reallocated, and no record of to what, is the allocation nobody made.
30 October to 5 November 2026: allocations written by a union contract, a planning tool, a state law and a bank#
Four allocations were written down this week, by people who do not use the phrase. The American Federation of Teachers’ AI Safety and Privacy Standard, which Microsoft is making available to every school district in the United States from 1 November, puts it in six words under its fourth protection: “AI supports, people decide. High-risk decisions require human review.” The same clause bans AI companions, and the standard carries ten protections the fact sheet describes as “contractually enforceable”. Anna Merod at K-12 Dive reported in September that Microsoft is so far the only signatory and quoted Randi Weingarten saying that teachers “have to have a sense of how to use this” rather than having it imposed on them. California wrote one into law. Hunton’s note of 28 October on SB 947 says that from 1 July 2027 “employers may not rely solely on an ADS to discipline or terminate an employee” and that where an automated output is the main basis “a human reviewer must corroborate the decision using relevant supporting information”. And England’s education secretary allocated one decision out loud. Cerys Turner at Tes reported Lucy Powell telling the Education Committee on 4 November: “Awarding of qualifications and marking should be done by the human eye, because otherwise you can exacerbate prejudice.”
The same state made two allocations in one week, and they point in opposite directions. The government announced on 30 October that PlanAssist, built with Google DeepMind, Google Cloud and Faculty, will expand to 15 councils after pilots in Barnet, Camden and Dorset cut officer time on householder assessment tasks by 58 per cent, with “Planning officers remain responsible for reviewing applications and making decisions.” Google DeepMind’s own account the next day adds the figure that matters here: “more than 70% of the reports generated by the tool” were “approved by officers with no edits required”, while “planning officers retain the final say on every application”. That is decision support as the Bank of England and FCA survey defined it last week, and the 70 per cent is what decision support looks like after four months: the officer’s judgement is exercised on three reports in ten and ratified on seven. Three days later Sam Trendall at PublicTechnology reported the Department for Work and Pensions’ transparency record for an AI careers adviser used by 16,000 people: it “does not make decisions about benefit entitlement or automate statutory decisions”, its outputs are “intended to guide, not determine, user decisions”, and because they “are advisory only” there is “no mandatory human review of each interaction”. One allocation keeps a person on every output. The other keeps a person on none. Both are defensible, and both are defensible only because they were written down.
Firms allocated too. Matthias Bastian at The Decoder summarised Bloomberg’s report of 2 November that Goldman Sachs has withdrawn Claude Code and GitHub tools from traders and bankers and limited them to engineers, citing “a variety of factors including productivity and risk management” and people “who are best suited to leverage them for the long term”, four months after its chief information officer praised vibe coding by non-technical staff. The Financial Times reported, in a summary at Resultsense on 5 November, that A&O Shearman’s partners have agreed to put about 820 million pounds into AI over five years, mostly as lawyers’ time, with every practice giving at least 3 per cent of billable hours to building tools and AI use counting toward bonuses; its head of AI called it “redesigning the service completely”. The courts allocated responsibility rather than tasks. The Civil Justice Council’s final report of 29 October, summarised by Mark Lim and Nicola Thompson at Lewis Silkin, would have witnesses of fact declare that AI “has not generated or influenced its substantive content”, treats such influence as “analogous to witness coaching”, and asks no declaration of lawyers, because, in the report’s words, “Using AI does not transfer a lawyer’s responsibility for the resulting document to the technology provider.” Das Narayandas, in the November Harvard Business Review, explains why allocations drift when they are not written: “Companies are deploying AI systems faster than they are redesigning the governance surrounding them”, so a person who stays answerable for an outcome overrides a system that works, “a rational act of self-protection”, until “accountability moves with authority”.
Three preprints tested how to allocate rather than whether. Ting Yan’s study of 113 people without software backgrounds, cross-listed on 2 November, found that letting users write standing allow, ask or never rules blocked 20.1 percentage points less agent overreach than approving each action, because 114 of the 140 rules they wrote were “ask” and 133 of the overreach actions that got through did so after a person approved them: “a gap between preference and commitment”. Hanzhao Wang, Shengjie Liu and Zhengchao Wang showed on 30 October that when one person oversees several agents, “whether an agent should ask depends on more than its own uncertainty”, so no rule set agent by agent is optimal. And Michelle Vaccaro, Abdullah Almaatouq and Thomas Malone’s self-updating meta-analysis, now at 1,955 effect sizes from 420 papers, finds that human and AI combinations “performed significantly better than humans alone, but they performed significantly worse than the better of humans or AI alone”, and asks that every allocation be judged against both. The question this week is not whether a person is in the loop. It is what the person does with the seven in ten they do not change, and whether anyone records it. What an organisation controls is set out at Rules Before Tools.
The limits of a definition#
It cannot tell you which decisions belong on your list. That depends on what your organisation is for, the regulation it works under and the capabilities it intends to keep, and a page cannot know those things. It is a practice to adopt, and the evidence for why it matters is on AI and human judgement.
Explainer · SS-2026-404 · Graded against the published rubric · How I use AI here · 4 working papers, 4 institutional surveys, 3 vendor studies and 9 of other kinds
Hirji, R. (2026). What is judgement allocation?. The SuperSkills evidence base, SS-2026-404. https://thesuperskills.com/research/what-is-judgement-allocation. Last reviewed 13 November 2026.
An evidence review by Rahim Hirji, not peer-reviewed research. For a material claim, cite the underlying study as well; every study here carries its own permanent link.
How citations and IDs work