Human in the loop is not a safeguard

"We keep a human in the loop" is the most widely deployed AI safeguard in the world and one of the least examined. It appears in board papers, procurement documents, regulatory submissions and press statements, and it is almost always offered as the end of a conversation rather than the beginning of one. It should be the beginning, because the best available evidence says the arrangement it usually describes makes results worse.

This is a disagreement page. It argues something the consensus does not, it names what would change my mind, and it takes the strongest objections seriously rather than the weakest.

The finding

In 2024, Vaccaro, Almaatouq and Malone published a systematic review and meta-analysis in Nature Human Behaviour, pooling 370 effect sizes from 106 experiments. Human-AI combinations performed significantly worse on average than the better of the human alone or the AI alone. Not worse than both. Worse than whichever party was stronger on its own.

The disaggregation matters more than the headline. Losses concentrated in decision tasks, where a person judges whether a system's output is right. Gains appeared in creation tasks, where person and system make something together. And the strongest single predictor of direction was the baseline: where the human alone outperformed the AI, combining them helped; where the AI alone outperformed the human, combining them dragged the result down towards the human's level.

Read that last sentence again with an organisational deployment in mind. The configuration described by "human in the loop" is a decision task, performed on output the person did not generate, usually under time pressure, frequently by someone who could not have produced the work themselves. It is the exact shape the evidence says underperforms, and it is the default because it is the easiest thing to install, not because anyone tested it.

Three mechanisms, all documented, none new

Monitoring is the hardest task, not the easiest. Bainbridge set this out in 1983 and called them the ironies of automation: automate the routine and you leave the human with the residue that requires the most skill, while removing the practice that built the skill. Forty years and one technology later nothing about that has been repealed.

Confident output suppresses scrutiny. Parasuraman and Manzey's review found automation bias and complacency in experts as well as novices, resistant to training and worsening under workload. Dzindolet and colleagues found something more awkward: explaining how an automated aid can fail can increase reliance on it. Awareness is not a control.

The average conceals opposite effects. Yu and colleagues found the effect of AI assistance on radiologists running from strongly positive to strongly negative between individuals, unpredicted by experience or prior familiarity. A policy of the form "clinicians will review the output" is a policy with unknown sign, applied uniformly to people it affects in opposite directions.

The strongest objections, taken seriously

"The meta-analysis predates current models." True, and it is the most substantial objection. The studies run to 2023 and the systems are less capable than today's. But note which direction that cuts: the finding is that combination fails most where the system already outperforms the human. Better systems make that condition more common, not less. Improving capability should be expected to worsen this problem, not solve it.

"Oversight is for accountability, not accuracy." This is the best defence and it is legitimate. Someone must be answerable, and a system cannot be. But it should be argued honestly: you are buying accountability and legitimacy at a measurable cost in accuracy. That is often the right trade, particularly in medicine, law and public administration. It is a trade. Presenting it as getting the best of both is the part that is not honest.

"Regulation requires it." Article 14 of the EU AI Act, in force since 2 August 2026, does require human oversight of high-risk systems. But read what it actually demands: that the overseer can detect anomalies, remain aware of automation bias, interpret output correctly, and disregard or stop the system. Those are capabilities. The Act is not endorsing nominal review; if anything it is legislating against it. See meaningful human oversight.

"Our people are experienced." Logg and colleagues found people frequently weight algorithmic advice more heavily than human advice, with domain experts the notable exception. That is genuinely reassuring, and it is also an argument for expertise rather than for the loop. It only helps if the reviewer has the specific competence, which is precisely the thing most deployments have not checked.

What would change my mind

A replication of the Vaccaro analysis using post-2023 systems that found combinations beating the better party in decision tasks. A field study showing that oversight arrangements meeting the Article 14 capability requirements outperform both parties alone. Or evidence that the individual heterogeneity Yu found is predictable in advance, which would make selective oversight designable rather than a lottery.

None of those exists yet. If any appears, this page changes and the change will be dated in the corrections ledger.

What to do instead

The argument is not to remove humans. It is that where you put them decides whether they add or subtract, and the loop is the worst available position.

Move the human to the front. Problem definition, intent, context, constraints and rejection criteria, all before anything is generated. That converts a decision task, where the evidence says combination subtracts, into a creation task, where it adds. It also gives the later reviewer a position of their own to compare against, which is the difference between reviewing and checking for fluency. This is Human at the Start, and the stage-by-stage working version is the Delegation Boundary Map.

Then apply four tests. Can the reviewer detect the error, honestly? Does the pairing beat the better of human alone and system alone, measured rather than assumed? Is the override rule written down before the decision? And is anyone counting overrides, given that zero is evidence of an untested right rather than a good system?

And fund the practice. The human half has to stay competent at work the machine is doing, or the oversight degrades to approval on a schedule nobody is tracking. That is capability debt, and it is why a safeguard installed today can be hollow in three years without anyone changing the process.

The wider point

"Human in the loop" has become an accountability comfort phrase: it ends discussions, satisfies committees, and survives in documents precisely because nobody asks which human, with what capability, at which point, on what grounds. A safeguard that cannot be tested is not a safeguard. It is a description of an org chart.

The honest version is harder to say and much more useful: we have decided where human judgement enters this work, we have checked that the person there can tell when the system is wrong, and we have measured whether the arrangement beats the alternatives. Almost nobody can say that sentence today. Everyone can say the other one.

Related SuperSkills research

On the definition and the evidence, what is human-AI collaboration. On the legal duty, meaningful human oversight. On who carries it, who owns verification. On the tendency underneath, automation bias. On the board test, what should a board ask about AI. See when to override AI.

Key sources

• Vaccaro, M., Almaatouq, A. and Malone, T. (2024). When combinations of humans and AI are useful. Nature Human Behaviour, 8.
• Yu, F. et al. (2024). Heterogeneity and predictors of the effects of AI assistance on radiologists. Nature Medicine, 30(3).
• Parasuraman, R. and Manzey, D. H. (2010). Complacency and Bias in Human Use of Automation. Human Factors, 52(3).
• Logg, J. M. et al. (2019). Algorithm appreciation. Organizational Behavior and Human Decision Processes, 151.
• Bainbridge, L. (1983). Ironies of Automation. Automatica, 19(6).
• Article 14, Human Oversight, Regulation (EU) 2024/1689.

About this research

Rahim Hirji is the author of SuperSkills: The Seven Human Skills for the Age of AI (Kogan Page, 2026) and founder of The SuperSkills Intelligence Company. This is a position page and states a view the field mostly does not hold. The findings are attributed to the studies that produced them, the objections are given in their strongest form, and what would change the position is stated above rather than on request. Reviewed quarterly.