What is human-AI collaboration?

Human-AI collaboration is any arrangement where a person and an automated system contribute to the same piece of work. The honest definition has to begin with an inconvenient finding: on average, it does not work. A 2024 meta-analysis in Nature Human Behaviour pooled 370 effect sizes from 106 experiments and found that human-AI combinations performed worse, on average, than the better of the human alone or the AI alone. Not worse than both. Worse than whichever was stronger. That is the baseline any claim about collaboration has to clear, and most organisational deployments never test against it.

Which does not mean collaboration is a myth. The same analysis found real synergy in a specific place, and the pattern of where it appears and where it disappears is the most useful thing anyone has published on this question.

What the systematic review covered#

Vaccaro, Almaatouq and Malone conducted the systematic review, covering studies published between 2020 and 2023. The headline average loss is striking, but the disaggregation matters more.

Synergy appeared reliably in creation tasks, where the human and the system are producing something. It failed in decision tasks, where the human is judging whether the system is right. And the strongest predictor of which way a study went was the baseline: when the human alone outperformed the AI alone, combining them helped; when the AI alone outperformed the human, combining them dragged the result down towards the human's level. In other words, human oversight of a system already better than you tends to subtract.

That is not an argument for removing oversight, and the medical evidence shows why. Yu and colleagues, in Nature Medicine in 2024, found the effect of AI assistance on radiologists varied enormously between individuals, was not predicted by experience or by prior AI exposure, and helped some readers while degrading others. An average effect concealed opposite effects on different people. Any policy of the form "clinicians will use the tool" is therefore a policy with unknown sign.

The mechanism behind decision-task failure is well documented and is not new. Bainbridge described the ironies of automation in 1983: automate the routine and you leave the human with the hardest residue, monitoring, while removing the practice that made them able to do it. Skitka and colleagues measured the two resulting error types directly in 1999, omission and commission. Parasuraman and Manzey's 2010 review established that these are properties of attention allocation rather than of carelessness. None of this required generative AI to be discovered, and generative AI has not repealed it.

The experiments stop at 2023#

The meta-analysis covers experiments up to 2023, using systems less capable than today's, and its tasks are mostly short and laboratory-based. Long-horizon collaboration, where a person and a system work together over weeks and the person learns the system's failure modes, is barely represented. It is plausible that experienced pairings behave quite differently from first-encounter ones, and equally plausible that they behave worse because familiarity increases complacency. Nobody has the data.

A fair objection also applies to the creation-versus-decision split: it is a pattern found across heterogeneous studies rather than a mechanism anyone has isolated experimentally. It should be treated as a strong working hypothesis, not a law.

The worst configuration, called responsible#

Most organisations are running the configuration the evidence says is worst, and calling it responsible practice. The machine drafts, the human reviews, and the human is described as being in the loop. That is a decision task, performed on output the reviewer did not generate, usually under time pressure, frequently by someone who could not have produced the work themselves. It is the exact shape the meta-analysis found to underperform. It is the default because it is the easiest to install rather than because it works.

The alternative I argue for is Human at the Start: the human sets the question, the constraints and the rejection criteria before the system produces anything. This is not a preference for humans over machines. It converts a decision task, where combination subtracts, into a creation task, where the evidence says it adds. It also gives the reviewer a position to compare against, which is what makes review something other than a fluency check.

Two corollaries follow, and they are the ones organisations skip. First, if the system is genuinely better than the human at a task, the honest options are to let it run without theatre and accept the accountability, or to keep the human and accept that you are paying for a slower and less accurate result to preserve something else you value, such as legitimacy or the ability to explain a decision. Both are defensible. Pretending you are getting the best of both is not, and that pretence is what I call usage theatre.

Second, collaboration has a maintenance cost nobody budgets. The human half of the pairing has to stay capable enough to disagree, which means retaining practice at work the machine is doing. Left alone, that capability erodes silently, which is capability debt. A collaboration design that does not include how the human stays good at the thing is a collaboration design with an expiry date on it.

Designing a pairing that clears the baseline#

• Test against the better half, not against nothing. If the pairing does not beat the stronger of human-alone and system-alone, it is costing you something. Most deployments have never measured this.
• Specify the point of human entry. Before generation, not after. Question, constraints and rejection criteria first.
• State the override rule in advance. On what grounds may the human overrule the system, and on what grounds must they defer? Unspecified, this collapses into whoever is more confident.
• Measure at the level of the individual. The Yu result means an average is not a finding. Some of your people are being helped and some are being made worse.
• Resource the retention of skill for whatever the human is supposed to supervise. Otherwise the supervision degrades to approval.

Related SuperSkills research#

On decision architecture, human and AI decision making. On the underlying tendency, automation bias and how to know when AI is wrong. On the organisational choice, drift versus design and AI workforce strategy. The claims themselves, banded by evidence strength and including what remains unknown, are in what we actually know about AI and human capability. The operational version, stage by stage with a downloadable grid, is the Delegation Boundary Map. On what oversight is now legally required to enable, meaningful human oversight. The position that follows from this, put simply, is that human in the loop is not a safeguard. On the definition, the jagged frontier.

Key research and primary sources

• Vaccaro, M., Almaatouq, A. and Malone, T. (2024). When combinations of humans and AI are useful: a systematic review and meta-analysis. Nature Human Behaviour, 8.
• Yu, F. et al. (2024). Heterogeneity and predictors of the effects of AI assistance on radiologists. Nature Medicine, 30(3).
• Bainbridge, L. (1983). Ironies of Automation. Automatica, 19(6).
• Skitka, L. J., Mosier, K. L. and Burdick, M. (1999). Does automation bias decision-making? International Journal of Human-Computer Studies, 51(5).
• Parasuraman, R. and Manzey, D. H. (2010). Complacency and Bias in Human Use of Automation. Human Factors, 52(3).

About this research#

Rahim Hirji is the author of SuperSkills (Kogan Page, 2026), keynote speaker on AI and human capability, and founder of The SuperSkills Intelligence Company. Human-AI collaboration is an established field term and is not his coinage; Human at the Start, usage theatre and capability debt are. Findings are attributed to the studies that produced them and kept separate from the interpretation. The graded evidence, including what each study does not support, is in the evidence base.

How this research works  ·  Reviewed quarterly  ·  Found an error? Tell me and it is corrected on the page.