Each entry gives the study, how it was done, what it found, what it supports and, as plainly, what it does not. Grades describe what kind of evidence a source is; how to read them is on the main page. Every entry has a permanent link and a copyable citation.
Peer-reviewed#
Bainbridge, L. (1983). Ironies of Automation
Automatica, 19(6)
- Method
- Theoretical analysis of automated process control systems and the human roles left within them.
- Finding
- Automating the routine parts of a task leaves the human with the hardest residue, monitoring and exception handling, while removing the routine practice that built the competence to do it. Automation makes the remaining human role harder, not easier.
- What it supports
- That monitoring is a demanding task rather than a light one, and that the design of automation determines whether the human retains the capability to supervise it.
- What it does not support
- Anything specific to AI. It is a process-control argument from 1983, and its application to generative systems is by analogy rather than by measurement.
Argued perspective#
Wooldridge, M. and Jennings, N. R. (1995). Intelligent agents: theory and practice
The Knowledge Engineering Review, 10(2), June 1995, 115-152. Cambridge University Press. Volume, issue, month, year and page range confirmed in both directions against the publisher's own issue listing and the article's own title page. Cambridge Core full text is paywalled; the text quoted here was read in the authors' own HTML at Oxford and cross-checked against a typeset published-version PDF, and the two differ only in one citation on the autonomy bullet and one word in a lead-in sentence.
- Method
- A survey and roadmap of the agent literature to 1995, proposing a two-level definition of agency and reviewing theories, architectures and languages. Reports no new data.
- Finding
- Sets out a weak notion of agency as a computer system with four properties, in the authors' words: autonomy, 'agents operate without the direct intervention of humans or others, and have some kind of control over their actions and internal state'; social ability, 'agents interact with other agents (and possibly humans) via some kind of agent-communication language'; reactivity, agents 'perceive their environment ... and respond in a timely fashion to changes that occur in it'; and pro-activeness, 'agents do not simply act in response to their environment, they are able to exhibit goal-directed behaviour by taking the initiative'. A stronger notion adds mentalistic concepts such as knowledge, belief, intention and obligation. The paper opens by recording that the term 'defies attempts to produce a single universally accepted definition' and warns that 'unless the issue is discussed, agent might become a noise term, subject to both abuse and misuse'.
- What it supports
- That the definitional problem is thirty-one years old rather than new, and that the most durable answer to it is a property list agreed before any of the current technology existed.
- What it does not support
- Anything about large language models, which postdate it by a quarter of a century. The four properties are a proposal by two authors in a survey article; they carry no standing beyond the field's long agreement with them, and nothing in the paper measures whether a system holding all four behaves better than one holding three.
Peer-reviewed#
Helmreich, R. L., Merritt, A. C. and Wilhelm, J. A. (1999). The Evolution of Crew Resource Management Training in Commercial Aviation
International Journal of Aviation Psychology, 9(1)
- Method
- Historical and evaluative review of CRM programmes from 1979 onwards.
- Finding
- CRM began with a 1979 NASA workshop prompted by an NTSB finding that a captain had failed to accept input from junior crew. Line audits show CRM produces the intended behavioural change, though measured attitudes decay over time even with recurrent training.
- What it supports
- That an industry built a training response to a named human-factors failure and measured behaviour rather than opinion.
- What it does not support
- That CRM reduces accidents. The authors state directly that accident rates are too rare to serve as a validation criterion.
Peer-reviewed#
Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E. and Nelson, C. (2000). Clinical versus mechanical prediction: a meta-analysis
Psychological Assessment, 12(1), 19-30
- Method
- Meta-analysis of 136 studies comparing clinical (expert, in-the-head) with mechanical (formula, consistently applied) prediction of human health and behaviour.
- Finding
- Mechanical prediction was superior in roughly half the comparisons, approximately equal in roughly half, and inferior in a small minority, with an average advantage of about ten per cent. The modal result is a tie.
- What it supports
- That consistency, rather than knowledge, is what a formula contributes; and that the advantage is real, directional and modest.
- What it does not support
- That algorithms beat experts, which is how it is usually quoted. It says nothing about a human reviewing a machine's output, and nothing about stochastic generative systems, which are neither transparent nor reproducible in the way an actuarial rule is.
Peer-reviewed#
Dekker, S. W. A. and Woods, D. D. (2002). MABA-MABA or Abracadabra? Progress on Human-Automation Co-ordination
Cognition, Technology & Work, 4(4), 240-244
- Method
- Conceptual analysis of function allocation methods in human factors. No data, no participants, no experiment.
- Finding
- Substitution-based function allocation, of which the Fitts list is the archetype, cannot deliver human-automation coordination, because the effects of automation are qualitative rather than quantitative. The authors name the underlying assumption the substitution myth and write that capitalising on a strength of automation does not replace a human weakness but creates new human strengths and weaknesses, often in unanticipated ways. Allocating a function also creates new functions for the other partner that did not exist before. They add that neither the list nor much of the supervisory control literature explains the cognitive work involved in deciding how and when to intervene or how to switch from level to level.
- What it supports
- That splitting the tasks is the wrong unit of design, and that the coordination at the boundary has been the acknowledged gap in this literature since 2002.
- What it does not support
- How large the effect is, or anything measurable at all. It is an argument. Its supporting accident examples are cited rather than analysed.
Peer-reviewed#
Lee, J. D. and See, K. A. (2004). Trust in automation: designing for appropriate reliance
Human Factors, 46(1), 50-80
- Method
- Theoretical review synthesising the trust and automation literature into a model of trust, reliance and calibration.
- Finding
- Trust is an attitude, reliance is the behaviour it produces, and appropriate reliance requires trust calibrated to the system's actual capability. Miscalibration runs in both directions: disuse and misuse.
- What it supports
- The vocabulary this territory now runs on, and the reason a policy naming a human reviewer is not yet a control.
- What it does not support
- Any intervention that reliably produces calibration. It is a synthesis rather than a trial, and the interventions tried since have mixed or null results.
Peer-reviewed#
Dietvorst, B. J., Simmons, J. P. and Massey, C. (2015). Algorithm Aversion: People Erroneously Avoid Algorithms After Seeing Them Err
Journal of Experimental Psychology: General, 144(1)
- Method
- Five experiments.
- Finding
- After seeing an algorithm err, people abandon it even when it demonstrably outperforms them.
- What it supports
- That trust in a model moves for reasons unrelated to its accuracy.
- What it does not support
- That this holds for conversational AI, which is far more recent and feels different to use.
Compiled review#
Strauch, B. (2018). Ironies of Automation: Still Unresolved After All These Years
IEEE Transactions on Human-Machine Systems, 48(5), 419-433. DOI 10.1109/THMS.2017.2732506. Manuscript received 4 January 2017, revised 1 May 2017, accepted 10 June 2017. Read at source 16 September 2026 in IEEE's accepted-article proof, which carries the full text and not the final pagination
- Method
- A narrative retrospective on Bainbridge (1983), setting her ironies against statutory accident investigations in aviation, rail, shipping and pipelines, plus a Google Scholar citation count. Cases are selected because they illustrate the ironies, so there is no sampling frame and no denominator. Reports no new data.
- Finding
- Google Scholar listed 1800 works citing Ironies of Automation as of early November 2016, against 564 for Wiener and Curry (1980) and 488 for Norman (1990), with ten further citing works appearing in a fortnight. Strauch reproduces Bainbridge's ironies in sequence with page citations and argues they are unresolvable rather than merely unresolved while systems can fail and human involvement is required. He names three later ironies: automation can disguise a shortcoming an operator already had; a minor anomaly can become catastrophic through the operator's attempt to resolve it with the same automation; and a person can become qualified to operate an automated system without the expertise to understand it.
- What it supports
- That the 1983 argument has been carried into the present by a reviewed source that engages with its evidence. For anyone without Elsevier access it is also the traceable route to Bainbridge's own words, recording at p. 775 and p. 776 the exact wording this research quotes.
- What it does not support
- Any rate. The accident narratives show the mechanism occurring and cannot show how often it occurs, and Strauch concedes in his own text that the fall in fatal US commercial jet accidents may be a short-term statistical aberration. His restatement that 'research demonstrating that visual monitoring quality deteriorates after about 30 min' carries NO CITATION, and Mackworth's name appears nowhere in the paper, so it must not be cited as a source for the half-hour figure. See the vigilance decrement page.
Argued perspective#
Elish, M. C. (2019). Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction
Engaging Science, Technology, and Society, 5 (2019), 40-60, published 23 March 2019. DOI 10.17351/ests2019.260. Data and Society Research Institute. Open access under CC BY-NC-ND. Read at source 18 September 2026, both the journal's own PDF and its article record, which classifies the piece under its own DC.Type as comparative textual analysis. Graded on this date because the research has argued from this concept on several pages for months without the paper ever entering the evidence base, which qa_citations cannot catch: it checks that every cited URL has an entry and never that a load-bearing concept has one. Not the term's first appearance: the argument is stated earlier in Elish and Hwang (2015) for Data and Society, which is co-authored, so the term is Elish's and 2019 is the citable statement of it.
- Method
- Comparative textual analysis of several high-profile accidents involving complex automated socio-technical systems, together with the media coverage that followed. No new data is generated.
- Finding
- Introduces the moral crumple zone, in the author's words, to describe how responsibility for an action <q>may be misattributed to a human actor who had limited control over the behavior of an automated or autonomous system</q>. The analogy is load-bearing and asymmetric: a car's crumple zone protects the driver, whereas <q>the moral crumple zone protects the integrity of the technological system, at the expense of the nearest human operator</q>.
- What it supports
- That the pattern is describable and has recurred across named, documented cases, and it gives the research the vocabulary for what happens to a person placed in an oversight role they cannot discharge. The concept is now standard in AI accountability writing.
- What it does not support
- Any rate. It is a concept paper working from selected accidents and their press coverage, chosen because they exhibit the pattern, so it establishes that the arrangement occurs and never how often. It also measures nothing about the operators themselves.
Practitioner framework#
Kahneman, D., Lovallo, D. and Sibony, O. (2019). A structured approach to strategic decisions
MIT Sloan Management Review, 60(3)
- Method
- Practitioner framework: decompose a decision into independent mediating assessments, score each separately, and delay the overall judgement until the end.
- Finding
- The authors argue that the structure that improves hiring interviews, independent assessment before global judgement, transfers to strategic decisions.
- What it supports
- A usable protocol, and the clearest statement of why an overall judgement made early contaminates everything after it.
- What it does not support
- Measured improvement on strategic decisions. The supporting evidence is the structured interview literature; the transfer is argued by analogy and has not been trialled.
Peer-reviewed#
Logg, J. M., Minson, J. A. and Moore, D. A. (2019). Algorithm Appreciation: People Prefer Algorithmic to Human Judgment
Organizational Behavior and Human Decision Processes, 151, 90-103, 2019. DOI 10.1016/j.obhdp.2018.12.005. Received 20 April 2018, accepted 4 December 2018. Harvard Kennedy School and Haas School of Business, Berkeley. Read in full at source on 17 September 2026 in the first author's own hosted copy, which carries the journal pagination, the DOI and both dates.
- Method
- Advice-taking experiments measuring Weight on Advice, the distance a participant moves from their first estimate towards advice, divided by the distance between the two. The advice is identical across conditions and only the source label changes. The abstract says six experiments; SEVEN numbered studies are reported (1A, 1B, 1C, 1D, 2, 3, 4) plus two further samples, an MBA definition sample of 77 and a benchmark study of 671. Ns: 1A 202, 1B 215, 1C 286 (MTurk), 1D 119 researchers, 2 154 university participants, 3 403, 4 301 MTurk plus 70 US national security professionals. Sample sizes set a priori by power analysis; 1B, 1C, 1D, 2, 3 and 4 pre-registered, 1A not, which the authors state; pre-registrations, materials and data on the Open Science Framework at https://osf.io/b4mk5/. A PRINTED INCONSISTENCY: the summary table gives study 1D as N=199 while the method text gives 119 from 120 completed surveys, and every 1D statistic carries df of 117 or 118, so 119 is the figure the analysis supports.
- Finding
- Identical advice was weighted more heavily under an algorithmic label. 1A: 0.45 against 0.30, F(1,200)=8.86, p=.003, d=0.42. 1C: 0.38 against 0.26, t(284)=3.50, p=.001, d=0.44. 1B on chart positions, beta=-0.34, t(214)=5.39, p<.001. Study 2: 0.50 against 0.35 seen separately, and 75 per cent chose the algorithm when both were offered. Study 3, using Dietvorst's own materials, 88 per cent chose the algorithm over another participant and 66 per cent over their own estimate, z=6.62, p<.001, so the effect shrinks when the rival is the self. Study 1D: academic researchers predicted the opposite result, mean 0.14 in the aversion direction against participants' observed appreciation, t(118)=14.03, p<.001, d=1.25, with graduate students no better calibrated than senior researchers. Study 4: 70 national security professionals discounted all advice more than a lay sample, interaction F(1,338)=5.05, p=.025, and were LESS accurate for it on Brier scores, interaction F(1,366)=4.16, p=.042. The authors' own words: algorithmic advice 'falls on deaf expert ears, with a cost to their accuracy'.
- What it supports
- Together with Dietvorst, that miscalibration runs in both directions and cannot be fixed by telling people to use judgement. Separately, that a published literature had been read as saying the opposite of what people do, and that the researchers who knew it best predicted wrongly by more than a standard deviation. That is the research's clearest instance of a field's consensus about behaviour failing against a direct test of the behaviour.
- What it does not support
- Which tendency dominates in any given workplace, and three further limits the authors state themselves. The expert result rests on 61 experts in the main analysis, 67 of the 70 were men, and the authors say the two samples 'likely differ in many aspects beyond just expertise in forecasting'. The human comparison is always a peer or an average of peers and never a human expert, which they say may be preferred strongly enough to swamp any appreciation of algorithms. And it does NOT show that people over-rely: in the benchmark study of 671, where the algorithm averaged 314 people and the correct weight was 100 per cent, participants fell short by a mean of 0.66, F(1,669)=275.08, p<.001, d=1.30, so BOTH labels were underweighted. The word 'algorithm' also meant something else in 2018: asked to define it, participants said mathematics, an equation or a calculation 42 per cent of the time and a step-by-step procedure 26 per cent. Nothing here tests a conversational model.
Practitioner framework#
Shrestha, Y. R., Ben-Menahem, S. M. and von Krogh, G. (2019). Organizational decision-making structures in the age of artificial intelligence
California Management Review, 61(4), 66-83
- Method
- Framework proposed in an edited management venue, setting out three structures for combining human and machine decision-making and five criteria for choosing between them.
- Finding
- Proposes full delegation to the machine, hybrid sequential structures in either order, and aggregated human-machine decisions, chosen on the specificity of the decision space, the interpretability of process and outcome, the size of the alternative set, decision speed, and replicability.
- What it supports
- That the choice of structure can be made on stated criteria rather than on enthusiasm. It is the most rigorous published criteria set at the level of an organisational decision.
- What it does not support
- That the criteria have been tested, or that they help the individual deciding what to do with a specific piece of work. No data is reported.
Peer-reviewed#
Zhang, Y., Liao, Q. V. and Bellamy, R. K. E. (2020). Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making
Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT* 20)
- Method
- Two online experiments on an income-prediction task from US census data, 40 trials each. Experiment 1: 72 Mechanical Turk participants, nine per cell in a two by two by two design. Experiment 2: nine further participants, analysed against two Experiment 1 cells for a total of 27. Human unaided accuracy 65 per cent, model accuracy 75 per cent.
- Finding
- Showing confidence scores significantly increased trust, F(1,64)=4.64, p=.035, and significantly improved trust calibration when model confidence was above 80 per cent, F(4,256)=15.8, p<.001. There was no significant difference in AI-assisted accuracy across the prediction and confidence conditions, a result the authors report as rejecting their own hypothesis. Local explanations produced no significant change against baseline on switching behaviour or accuracy, with a reverse trend on accuracy.
- What it supports
- That a confidence score moves how a person feels about a system more reliably than it moves what they catch, and that explanation alone did neither in this setting.
- What it does not support
- Much on its own. Nine participants per cell in Experiment 1 and 27 in Experiment 2, no effect sizes, confidence intervals or standard deviations reported, no exclusions or attention checks described, non-expert participants, and a contrived task carrying no responsibility. The authors also note the approach depends on the model's probabilities being well calibrated in the first place, which the calibration literature says they usually are not.
Peer-reviewed#
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T. and Weld, D. (2021). Does the whole exceed its parts? The effect of AI explanations on complementary team performance
CHI Conference on Human Factors in Computing Systems 2021
- Method
- Controlled experiments on human-AI team performance across tasks, comparing conditions with and without explanations of the model's output.
- Finding
- Explanations increased the rate at which people accepted the model's recommendation without improving the accuracy of the human-AI team. Acceptance rose for correct and incorrect outputs alike.
- What it supports
- That explanation is not a mechanism for calibrated reliance, and that the intuitive fix for oversight can make the measured outcome worse.
- What it does not support
- That explanations are worthless. They serve contestability, auditability and legal duties, which this study does not measure.
Peer-reviewed#
Bucinca, Z., Malaya, M. B. and Gajos, K. Z. (2021). To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making
Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), April 2021. DOI 10.1145/3449287
- Method
- Experiment with 199 participants comparing three cognitive forcing interventions, designed from dual-process theory to compel more thoughtful engagement with AI-generated explanations, against two simple explainable-AI approaches and a no-AI baseline. Includes an audit for intervention-generated inequalities using the Need for Cognition scale.
- Finding
- Cognitive forcing significantly reduced overreliance compared with the simple explainable-AI approaches. Participants gave the least favourable subjective ratings to the designs that reduced overreliance the most. On average the interventions benefited participants higher in Need for Cognition more, so human cognitive motivation moderates the effectiveness of explainable AI. The authors argue people rarely engage analytically with each individual recommendation and instead develop general heuristics about when to follow the AI.
- What it supports
- That deliberately adding friction to an AI-assisted decision measurably reduces acceptance of wrong suggestions, and that the intervention people dislike most is the one that works best.
- What it does not support
- That slower decisions produce better organisational outcomes. This is a controlled task with 199 participants, not a field study, and it measures overreliance rather than downstream results. The uneven benefit by Need for Cognition means the effect will not be uniform across a workforce.
Working paper#
Dell'Acqua, F. (2022). Falling Asleep at the Wheel: Human/AI Collaboration in a Field Experiment on HR Recruiters
Working paper, Laboratory for Innovation Science at Harvard, 2022. No journal publication found. GRADED FROM THE ABSTRACT PAGE (innovationgrowthlab.org/node/1447) AND THE HARVARD BUSINESS SCHOOL AI INSTITUTE'S SUMMARY (aiinstitute.hbs.edu/is-ai-making-your-team-lazy), read on 2 November 2026. The full text is hosted on a page that refused the fetcher, so it was not read
- Method
- Preregistered field experiment per the abstract. 181 professional recruiters screened 44 résumés for a software engineering role. Per the Harvard Business School AI Institute's summary they were randomly assigned to no AI or to AI recommendations of three quality levels, described as about 99, 85 and 75 per cent accurate. A formal model of effort and AI quality accompanies it. Sample size and design come from the abstract page and the institute summary; allocation procedure, effect sizes and the author's own limitations were not read.
- Finding
- Recruiters with the higher-quality AI were less accurate than those with the lower-quality AI. Those with the lower-quality AI worked harder, spent more time on résumés, were less likely to accept the AI's pick automatically, and improved over time, with the gains concentrated among more experienced recruiters. The abstract concludes that the best combined performance may sometimes call for lower-quality AI, depending on human effort, learning and skill.
- What it supports
- That in one screening task effort and accuracy moved with the quality of the AI recommendation, in the direction the effort model predicts.
- What it does not support
- That it generalises beyond résumé screening, to people accountable for the outcome, or beyond a single session. No effect sizes are reported here because the full text was not read. It does not support deploying weaker AI: the Harvard summary itself says the aim is to keep people attentive. A working paper from 2022 with no journal version found.
Working paper#
Dell'Acqua, F. et al. (2023). Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality
Harvard Business School and BCG working paper
- Method
- Field experiment, 758 BCG consultants, tasks inside and just outside GPT-4's competence.
- Finding
- Inside the frontier, AI-assisted consultants were dramatically better and faster. Outside it, they performed worse than consultants with no AI at all.
- What it supports
- That model competence is jagged rather than smooth, and that confident output suppresses scrutiny at exactly the wrong moment.
- What it does not support
- Where the frontier runs in your domain. That is local and must be learned.
Peer-reviewed#
Noy, S. and Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence
Science, 381(6654), 187-192, 13 July 2023. DOI 10.1126/science.adh2586. Preregistered, AEA RCT Registry trial 10882
- Method
- Preregistered online randomised experiment. 453 college-educated professionals, including marketers, grant writers, consultants, data analysts, HR professionals and managers, given occupation-specific incentivised writing tasks of 20 to 30 minutes. Half were randomly exposed to ChatGPT. Run 27 January to 21 February 2023 with GPT-3.5.
- Finding
- Average time taken fell by 40 per cent and output quality rose by 18 per cent. Time on the post-treatment task dropped by 11 minutes, 0.75 standard deviations, against a control mean of 27 minutes, and evaluator grades rose by 0.45 standard deviations. Inequality between workers decreased: in the treatment group initial inequalities were more than half-erased, with the correlation between first-task and second-task grades falling to 0.14.
- What it supports
- That the tool compresses the performance distribution on tasks it does well, raising the floor much more than the ceiling. That is a pricing fact about expertise before it is a productivity fact.
- What it does not support
- That the effect generalises. The authors say they examined a limited range of occupations and tasks in which ChatGPT may be unusually useful, and speculate that real-economy effects will be somewhat lower. Short one-off tasks, online sample, an early model. Nothing about capability retention.
Working paper#
Peng, S., Kalliamvakou, E., Cihon, P. and Demirer, M. (2023). The Impact of AI on Developer Productivity: Evidence from GitHub Copilot
arXiv:2302.06590, 13 February 2023. DOI 10.48550/arXiv.2302.06590. Authors at Microsoft Research, GitHub and MIT Sloan. Not peer reviewed
- Method
- Randomised controlled trial. 95 professional programmers recruited through Upwork from 166 offers, randomised to 45 treated and 50 control, given a standardised task of implementing an HTTP server in JavaScript. 35 in each group completed the task and survey.
- Finding
- Conditional on completion, the treated group averaged 71.17 minutes against 160.89 for control, a 55.8 per cent reduction in completion time, p = 0.0017, with a 95 per cent confidence interval on the improvement of 21 to 89 per cent. Participants in both groups estimated a 35 per cent productivity increase, which the authors describe as an underestimation of the 55.8 per cent revealed increase.
- What it supports
- A large measured speed gain on a well-specified task, and, separately, that self-reported productivity can UNDERSTATE the measured effect. Any claim that self-report systematically overstates gains has to answer this result.
- What it does not support
- Anything about quality. The authors state the study does not examine the effects of AI on code quality. The effect is estimated on 35 completers per arm rather than 95 participants, the task is standardised and greenfield rather than work inside a mature codebase, the authors are employed by the vendor and its parent. Not peer reviewed.
Peer-reviewed#
Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B. and Yang, Q. (2023). Why Johnny Can't Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts
CHI 2023
- Method
- Design probe study with non-experts using a purpose-built prompt design tool.
- Finding
- Non-experts approached prompting opportunistically rather than systematically, over-generalised from single successes and failures, and struggled to form an accurate model of the system.
- What it supports
- That prompting is genuinely harder than it looks, which is the strongest case FOR teaching it.
- What it does not support
- That this persists. It used 2023 models, and providers are actively engineering the difficulty away.
Practitioner method#
Anthropic (Schluntz, E. and Zhang, B.) (2024). Building effective agents
Anthropic Engineering, published 19 December 2024, date read from the page itself. The page H1 reads 'Building effective agents' and its HTML title reads 'Building Effective AI Agents'; the H1 is cited. The live page now carries a note added after publication saying much of the tooling landscape it describes has changed, and the definitional passage is unaltered.
- Method
- An engineering practice note setting out implementation patterns and one architectural distinction. No study, no sample, no measurement.
- Finding
- Draws the line most widely used in practice: 'Workflows are systems where LLMs and tools are orchestrated through predefined code paths', while 'Agents, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks'. Both sit under the heading agentic systems. The post opens by conceding the problem in its own words: 'Agent can be defined in several ways.'
- What it supports
- That the operationally useful line runs through who chooses the next step, and that a firm building these systems draws it there.
- What it does not support
- That either pattern performs better, and nothing about any product. It is a taxonomy published by a company that sells the systems being taxonomised, with no data attached. Useful because it is operational and widely adopted, not because it is authoritative.
Peer-reviewed#
Kim, S. S. Y., Liao, Q. V., Vorvoreanu, M., Ballard, S. and Wortman Vaughan, J. (2024). I'm Not Sure, But...: Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust
Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT 24). Pre-registered at osf.io/mnrp9
- Method
- Pre-registered between-subjects experiment, four conditions, eight yes-or-no medical questions. 656 responses collected, 252 excluded on pre-registered criteria, final sample 404. Responses pre-generated and presented as a fictional system whose answers were correct on exactly half the questions. Conditions varied only the presence and perspective of uncertainty expression.
- Finding
- Access to the system raised agreement to 80.9 per cent against 58.4 per cent without, and lowered accuracy to 63.9 per cent against 74.2 per cent without. First-person uncertainty expression significantly reduced agreement to 74.8 per cent and significantly raised accuracy to 72.8 per cent. Impersonal expression moved both in the same direction without reaching significance. Intention to use fell significantly under first-person hedging, 2.91 against 3.25 in control and 3.36 for the impersonal version. Hedging reduced accuracy slightly when the system was correct and raised it more when the system was wrong.
- What it supports
- That hedging changes reliance in the direction that helps, that the perspective of the hedge matters, and that the version which helps most is the version users least want to keep using.
- What it does not support
- That uncertainty expression should be mandated. The authors state directly that regulators should avoid blanket requirements until more research is done, and flag that their system had low accuracy and expressed uncertainty often in a poorly calibrated manner. It did not eliminate over-reliance: participants without AI access still performed best. Reported as model-estimated means with significance stars, no test statistics, exact p values or effect sizes, and 38.4 per cent of collected responses were excluded.
Peer-reviewed#
Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R. and Gal, Y. (2024). AI models collapse when trained on recursively generated data
Nature, 631, 755-759, published 24 July 2024. DOI 10.1038/s41586-024-07566-y. An Author Correction was published 21 March 2025 (DOI 10.1038/s41586-025-08905-3), fixing a single Greek-letter notation error in the theoretical section; no finding or figure changed. The term Model Collapse was introduced in the authors' own earlier preprint, The Curse of Recursion: Training on Generated Data Makes Models Forget, arXiv:2305.17493, submitted 27 May 2023, read at source.
- Method
- Controlled experiments training language models, variational autoencoders and Gaussian mixture models recursively on data generated by earlier versions of themselves, across successive generations, plus a theoretical analysis of why the effect must occur.
- Finding
- Indiscriminate training on model-generated content causes 'irreversible defects' in the resulting models: the tails of the original data distribution, meaning rare or unusual examples, disappear first, and generations converge on an increasingly narrow, generic approximation of the average. The authors demonstrate the effect across three different kinds of generative model, which they take as evidence that it is a general property of learning from recursively generated data rather than a quirk of language models specifically.
- What it supports
- That a model trained on its own kind's output degrades in a specific and measured direction, losing diversity before losing coherence, and that access to genuine, human-generated data becomes more rather than less valuable as AI-generated content accumulates online.
- What it does not support
- Anything about human cognition, and nothing about how much of today's open web is already AI-written or how far any real production model has moved towards collapse. It is a technical finding about training pipelines, not a measurement of the internet's current state, and is not to be confused with knowledge collapse, a separate and later argument about human societies rather than about models.
Peer-reviewed#
Vaccaro, M., Almaatouq, A. and Malone, T. (2024). When combinations of humans and AI are useful: a systematic review and meta-analysis
Nature Human Behaviour, 8, 2293-2303
- Method
- Preregistered systematic review and meta-analysis: 106 experimental studies, 370 effect sizes, published January 2020 to June 2023.
- Finding
- Human-AI combinations performed significantly WORSE on average than the better of human or AI alone (Hedges' g = -0.23). Losses concentrated in decision-making; gains in content creation. Pairing gained where humans beat the AI and lost where the AI beat humans.
- What it supports
- That adding a human is not a control, and that undesigned pairing can subtract. The most under-absorbed result in the field.
- What it does not support
- That human-AI teams are useless. The benchmark is an oracle-selected best performer, which you rarely know in advance. Also predates current frontier models.
Peer-reviewed#
Xiong, M., Hu, Z., Lu, X., Li, Y., Fu, J., He, J. and Hooi, B. (2024). Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs
ICLR 2024. arXiv:2306.13063
- Method
- Systematic evaluation of confidence elicitation across five models, Vicuna 13B, GPT-3, GPT-3.5-turbo, GPT-4 and LLaMA 2 70B, on eight datasets spanning arithmetic, commonsense, symbolic and professional reasoning. Metrics: expected calibration error, AUROC and AUPRC. No human subjects.
- Finding
- Average expected calibration error for plain verbalised confidence is 0.520 for GPT-3, 0.461 for Vicuna, 0.436 for LLaMA 2, 0.377 for GPT-3.5 and 0.180 for GPT-4. GPT-4's average AUROC is 62.7 per cent against a 50 per cent chance baseline. Stated confidences cluster in the 80 to 100 per cent range in multiples of five, which the authors suggest means models may be imitating human expressions when verbalising confidence.
- What it supports
- That a number a model gives when asked how sure it is is close to unusable as a calibrated quantity, and that its shape suggests it is generated as plausible text rather than measured.
- What it does not support
- Anything about how users read these numbers, since no humans were involved. Mitigation strategies reduce ECE substantially, to 0.028 in the best case, while still failing to predict incorrect answers on knowledge-heavy tasks. Models are of the GPT-4 and LLaMA 2 generation.
Peer-reviewed#
Zhou, K., Hwang, J. D., Ren, X. and Sap, M. (2024). Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertainty
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), Long Papers, 3623-3643. Also arXiv:2401.06730
- Method
- Model study: nine models prompted with 49 prompts over 284 MMLU questions, 125,244 queries. Human study: Prolific participants across a control setting and three interactive settings with calibrated, overconfident and underconfident systems, 25 recruited per setting.
- Finding
- Only about 5 per cent of generated answers include any epistemic marker. Among confidently expressed responses the error rate averages 47 per cent, and only 53 per cent of generations expressing certainty are correct. In the human study, hedged answers were relied on around 10 per cent of the time and confident ones around 90, but plain unmarked statements were also relied on nearly 90 per cent of the time, which the authors read as users interpreting the absence of a marker as certainty. Exposure to an overconfident system left mental models uncorrected, with participants averaging 76 per cent during miscalibrated rounds and 86 afterwards, while an underconfident system produced 66 and then 98. Reward modelling scores plain statements at 4.03, expressions of certainty at 0.82 and expressions of doubt at minus 1.86.
- What it supports
- That silence about uncertainty is read as confidence rather than as neutrality, and that the confident register is a product of a preference model that penalises hedging more than it rewards assurance.
- What it does not support
- Any of it with statistical rigour on the human side. No total human N is stated in the text, and no p values, confidence intervals or effect sizes are reported for any human result. Twenty-five participants per setting. US-only, which the authors themselves call a narrow and US-centric view. Body text and Table 1 disagree slightly on the marker rate, 5 against 6 per cent. Peer-reviewed venue corrected to ACL 2024 on 4 September 2026 after an earlier draft of this entry read the arXiv preprint as unpublished.
Working paper#
Becker, J., Rush, N., Barnes, E. and Rein, D. (METR) (2025). Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
METR, 10 July 2025; arXiv:2507.09089, v2 25 July 2025. No journal reference as at 9 September 2026.
- Method
- Randomised controlled trial. 16 experienced open-source developers drawn from repositories averaging over 22,000 stars and a million lines of code, 246 real issues from their own projects, each randomly assigned to permit or prohibit AI tools. Tasks averaged about two hours. Screens recorded, implementation time self-reported, developers paid 150 dollars an hour. Tooling was mostly Cursor Pro with Claude 3.5 and 3.7 Sonnet. Twenty candidate explanations for the slowdown were tested and five judged contributory.
- Finding
- Developers were measured as 19 percent SLOWER when permitted to use AI tools. They had forecast a 24 percent speed-up beforehand, and after completing the tasks and experiencing the slowdown, still estimated AI had made them about 20 percent faster.
- What it supports
- That self-reported productivity gain is an unreliable measure of actual productivity gain, and that the error can run in the opposite direction to the truth by a wide margin.
- What it does not support
- That AI slows all developers or all software work, and NOT the current position. Sixteen participants, all experienced, all working on large mature codebases they knew well, using early-2025 tooling. METR themselves withdrew this as a current signal on 24 February 2026: see metr-2026-update. The durable finding is the perception gap, not the 19 per cent.
Peer-reviewed#
Cemri, M., Pan, M. Z., Yang, S., Agrawal, L. A., Chopra, B., Tiwari, R., Keutzer, K., Parameswaran, A., Klein, D., Ramchandran, K., Zaharia, M., Gonzalez, J. E. and Stoica, I. (2025). Why Do Multi-Agent LLM Systems Fail?
NeurIPS 2025 Datasets and Benchmarks Track. arXiv:2503.13657
- Method
- Empirical failure taxonomy built from expert annotation of 150 execution traces at inter-annotator kappa 0.88, then applied at scale with LLM-as-judge across more than 1,600 traces from seven multi-agent frameworks. Failure distribution computed on 210 traces.
- Finding
- Failures divide into system design issues 41.8 per cent, inter-agent misalignment 36.9 per cent and task verification 21.3 per cent. Inter-agent misalignment is defined as a breakdown in critical information flow during interaction and coordination, and comprises conversation resets 2.20 per cent, proceeding on wrong assumptions rather than seeking clarification 6.80 per cent, task derailment 7.40 per cent, withholding information another agent needed 0.85 per cent, ignoring another agent's input 1.90 per cent, and mismatch between reasoning and action 13.2 per cent. The authors state that context and communication protocols are often insufficient, because the errors occur even when agents in the same framework communicate in natural language. Improving role specification alone raised ChatDev success by 9.4 percentage points.
- What it supports
- That more than a third of observed multi-step agent failure sits at the joins rather than in any single agent's competence, and that standardising the message format does not fix it.
- What it does not support
- Anything about human-agent boundaries. Every handoff in the dataset is agent to agent and no humans appear anywhere. The authors caveat that the 210-trace distribution illustrates system-specific profiles rather than comparing performance across systems, so the percentages are not cross-benchmark comparable.
Peer-reviewed#
Dell'Acqua, F., Ayoubi, C., Lifshitz, H., Sadun, R., Mollick, E., Mollick, L., Han, Y., Goldman, J., Nair, H., Taub, S. and Lakhani, K. (2025). The Cybernetic Teammate: A Field Experiment on Generative AI Reshaping Teamwork and Expertise
NBER Working Paper 33641, April 2025. Published as 'The Cybernetic Teammate: A Field Experiment on Generative AI and Teamwork', Organization Science, June 2026. DOI 10.1287/orsc.2025.20702
- Method
- Pre-registered field experiment with 776 professionals at Procter and Gamble working on real product innovation challenges, randomised both on AI access and on working individually or in a two-person new product development team.
- Finding
- Individuals working with AI matched the performance of two-person teams working without it. AI use removed the functional split in proposals: without AI, research and development professionals proposed more technical solutions and commercial professionals more commercially oriented ones, while professionals using AI produced balanced solutions regardless of background. Participants using AI reported more positive emotional responses.
- What it supports
- That a model can substitute for measurable parts of what a second human teammate contributes, including some of the social and motivational function, and that it flattens the differences in output that come from professional training.
- What it does not support
- Client or business outcomes, which were not measured, or any effect over time. One firm, one task type, a single session. Procter and Gamble provided financial support to the institute involved and one author had consulted for the firm, both disclosed in the paper. That the flattened output is better rather than merely more balanced is not established.
Working paper#
Kwa, T. and Cheng, V. (METR) (2025). How Does Time Horizon Vary Across Domains?
METR blog, 14 July 2025. Not peer-reviewed. Read at source 3 October 2026
- Method
- Time-horizon analysis applied to nine benchmarks across several domains, including software, computer use, mathematics contests, scientific question answering, video understanding and autonomous driving, using human-time estimates that differ by benchmark.
- Finding
- METR's summary reports generally similar rates of improvement to the seven-month doubling of the original work. The authors state that economically valuable tasks are much more diverse than the domains examined, that the key skills of management and nursing are mostly unrepresented, and that all benchmarks are expected to be easier than real-world tasks.
- What it supports
- That the rapid-improvement pattern is not confined to software benchmarks, within the benchmarks tested.
- What it does not support
- Any rate of improvement for unmeasured work. The authors call individual benchmark results fairly noisy, note that many benchmarks lack human baselines so task lengths are estimated, and say some horizons are extrapolations beyond the task lengths available. Per-domain figures were NOT used on any page because the fetched summary could not be checked against the post's own table.
Peer-reviewed#
Kwa, T., West, B., Becker, J., Deng, A., Garcia, K., Hasin, M., Jawhar, S., Kinniment, M., Rush, N., Von Arx, S., Bloom, R., Broadley, T., Du, H., Goodrich, B., Jurkovic, N., Miles, L. H., Nix, S., Lin, T., Painter, C., Parikh, N., Rein, D., Sato, L. J. K., Wijk, H., Ziegler, D. M., Barnes, E. and Chan, L. (METR) (2025). Measuring AI Ability to Complete Long Software Tasks
NeurIPS 2025; arXiv:2503.14499, v1 18 March 2025, v4 10 July 2026. Venue taken from arXiv's journal-reference field on v4; the proceedings text was not read.
- Method
- A benchmark metric fitted by logistic regression. 170 tasks in three suites: 97 software tasks from HCAST running 1 minute to 30 hours, 7 machine-learning research engineering tasks from RE-Bench all 8 hours, and 66 software atomic actions of 1 to 30 seconds. Task length is the geometric mean time of SUCCESSFUL human baselines. Over 800 baselines totalling 2,529 hours; 148 of 169 tasks have a human baseline and 21 carry a researcher estimate. Baseliners were paid professionals with about five years of experience, recorded, and bonused for speed. 12 frontier and 4 near-frontier models, 8 runs per model-task pair. The paper is internally inconsistent on its own counts, giving 170 tasks in the abstract and figure captions and 169 in the method, and 12 models in one place and 11 in another.
- Finding
- The 50%-task-completion time horizon, defined as the human-expert task length a model completes with about 50 per cent success, doubled every 207 days from 2019 to 2025 (95 per cent bootstrapped CI 166-240 days). GPT-2 sits at 2 seconds and o3 at 110 minutes. The 80 per cent horizon doubles on a similar clock at 204 days but is 4 to 6 times SHORTER in level. The authors labelled their own tasks against 16 messiness factors and report a mean of 3.2/16 with none above 8, estimating 9 to 15 for a task like writing a good research paper; each messiness point costs roughly 8.1 percentage points of success. Rate of improvement was the same in the messy and clean halves of the suite.
- What it supports
- That the length of software task a model will attempt at a coin-flip success rate has grown on an exponential measured across six years, and that the slope survives moving the success threshold from 50 to 80 per cent.
- What it does not support
- Reliability, and not general capability. The authors state the tasks are 'systematically different from real tasks' because all are automatically scored, none involves other agents, few constrain a scarce resource, few punish a single mistake and environments are static. They say they 'cannot confidently measure time horizons at very high success rates (e.g. 95%)'. Their confidence in the 2024-25 acceleration is 'low because there are only seven frontier models in this time span', and the 2024-only fit gives three months with the note that 'any extrapolation into the future would not be robust'. They are 'more confident in the slope of the time horizon trend than in the time horizon of any particular model', so a single model's horizon is the weaker half of the paper. Their contract baseliners took 5 to 18 times longer than repository maintainers, so horizons correspond 'to the labor of a low-context human'. Their own footnote: the forecasts concern 'a 1-month horizon on software tasks, not 1-month AGI'. NO ABSOLUTE 80 PER CENT HORIZON IS STATED ANYWHERE IN THE PAPER: only the 4-6x ratio and the 204-day doubling. Any minute figure for an 80 per cent horizon is somebody's arithmetic and must not be attributed to the authors. NOT the same study as metr-2025, the developer trial, and the two must never be cited as one result.
Argued perspective#
Narayanan, A. and Kapoor, S. (2025). AI as Normal Technology
Knight First Amendment Institute at Columbia University, 15 April 2025
- Method
- Long argued essay published by a research institute. Not peer reviewed. The first 100,000 characters of the page were read on 12 October 2026; the final part was not.
- Finding
- Uses the label capability-reliability gap for a pattern the authors say shows up over and over, and says it has been a major barrier to building useful AI agents that can automate real-world tasks. Illustrates the pattern with self-driving cars.
- What it supports
- That the label is in print with this attribution, and what the authors claim about its role in agents.
- What it does not support
- That the gap is measurable or its size. No measurement of it was found in the part read. Whether the authors define it more fully elsewhere in the essay is unchecked.
Practitioner framework#
Sebastian, I. M., Weill, P., Haskamp, T. and vom Brocke, J. (2025). A framework for determining when AI can make decisions
MIT Center for Information Systems Research, reported by MIT Sloan
- Method
- Framework from a research centre, sorting decisions on two axes, ambiguity and risk, into four types with a different human role in each.
- Finding
- Proposes routine, consequential, exploratory and strategic decisions, with the human role moving from monitoring at one corner to owning the decision at the other.
- What it supports
- That a two-axis sort of decisions by ambiguity and risk is the current management-side answer to when a machine may decide.
- What it does not support
- That the four types are exhaustive or that the sort has been validated. It addresses the executive allocating work, not the person doing it, and reports no data.
Peer-reviewed#
Steyvers, M., Tejeda, H., Kumar, A., Belem, C., Karny, S., Hu, X., Mayer, L. W. and Smyth, P. (2025). What large language models know and what people think they know
Nature Machine Intelligence, 7, 221-231
- Method
- Two behavioural experiments, 301 participants recruited through Prolific, each assigned 40 questions from pools of 350 multiple-choice and 336 short-answer items. Model confidence from GPT-3.5, PaLM2 and GPT-4o compared against participant confidence after reading model explanations.
- Finding
- Model confidence discriminates correct from incorrect answers at AUC 0.751 for GPT-3.5, 0.746 for PaLM2 and 0.781 for GPT-4o. Participants reading default explanations reached 0.589, 0.602 and 0.592, which the authors describe as only slightly better than random guessing. They name the shortfalls the calibration gap and the discrimination gap, and attribute human miscalibration primarily to overconfidence, people believing LLMs are more accurate than they are. Longer explanations significantly raised participant confidence without improving discrimination, mean participant AUC 0.54 for long explanations.
- What it supports
- That the reader is a worse judge of an answer's correctness than the model is, on the same items, and that adding explanation length makes readers surer rather than righter.
- What it does not support
- That closing the gap improves task accuracy. The modified-explanation result is a simulation via post-hoc filtering rather than a live deployment. Participants had no domain expertise and their own accuracy was 33 per cent against the model's 39. Statistics are Bayes factors only, with no p values or effect sizes, and the ECE values sit inside a figure rather than in the text.
Institutional survey#
Becker, J. (METR) (2026). Measuring the Self-Reported Impact of Early-2026 AI on Technical Worker Productivity
METR, 11 May 2026
- Method
- Survey of 349 technical workers fielded February to April 2026: 87 software engineers, 71 researchers, 129 academics and PhD students, 48 founders and managers. Convenience sample sourced from GitHub, academic directories, METR and staff networks, and X. Email response rate around 2 per cent. Roughly 70 per cent of participants paid, averaging 200 dollars. Respondents averaged 12 years programming, 19 months using AI for programming and 7 months using agentic coding tools; 50 per cent regularly use Claude Code. Ten respondents removed for inconsistent answers. The design separates value produced from speed, on the argument that speed overstates value when AI changes which tasks a person takes on.
- Finding
- Median self-reported change in the value of work was between 1.4 and 2 times; median self-reported speed change was 3 times. Asked the same question about different years, respondents put themselves at 1.3 times in March 2025, 2 times in March 2026, and forecast 2.5 times for March 2027. METR's own staff gave the lowest value-change answers of any subgroup studied, which the authors suggest reflects those staff having the perception-gap finding in mind. METR restate the size of that gap here: participants in the early-2025 trial overestimated AI's effect on their time by 40 percentage points on average.
- What it supports
- That the value-versus-speed distinction is large and measurable in self-report, with speed running roughly double value on the same respondents. It is also the clearest statement by the authors of the 19 per cent trial of how big the perception error was.
- What it does not support
- Any actual productivity effect. Every number here is self-reported, the sample is a convenience sample with a 2 per cent response rate and heavy selection towards enthusiastic adopters, and METR say so. The staff subgroup result is an observation on a small non-random group and not a designed test of whether knowing about the perception gap changes an answer.
Working paper#
Becker, J. (METR) (2026). Measuring the Self-Reported Impact of Early-2026 AI on Technical Worker Productivity
METR, 11 May 2026. Not peer reviewed
- Method
- Survey of 349 technical workers on self-reported change in speed and in value of work attributable to AI tools, set against the published field-experiment literature.
- Finding
- Median self-reported change in the VALUE of work is 1.4 to 2 times, and median self-reported SPEED change is 3 times. METR state that to their knowledge only one study has gathered survey and field experiment results on the same population and metric, Becker et al. (2025), which finds developers overestimate productivity gains by over 40 percentage points. They add that public survey estimates have tended to exceed field-experiment estimates, while stating it is difficult to determine the extent to which surveys overestimate gains relative to experimental data, and that speed measures are likely biased upwards relative to value measures.
- What it supports
- That the evidence base for the most repeated claim in enterprise AI, that surveys overstate productivity, rests on a single population, and that the people who own that finding say so themselves. It also separates speed from value, which almost no business case does.
- What it does not support
- That self-report always overstates. The GitHub Copilot randomised trial found the opposite direction, with participants estimating 35 per cent against a measured 55.8 per cent. A survey of a self-selected technical population, not peer reviewed, and its own authors decline the general claim.
Working paper#
Becker, J., Rush, N., Cunningham, T., Rein, D. and Mahamud, K. (METR) (2026). We are Changing our Developer Productivity Experiment Design
METR, 24 February 2026
- Method
- Second randomised task-level study, begun August 2025: 57 developers (10 returning from the original study, 47 newly recruited), 143 repositories, more than 800 tasks, paid 50 dollars an hour against 150 in the original. Accompanied by participant surveys and interviews.
- Finding
- METR state the data gives an unreliable signal of the current productivity effect of AI tools, because 30 to 50 per cent of developers reported declining to submit tasks they did not want to do without AI, and an increased share declined to take part at all. Raw results now point the other way: an estimated speedup of -18 per cent for returning developers (CI -38 to +9) and -4 per cent for new recruits (CI -15 to +9), against the original +19 per cent slowdown (CI +2 to +39). They believe developers are likely more sped up in early 2026 than in early 2025, while stating their own data is only very weak evidence for the size of that change.
- What it supports
- That the 19 per cent slowdown belongs to early 2025 and should not be quoted as the current effect. It also demonstrates a measurement problem that will worsen: as adoption rises, the people most helped by AI are the ones most likely to select themselves out of any study that asks them to work without it.
- What it does not support
- That AI now speeds developers up by a specific amount. Every confidence interval here crosses zero, and METR say so. It does not retract the original study, whose perception-gap finding, that participants estimated a 20 per cent speed-up while measured slower, is untouched.
Working paper#
Li, Z., Ye, S., Guo, F. and Dang, Z. (2026). SilentProbe: Measuring Silent Failure in Production APIs Used as Agent Tools
arXiv 2609.00035, 29 August 2026, 12 pages, CC BY 4.0. Read at source 17 September 2026
- Method
- Three studies. An audit of 721,320 parameters across 2,501 independently published OpenAPI documents. 219 schema-derived perturbations executed against live commercial endpoints from 27 vendors, all reached through one aggregation layer. Then twelve models across eight families run on ordinary tasks against those endpoints, and the full agent loop run on the resulting failures. Code, schemas, perturbation sets, agent transcripts and per-call run identifiers are released. Preprint, not peer reviewed, and the endpoints are reached through a single intermediary rather than called directly.
- Finding
- An agent calling a production API cannot tell a query that matched nothing from a query the server did not understand: both return HTTP 200 with a parsable body. Of the documents audited, 7.5 per cent declare an enumeration and 15.2 per cent declare any machine-checkable constraint at all, while 40.1 per cent state at least one constraint in prose that their schema does not encode. Constraint form rather than vendor identity predicted honesty: machine-checkable constraints produced an honest error in 111 of 111 cases and prose-only constraints failed silently in 44 of 61 (p = 2e-13). A vocabulary that the description gave only examples of was missed by every model on 88 of 88 attempts; written out in full it was used correctly 88 to 91 per cent of the time. Running the full loop on the resulting silent failure, models detected it in 12 per cent of cases, repaired it in 0 per cent, asserted a false negative to the user in 41 per cent and invented a figure in 12 per cent. Promoting the vocabulary into the schema took the failure from 88 of 88 to 0 of 89.
- What it supports
- That when a tool fails without saying so, current models mostly do not notice, never repair it, and in a majority of cases tell the user something untrue: either that there is nothing there, or a number they made up. It also shows the fault is in how the interface declares its constraints rather than in model quality, because one schema change removed it.
- What it does not support
- Anything about a human's ability to catch the same failure, which is not tested. The percentages are conditional on this perturbation set, these 27 vendors and one aggregation layer, and the twelve models are not named in the abstract. It is a preprint with no independent replication. The 41 and 12 per cent describe what a model said, not what a person then believed or acted on.
Working paper#
METR (2026). Time Horizon 1.1
METR blog, 29 January 2026. Not peer-reviewed. Read at source 3 October 2026
- Method
- Re-estimate of METR's time-horizon measure on an expanded suite of 228 tasks, up from 170 (73 added, 15 removed, 53 updated), with tasks of eight hours or more rising from 14 to 31, on new evaluation infrastructure.
- Finding
- Doubling time for the full 2019 to 2025 period is 196 days, the same as the first version. Since 2023 it is 131 days against 165, and since 2024 89 days against 109. 50 per cent time horizons: Claude Opus 4.5 320 minutes (interval 170 to 729), GPT-5 214 (117 to 480), o3 121 (74 to 201).
- What it supports
- That on this suite, estimated by its builders, the measured horizon has lengthened with a doubling time of months, and that the recent trend depends on how the suite is composed.
- What it does not support
- Capability on tasks outside the suite, or reliability. METR writes that the trend is somewhat sensitive to task composition, that the confidence intervals are still very wide, and that human baseline times exist for only 5 of the 31 long tasks. Software tasks only.
Vendor research#
Massenkoff, M., Lyubich, E., McCrory, P., Appel, R. and Heller, R. (2026). Anthropic Economic Index report: Learning curves
Anthropic, 24 March 2026. Read at source 2 October 2026
- Method
- Descriptive and regression analysis of a sample of one million conversations from Claude.ai and Anthropic's first-party API, 5 to 12 February 2026, comparing users who signed up at least six months before the data pull (high tenure) with newer users. Success is Claude's own assessment of whether a conversation was successful.
- Finding
- High-tenure users are more likely to iterate on their work and much less likely to delegate through directive use patterns, and are 7 percentage points more likely to use Claude for work. The headline reads a 10% higher success rate; the regression table gives about 5 percentage points bivariately, about 3 with O*NET task and request-cluster fixed effects, and about 4 with full controls for model, language, use case and country.
- What it supports
- That in Anthropic's own logs, longer-serving users work more interactively and score slightly higher on the system's own success measure, after controlling for task and country.
- What it does not support
- That practice caused either. The authors state that high-tenure users are self-selected, that the differences could reflect stable characteristics, and that survivorship bias is present because people who stopped using the product are missing. Success is judged by the model being used, not by an outside check of the work. Tenure is time since sign-up, not expertise in any domain. The usage data and the success measure are the vendor's and cannot be checked from outside.
Vendor research#
Microsoft (WorkLab), with Edelman Data x Intelligence and LinkedIn (2026). 2026 Work Trend Index Annual Report: Agents, human agency, and the opportunity for every organization
Microsoft, May 2026
- Method
- Edelman survey of 20,000 full-time knowledge workers who use AI at work in ten countries (Australia, Brazil, France, Germany, India, Italy, Japan, Netherlands, UK, US), fielded 18 February to 7 April 2026. Combined with analysis of over 100,000 anonymised Microsoft 365 Copilot chats (105,000 samples from one week in February 2026, trend data March 2025 to March 2026), a July 2025 Microsoft People Science survey of 1,800 employees, and LinkedIn labour market data.
- Finding
- Organisational factors (culture, manager support, talent practices) account for 67% of the variance in AI's reported impact, against 32% for individual mindset and behaviour. 49% of analysed Copilot conversations support cognitive work: analysing information, solving problems, evaluating and thinking creatively. The report describes four modes of working with AI (delegation, collaboration, asking, exploration) by human and agent intensity. Agent adoption is broad in software and technology, which accounts for nearly one in five firms using agents, while manufacturing has fewer adopting firms deploying at greater scale. The Transformation Paradox: employees are ready to change how they work but metrics, incentives and norms reinforce the old way.
- What it supports
- That among AI-using knowledge workers, organisational conditions explain more of reported impact than individual attitude, and that most Copilot use is cognitive rather than clerical.
- What it does not support
- The sample is restricted to people already using AI, so it cannot speak to non-adopters. Impact is self-reported and the telemetry is Microsoft's own product. Microsoft sells Copilot and agents.
Argued perspective#
Romasanta, A., Thomas, L. D. W. and Levina, N. (2026). Researchers asked LLMs for strategic advice. They got trendslop in return
Harvard Business Review, 16 March 2026. Article read at source 26 September 2026; the underlying study was not accessible and the article page discloses no sample, no measures and no figures
- Method
- An article by three business school researchers reporting their own testing of leading language models on strategic trade-offs. The method is described only in general terms on the page read: models were asked for strategic advice across classic trade-offs and their recommendations compared.
- Finding
- Reports that leading models consistently recommend strategies that match current managerial trends and vocabulary rather than with the logic of the situation put to them, a pattern the authors name trendslop. Their recommendation is to use these systems to expand options rather than to make choices.
- What it supports
- That researchers at three business schools have named and published a convergence pattern in model-generated strategic advice, and that the recommendation from inside that finding is to treat model output as option generation rather than as selection.
- What it does not support
- Any rate or effect size. No sample, no measures and no figures were available on the page read, so nothing here supports a quantitative claim, and the underlying study has not been examined.
Working paper#
Tomasev, N., Franklin, M. and Osindero, S. (2026). Intelligent AI Delegation
Google DeepMind. arXiv:2602.11865, submitted 12 February 2026. Preprint, not peer-reviewed
- Method
- A framework paper proposing how AI agents should decompose problems and delegate across other agents and people, covering task assignment, monitoring, trust, permissions and accountability in long delegation chains. No experiment, no data. The capability material is section 5.6, Risk of De-skilling.
- Finding
- The authors name oversight readiness as a property a future workforce may lack, arguing that expertise is built through the repetitive execution of narrowly scoped tasks, that those are the tasks most likely to be delegated to agents first, and that fully automating them would deprive junior staff of the experience needed for strategic judgement. Their proposed remedies are unusual for a frontier lab: curriculum-aware task routing that allocates work inside a junior's zone of proximal development, and a delegation framework that should occasionally introduce minor inefficiencies by routing tasks to humans deliberately, to maintain their skills.
- What it supports
- That the missing rungs argument is being reached independently by people building delegation infrastructure rather than only by those writing about its consequences, and that at least one frontier lab has proposed deliberate inefficiency as a capability-preservation mechanism.
- What it does not support
- Anything measured. It is a preprint framework paper with no empirical component, so it establishes that the risk is taken seriously by practitioners and not that the risk has been observed. The proposed routing systems also assume an organisation can assess what a junior can currently do, which is the unsolved part.
Working paper#
Welsch, R. (2026). When the AI Leaves the Tailorshop: Measuring What an LLM Advisor Leaves Behind in Complex Problem Solving
arXiv 2610.00163, submitted 14 September 2026. Abstract read at source 9 October 2026
- Method
- Two preregistered experiments (N = 200 and N = 198) in which participants managed a simulated clothing factory with or without an LLM adviser, followed by a phase without it.
- Finding
- 'Across studies, AI-supported participants reported greater confidence and understanding with less effort.' In the first study assistance raised company value with no detectable difference in prediction accuracy, and 'Within the AI-supported group, more frequent recommendation alterations predicted better unaided performance.' In the second, 'More frequent recommendation alterations predicted higher knowledge within the AI-supported group.'
- What it supports
- That in a controlled task people felt they understood more with AI than the measures showed, and that those who changed the adviser's recommendations more often did better or knew more afterwards.
- What it does not support
- Cause: the link with altering recommendations is an association inside the supported group. The results are mixed, with a small knowledge advantage for supported participants in the second study. A simulation by one author, read as an abstract, not peer-reviewed.
Where this evidence is used
The research that rests most on this section:
- Does AI actually make people more productive?
- How should an AI agent communicate uncertainty to a human?
- What becomes more valuable in a business as AI gets cheaper?
- How Much Can One Firm Check?
- What happens when AI and human judgement conflict?
- Human and AI decision making
- Does AI weaken human judgement?
- Will AI replace programmers?
The other sections
- Judgement, thinking and offloading 79
- Learning, practice and expertise 92
- Empathy, creativity and what stays human 36
- Work, jobs and the labour market 94
- The seven human capabilities 38
- The institutional record 180
- Professions and sectors 25
- Frontline, clinical and physical work 24
- The international picture 29
The whole base is also machine-readable at evidence.json under CC BY 4.0.
Reference · SS-2026-445 · How I use AI here
Cite this page
Hirji, R. (2026). Human and AI collaboration. The SuperSkills evidence base, SS-2026-445. https://thesuperskills.com/research/evidence/human-ai-collaboration. Last reviewed 3 November 2026.
An evidence review by Rahim Hirji, not peer-reviewed research. For a material claim, cite the underlying study as well; every study here carries its own permanent link.
How citations and IDs work