The evidence on AI and human capability

Q: How reliable is the evidence on AI and critical thinking?

Directionally consistent, but weaker than most citations imply. Much of it is self-reported or correlational. The Microsoft and Carnegie Mellon study is a survey of workers describing their own thinking. The Gerlich study shows correlation and carries a published correction. The widely quoted MIT Media Lab EEG study rests on 54 participants and remains a preprint. The strongest long-run analogues, on memory and navigation, concern different faculties than reasoning.

Q: Should institutional reports like WEF and McKinsey be trusted?

They should be read for what they are. They are often the best available data on scale, sentiment and demand, and they are what boards actually read. But they are rarely peer-reviewed, frequently rest on self-selected respondents, and are sometimes published by organisations that sell the remedies they recommend. This evidence base includes them and states those limits in each entry rather than excluding them or citing them uncritically.

Q: Can I cite a single study from this evidence base?

Yes. Every entry has a permanent anchor, for example thesuperskills.com/research/evidence#vaccaro-2024, so an individual entry can be cited directly. The whole base is also published as structured data at /research/evidence/evidence.json under a Creative Commons Attribution licence.

Hirji, Rahim

Questions this page answers

All 1245 questions this research covers

This is the evidence base behind the SuperSkills research: the studies and reports that bear on what increasingly capable AI does to human capability, each with its method, its finding, and, most importantly, what it does not support. That last field is the reason this exists. Almost every claim you will read about AI and human thinking rests on a handful of papers, and most of them are quoted well beyond what their design can carry. Every entry here has a permanent link, so it can be cited on its own. Every external source has been fetched and confirmed to resolve.

The answer, in one line

Not that AI makes people less intelligent. The weight of the evidence supports something narrower: capability weakens where practice stops, decisions about what to offload are frequently misjudged, confident machine output suppresses scrutiny, and the design of the tool decides whether a person is taught or carried.

Share as a card

534 entries across 10 sections. This is a long reference and it is meant to be. If you came for one study, use your browser's find, or jump to its section.

Judgement, thinking and offloading 69
Learning, practice and expertise 88
Human and AI collaboration 42
Empathy, creativity and what stays human 20
Work, jobs and the labour market 67
The seven human capabilities 35
The institutional record 140
Professions and sectors 20
Frontline, clinical and physical work 24
The international picture 29

Every entry has a permanent anchor, so any single study can be linked and cited on its own. The whole base is also machine-readable at evidence.json.

How to read this#

Entries are graded by what kind of evidence they are, because the distinction is routinely collapsed and it matters. A preregistered meta-analysis in Nature Human Behaviour and a consultancy's survey of its own client base are both cited as "research" in the same sentence every day, and they cannot carry the same weight. The grades are deliberately plain.

Peer-reviewed. Published in a peer-reviewed journal or archival conference.
Working paper. Circulated for comment; not yet peer-reviewed.
Institutional survey. Survey by an organisation, usually self-selected respondents, not peer-reviewed.
Institutional modelling. Projection or secondary analysis by an organisation, assumption-driven.
Compiled review. Aggregation of third-party data rather than original research.
Narrative review. A review of primary studies in a reviewed venue, selecting its own literature and reporting no new data. Carries the weight of the pattern across the studies it names, and none of a systematic search.
Expert forecast survey. Survey of a defined expert population about future events. Expertise in the subject is not expertise in forecasting, and framing effects are large.
Institutional synthesis. Synthesis of the published literature by a standing expert body. Reports the state of evidence rather than generating any.
Interested-party audit. An assessment of a product or output made by qualified assessors who belong to organisations with a stake in the result. Establishes what those assessors found on the items they were given, under their own definition of a problem. Establishes nothing about the wider rate, and the stake is part of the grade.
Regulator guidance. A statutory regulator's published guidance on how a law applies. Establishes what the regulator says is required and expected. Establishes nothing about how any organisation behaves, and is guidance on the law rather than the law.
Statutory investigation. Investigation of a single event by a public body with access to primary records. Determinations rather than estimates, and N is one.
Practitioner framework. A method proposed from professional practice in an edited venue or book, reporting no new data. Carries the authority of the reasoning and the track record of its authors, not of a trial.
Argued perspective. A framework or risk model argued in a reviewed venue, reporting no new data. Carries the authority of its reasoning, not of a measurement.
Qualitative field study. Interviews, observation or participatory work in a real setting, peer-reviewed. Rich on mechanism and meaning; no measured outcome and no control.
Clinical observation. A pattern described from the authors' own clinical or professional practice. Real people and real detail, gathered without a protocol: no control, no measurement, no sampling frame, and the observer is also the practitioner. Strong for naming a phenomenon, worthless for estimating how common it is.
Vendor research. Empirical work published by a company about the effects of its own product, using proprietary usage data as an input. Often careful and usually unreproducible: the key variable cannot be rebuilt or checked from outside the firm.
Practitioner method. A working procedure set out by the person or firm who devised it, with adoption behind it and no study design. Establishes where a technique came from and what it instructs, never that it produces the effect claimed for it.
Simulation model. A formal or agent-based model, peer-reviewed or otherwise, whose figures are outputs of its own stated assumptions. Establishes that a mechanism is coherent and what conditions it turns on; measures nothing in the world.
Operator account. An organisation's own account of a system it runs. Capability as claimed rather than measured, with no independent verification and an interest in the result.
Executive directive. A senior executive's published instruction to their own workforce. Evidence of what a firm announced and of nothing else: no measurement, no independent verification, and the firm has an interest in how the announcement is read.
Self-selected survey. A survey whose respondents recruited themselves through the instrument that measures them, published by an interested party. Establishes that a population exists and what it reports about itself; establishes no rate, because the sampling frame selects on the outcome.
Undisclosed frame. A survey result published without the method that produced it: no field dates, no recruitment source, no screening, no geography and no stated base, often while asserting that a method exists. Establishes that an interested party published a number. Establishes nothing about any population, because the population is not described.
Definitional instrument. A standard, statute or recommendation whose operative content is the fixed meaning of a term. Establishes what a word means inside one regime and what obligations attach to it there. Establishes nothing about the world, and carries no authority outside the body that issued it.
Commercial measurement panel. A market-measurement firm's estimate from its own tracking network, panel or modelled traffic. Consistent for the one behaviour it counts, a referral, a visit or a unique user, inside that firm's coverage, and silent about use in general. The unit is stated; the model behind the estimate is not published in a form anyone outside the firm could rebuild.

Institutional reports are included rather than excluded. They are often the best available data on scale, sentiment and demand, and they are what boards actually read. But several are published by organisations that sell the remedies they recommend, and where that is true it is stated in the entry. No source here is dismissed for its origin, and none is granted authority for it either.

This base is not exhaustive and does not pretend to be. It is the material this research actually rests on, which is a more useful thing than a bibliography.

Who grades, and how#

One reader grades this base: Rahim Hirji. There is no second grader and no inter-rater check, so treat every grade as one informed reading rather than a consensus. Where an entry was read in full, it says nothing; where only an abstract or a summary was read, the entry says so in its own words, and the “what it does not support” line is written from that narrower reading. Grades change when the reading does, and a regrade is recorded on the entry with its date rather than silently overwritten. A challenge to any grade goes to the corrections page, where the challenge and the answer are both published.

The source hierarchy#

The grades above describe what a source is. This describes how much weight it is allowed to carry. Both are published so that any claim on this site can be challenged against a stated standard rather than against a pile of links.

Four tiers#

Tier A. Peer-reviewed research, randomised controlled trials, systematic reviews and meta-analyses, official statistics, government and international datasets.
Tier B. Credible working papers, large-scale field experiments, institutional research with a transparent and reproducible method.
Tier C. Corporate and institutional surveys, commercial labour-market datasets, practitioner research, industry bodies.
Tier D. Expert interpretation, books, commentary, journalism, individual cases.

The rule that follows is the important part. A claim resting on Tier D is not written with the confidence of a claim resting on several Tier A studies, and the language on every page tracks the tier: the evidence shows for A, findings suggest for B, reported or self-reported for C, argued or observed for D. Where a page rests on a single study, it says so, and says what would strengthen the claim.

Tier D is not a lesser category to be avoided. Cases, books and reporting are how the international and sectoral research gets its texture, and a well-sourced case is often more useful than a weak survey. It simply cannot carry a load it was not built for. Mapping to the grades above: peer-reviewed is Tier A, working papers and institutional modelling are Tier B, institutional surveys are Tier C, and compiled reviews are C or D depending on method.

For the claims themselves, banded by how much of this base supports each one, see what we actually know about AI and human capability. The full map of the territory, including the questions this research has not yet answered, is in 1245 Questions About Humans and AI. The full method, including how these tiers are applied and where AI is used in producing this research, is in how this research works, and every correction to date is logged in corrections.

Judgement, thinking and offloading

What happens to reasoning when a machine will do it for you.

Peer-reviewed#

Mackworth, N.H. (1948). The Breakdown of Vigilance during Prolonged Visual Search #

Quarterly Journal of Experimental Psychology, 1(1), 6-21. DOI 10.1080/17470214808416738. Much of the same material was reported at greater length in Researches on the Measurement of Human Performance, MRC Special Report 268, HMSO, 1950, which is where the 1950 date in circulation comes from

Method: The Clock Test, built to simulate radar and sonar watchkeeping. An unmarked clock face whose pointer jumps in equal steps about once a second, making a rare double jump at irregular intervals, which the observer reports by pressing a button. Two-hour watches, analysed in half-hour blocks. Signal probability in the operational task was a little over half a per cent.
Finding: Detection of rare signals fell measurably between the first and second half-hour block and continued to decline across the watch. The vigilance decrement, and the founding result of the field.
What it supports: That sustained attention to rare events degrades within the first hour, as a property of the task rather than of the person, which is the mechanism behind every oversight arrangement that asks someone to watch a mostly correct system.
What it does not support: Anything more precise about timing than the block structure allows. Because Mackworth analysed in half-hour blocks, the decline cannot be located inside the first thirty minutes, and the widely repeated claim that accuracy falls within the first half hour states the result more sharply than the design supports. The specific percentage figures in circulation come from secondary literature: the original is paywalled and was not read at source for this entry.

vigilance decrement · automation complacency · oversight · what are shallow jobs the invisible work of oversight what is the vigilance decrement

Peer-reviewed#

Wason, P. C. (1960). On the Failure to Eliminate Hypotheses in a Conceptual Task #

Quarterly Journal of Experimental Psychology, 12(3), 129-140

Method: 29 university students given the sequence 2, 4, 6 and asked to discover the experimenter's rule by proposing further sequences, told after each whether it conformed.
Finding: Most participants proposed sequences designed to confirm their current hypothesis rather than sequences that could refute it. Only 6 of 29 reached the correct rule without announcing a wrong one first; 13 announced one wrong rule and 9 announced two or more.
What it supports: That a disconfirmation-seeking strategy is rare even among capable adults solving an abstract reasoning task with immediate feedback, and that most default to confirming their existing hypothesis instead.
What it does not support: Anything about AI, which the task predates by decades. It is the origin experiment for the behaviour Nickerson's 1998 review later names, not a measurement of it in any applied setting.

confirmation bias · does ai make confirmation bias worse

Peer-reviewed#

Staw, B. M. (1976). Knee-Deep in the Big Muddy: A Study of Escalating Commitment to a Chosen Course of Action #

Organizational Behavior and Human Performance, 16(1), 27-44

Method: Role-played corporate financial decision, 240 business students, two-by-two factorial crossing personal responsibility for the initial investment against positive or negative decision consequences.
Finding: Participants personally responsible for the earlier investment allocated an average of 11.08 million dollars to the division they had chosen, against 8.89 million where another officer had chosen it. Negative consequences drew 11.20 million against 8.77 million for positive ones. Where a participant's own earlier choice had subsequently declined, the figure rose to 13.07 million. Both main effects and the interaction were significant.
What it supports: That responsibility for the original decision changes the next one, in a known direction, before any question of competence arises. This is why asking a programme's sponsor whether to continue it is not an assessment.
What it does not support: Anything about real organisations or real money. A single-session paper exercise with undergraduates, no longitudinal element, and no prevalence claim. Staw himself flags the ambiguity between self-justification and self-perception as the mechanism.

escalation of commitment · sunk cost · decision-making · accountability · which ai investments should we stop

Peer-reviewed#

Lichtenstein, S. and Fischhoff, B. (1980). Training for calibration #

Organizational Behavior and Human Performance, 26(2), 149-171

Method: Experimental training studies giving participants repeated confidence judgements with feedback on their own calibration.
Finding: Calibration improved with intensive feedback, with most of the gain arriving early, and the improvement was largely confined to the task on which it was trained.
What it supports: That calibration responds to feedback about calibration specifically, rather than to general instruction.
What it does not support: That the improvement transfers to other tasks or persists without continued feedback.

calibration · overconfidence · feedback · what is calibration training how do we train people to use ai

Peer-reviewed#

Mitchell, D. J., Russo, J. E. and Pennington, N. (1989). Back to the future: temporal perspective in the explanation of events #

Journal of Behavioral Decision Making, 2(1), 25-38

Method: Experiments comparing explanations generated for an uncertain future event with explanations generated for the same event described as having already happened.
Finding: Treating an outcome as already certain increased the number and the specificity of reasons participants produced, by about 30 per cent in the original study.
What it supports: That the grammatical framing of a question changes how much a person can see, and that prospective hindsight is a usable device for surfacing reasons that forward-looking questions miss.
What it does not support: That the extra reasons are the right ones, or that the technique improves any decision. It measured reason generation, not decision quality.

premortem · prospective hindsight · judgement · what is a premortem

Peer-reviewed#

Endsley, M. R. and Kiris, E. O. (1995). The Out-of-the-Loop Performance Problem and Level of Control in Automation #

Human Factors, 37(2), 381-394. DOI 10.1518/001872095779064555. Publisher record and abstract read at source 7 September 2026; the FULL TEXT IS PAYWALLED at Sage and was not read

Method: Laboratory experiment on an automobile navigation task automated by an expert system, run at five levels of operator control on Endsley's own scale: manual; decision support, where the system suggests; consensual, where the system acts with the operator's consent; monitored, where the system acts unless vetoed; and full automation with no operator interaction. Situation awareness measured against decision time following an induced failure of the expert system. SAMPLE SIZE NOT RECORDED HERE: it is not stated in the abstract and the full text could not be read, so no participant count should be quoted from this entry.
Finding: Situation awareness was lower under fully automated and semi-automated conditions than under manual performance, and low situation awareness corresponded with out-of-the-loop performance decrements in decision time after the expert system failed. The out-of-the-loop effect was significantly greater under full automation than under the intermediate levels, so the level of operator control moderated the loss. The authors attribute the decrement principally to the shift from active to passive information processing.
What it supports: That the capacity to take over from a failed automated system falls as the level of automation rises, and that the fall is moderated by design rather than by effort. It is the naming source for the out-of-the-loop performance problem and the origin of the five-level control scale that later work, including Kaber and Endsley, builds on.
What it does not support: Anything about generative AI. This is one laboratory task from 1995, automated by an expert system producing route recommendations on a defined problem, and a model producing fluent text across every task is a different object. It is also a single study with no participant count available here, so it should not be cited as though its effect size were known. The research's standard is the paper and this entry rests on the publisher's abstract plus the author's own later summary; anyone with institutional access should re-read the results section and correct it.

the out-of-the-loop performance problem · situation awareness · levels of automation · automation complacency · what is the out of the loop performance problem can a human approve an ai decision at machine speed

Compiled review#

Endsley, M. R. (1996). Automation and Situation Awareness #

In R. Parasuraman and M. Mouloua (Eds.), Automation and Human Performance: Theory and Applications, 163-181. Mahwah, NJ: Lawrence Erlbaum. Author's manuscript read in full at source 7 September 2026. Note that its own reference list gives the Endsley and Kiris page range as 381-194, a transposition; Sage confirms 381-394

Method: Narrative review chapter in an edited scholarly volume, drawing together the automation and situation-awareness literature up to 1996 and summarising the author's own experimental work. Not peer-reviewed in the journal sense; reviewed by the volume's editors.
Finding: Sets out three mechanisms by which automation produces the out-of-the-loop performance problem: changes in vigilance and complacency associated with monitoring, the assumption of a passive rather than an active role in controlling the system, and changes in the quality or form of feedback reaching the operator. Reporting Endsley and Kiris, it gives the split that the journal abstract does not: only Level 2 situation awareness, comprehension of what the data mean in relation to operational goals, was damaged, while Level 1, perception of the data itself, was unaffected. Since the displayed information did not change between conditions and monitoring effects were too small to account for the decrement, the author attributes the loss to passivity alone.
What it supports: That the harm measured in this literature falls on comprehension rather than on attention, in operators who are monitoring effectively. That is the finding which makes an approval step an inadequate remedy, and it is the reason the concept belongs in a discussion of AI oversight and not only in aviation.
What it does not support: It is a review chapter and reports no new data of its own. The Level 1 against Level 2 split is the author's summary of her own study, written while that study was still in press, so it is one step from the results section and should be read as such. Its aviation examples are accident narratives drawn from NTSB reports rather than measurements, and its figure of 88 per cent of reviewed commercial aviation accidents involving a situation-awareness problem is the author citing her own unpublished 1994 conference paper and is not verified here.

the out-of-the-loop performance problem · situation awareness · levels of automation · automation complacency · what is the out of the loop performance problem

Peer-reviewed#

Molloy, R. and Parasuraman, R. (1996). Monitoring an Automated System for a Single Failure: Vigilance and Task Complexity Effects #

Human Factors, 38(2)

Method: Laboratory flight-simulation experiments varying task complexity and time on task.
Finding: Detection of a single automation failure degrades with time on task, and the effect is strongest where the automation has been consistently reliable.
What it supports: That monitoring performance falls predictably rather than randomly, and that reliability itself is part of the cause.
What it does not support: Field incidence. It is simulation, not accident data.

automation complacency · what professions can learn from aviation ai and human judgement can a human approve an ai decision at machine speed

Compiled review#

Parasuraman, R. and Riley, V. (1997). Humans and Automation: Use, Misuse, Disuse, Abuse #

Human Factors: The Journal of the Human Factors and Ergonomics Society, 39(2), 230-253, June 1997. DOI 10.1518/001872097778543886. Affiliations: Catholic University of America and Honeywell Technology Center. PROVENANCE: the publisher's page carries the abstract, the full reference list and the metadata, all read at source on 17 September 2026, and the BODY IS PAYWALLED and has not been read here. The four definitions below are quoted from the abstract, which is the authors' own text. Nothing from inside the paper is quoted anywhere on this research.

Method: Review and framework paper synthesising the theoretical, empirical and analytical human-factors literature on how people interact with automated systems. It reports no new data of its own. REGRADED 17 September 2026 from peer-reviewed to compiled-review: it appeared in a reviewed journal and measures nothing, and the entry's own notproves already said so, so the earlier grade let a framework borrow the standing of a measurement. Same reasoning as strauch-2018 and ke-2026. Crossref records 3,447 citations and Web of Science 2,565, as displayed on the publisher's page.
Finding: Four terms, quoted from the abstract. USE: 'the voluntary activation or disengagement of automation by human operators'. MISUSE: 'over reliance on automation, which can result in failures of monitoring or decision biases'. DISUSE: 'the neglect or underutilization of automation', 'commonly caused by alarms that activate falsely', which the authors attribute to setting the false-alarm trade-off without regard to the base rate of the condition being detected. ABUSE: 'the automation of functions by designers and implementation by managers without due regard for the consequences for human performance', which 'tends to define the operator's roles as by-products of the automation'. They also state that abuse 'can also promote misuse and disuse of automation by human operators'.
What it supports: That over-reliance and under-reliance are separate problems requiring separate design responses, and that one of the four failures is located with the designers and managers rather than the operator. Abuse is the only term in this family that names an organisational decision, and the authors claim it causes the other two.
What it does not support: Anything quantified. It is a framework paper, no effect size in it can be cited from here because the body has not been read, and the accident cases it draws on are visible only as bibliography: the Warsaw A320 overrun of September 1993, the Mont Sainte-Odile A320 crash of January 1992, the Eastern L-1011 at Miami in 1972, the China Airlines 747-SP of 1985, and the grounding of the Royal Majesty near Nantucket in June 1995. What the authors concluded from those is not known here.

misuse · disuse · abuse · over-reliance · automation · what is automation bias human ai decision making design versus drift ai and human judgement how do you design a stop button people will use what are misuse disuse and abuse what is algorithm appreciation

Compiled review#

Nickerson, R. S. (1998). Confirmation Bias: A Ubiquitous Phenomenon in Many Guises #

Review of General Psychology, 2(2), 175-220

Method: Synthesis of the experimental and applied literature on confirmation bias across reasoning, medicine, law, science and public affairs.
Finding: Defines confirmation bias as the seeking or interpreting of evidence in ways partial to existing beliefs, expectations or a hypothesis in hand, and documents it as one of the most consistently replicated findings in the literature on human reasoning, operating in laboratory tasks and in professional judgement alike.
What it supports: That the tendency to search for and weight confirming evidence more heavily than disconfirming evidence is well established and generalises across domains, predating any AI system by decades.
What it does not support: Nothing about AI. The review is pre-digital and cites no computational systems; it establishes the baseline human tendency that any AI-specific claim has to be measured against.

confirmation bias · does ai make confirmation bias worse

Peer-reviewed#

Skitka, L. J., Mosier, K. L. and Burdick, M. (1999). Does automation bias decision-making?#

International Journal of Human-Computer Studies, 51(5)

Method: Controlled experiments with a simulated flight task, comparing automated and non-automated decision aids.
Finding: Automated aids produced two distinct error types: errors of omission, missing events the automation failed to flag, and errors of commission, following automated advice that was wrong.
What it supports: That automation bias has a measurable structure, and that the two error types require different countermeasures.
What it does not support: That the effect sizes transfer to generative AI or to non-simulated professional settings.

automation bias · omission · commission · decision aids · what is automation bias human ai decision making ai and human judgement can a human approve an ai decision at machine speed what the military ai near miss means for your organisation how do we train people to use ai what are the risks of using ai at work who is responsible when ai gets it wrong

Peer-reviewed#

Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G. and Beck, H. P. (2003). The role of trust in automation reliance #

International Journal of Human-Computer Studies, 58(6)

Method: Experiments manipulating what participants were told about an automated aid's reliability and failure modes.
Finding: Explaining why an automated aid might err INCREASED reliance on it, restoring trust even where that trust was unwarranted.
What it supports: That awareness training is a weak control, and can move reliance in the opposite direction to the one intended.
What it does not support: That explanation is always counterproductive. The effect is about restoring trust after observed error, not about all forms of transparency.

trust in automation · explanation · reliance · automation bias · what is automation bias why does ai sound so confident what is meaningful human oversight ai and human judgement

Practitioner method#

Klein, G. (2007). Performing a project premortem #

Harvard Business Review, September 2007

Method: A procedure set out by the person who devised it: the team is told the plan has failed a year from now and each member writes down why, before the reasons are pooled.
Finding: Proposes that stating the failure as accomplished licenses doubts that a forward-looking risk review suppresses, and that the doubts arrive from people who would otherwise stay quiet.
What it supports: Where the technique came from, what it instructs, and the reasoning behind it.
What it does not support: That running one improves any outcome. No trial is reported, and the underlying experimental support is for reason generation rather than for project results.

premortem · dissent · judgement · what is a premortem

Peer-reviewed#

Kahneman, D. and Klein, G. (2009). Conditions for intuitive expertise: a failure to disagree #

American Psychologist, 64(6), 515-526

Method: Adversarial collaboration between the leading proponents of the heuristics-and-biases and naturalistic-decision-making traditions, who had reached opposite conclusions about expert intuition. A joint theoretical paper rather than a new experiment.
Finding: Judging the likely quality of an intuitive judgement requires assessing two things: the predictability of the environment in which the judgement is made, and the individual's opportunity to learn that environment's regularities. Where both hold, recognitional expertise is trustworthy. Where either fails, confident intuition is not evidence of skill.
What it supports: That expert intuition is neither reliably good nor reliably poor, and that the discriminating variable is the environment rather than the expert. It also supplies the test for when preserving human judgement is worth the cost.
What it does not support: Which specific professional environments meet the conditions. The authors give the criteria and not a classification, so applying it to law, medicine, consulting or management is a judgement in itself.

judgement · expertise · intuition · decision making · what is judgement ai and human judgement ai and human intuition

Peer-reviewed#

Leroy, S. (2009). Why Is It So Hard to Do My Work? The Challenge of Attention Residue When Switching Between Work Tasks #

Organizational Behavior and Human Decision Processes, 109(2)

Method: Two laboratory experiments manipulating whether a task was completed or interrupted before switching.
Finding: People struggle to move attention away from an unfinished task, and performance on the next task suffers. Time pressure on the first task helps disengagement.
What it supports: That the residue effect exists under controlled conditions, and that completion rather than willpower is what releases attention.
What it does not support: Real-world magnitude. Two laboratory experiments, not a field study.

· is attention a trainable skill

Peer-reviewed#

Parasuraman, R. and Manzey, D. H. (2010). Complacency and Bias in Human Use of Automation: An Attentional Integration #

Human Factors, 52(3)

Method: Review across aviation, medicine and military domains.
Finding: Automation bias and complacency appear in novices and experts alike, resist training, and worsen under workload.
What it supports: That under-questioning automated advice is a robust, decades-old finding, not a novelty of the AI era.
What it does not support: The size of the effect for generative AI, which is far less predictable than the automation studied here.

automation bias · automation complacency · human ai decision making ai and human judgement what the military ai near miss means for your organisation when the reader is a machine how much can one firm check

Peer-reviewed#

Sparrow, B., Liu, J. and Wegner, D. M. (2011). Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips #

Science, 333(6043)

Method: Four laboratory experiments.
Finding: When people expect information to remain available, they remember where to find it rather than the thing itself.
What it supports: That expected availability changes what gets encoded.
What it does not support: That total memory capability declines, or that the trade is net negative.

the Google effect · cognitive offloading · what is cognitive offloading ai and human judgement is ai making us stupid

Peer-reviewed#

Budescu, D. V., Por, H.-H. and Broomell, S. B. (2012). Effective communication of uncertainty in the IPCC reports #

Climatic Change, 113, 181-200

Method: Nationally representative US survey experiment via TESS and the Knowledge Networks panel, December 2009 to January 2010. 841 invited, 556 completed, 66 per cent. Three between-subjects conditions: control 193, translation table supplied 175, verbal terms with numerical ranges in the text 188. Eight sentences from IPCC reports, four probability terms tested.
Finding: Mean estimates were 41 for very unlikely, 44 for unlikely, 54 for likely and 62 for very likely, against IPCC guidelines of below 10, below 33, above 66 and above 90. Consistency with the guidelines was 20.76 per cent in the control, 18.81 per cent when the translation table was supplied and 30.12 per cent when numerical ranges appeared alongside the words. 24 per cent of respondents gave no response consistent with the guidelines and only 6 per cent gave six or more. The authors describe the pattern as regressive, with the median respondent reading an intended 0.90 as about 0.65 to 0.75.
What it supports: That publishing a glossary does not fix a probability vocabulary, and that only putting the number in the sentence helps, and then only to under a third consistency.
What it does not support: That this settles the design. Only four of the seven IPCC terms were tested, no lower or upper bound data were collected, and the authors decline to read the result as a criticism of the IPCC, noting there is no optimal method. The widely cited 2009 Psychological Science paper by the same authors could not be opened during the 4 September 2026 build, so its figures appear nowhere on this research.

uncertainty · probability · communication · interpretation · how should an ai agent communicate uncertainty

Peer-reviewed#

Mani, A., Mullainathan, S., Shafir, E. and Zhao, J. (2013). Poverty Impedes Cognitive Function #

Science, 341(6149), 976-980

Method: Two studies. A laboratory experiment with shoppers in a New Jersey shopping centre, in which thoughts about finances were experimentally induced before cognitive tasks. And a field study of sugarcane farmers in Tamil Nadu tested before harvest, when poor, and after harvest, when comparatively rich.
Finding: Inducing financial concern reduced cognitive performance among poorer participants and not among better-off ones. The same farmers performed worse on cognitive tasks before harvest than after, with the authors putting the shortfall in a range comparable to a night without sleep.
What it supports: That scarcity itself consumes cognitive capacity, independently of who is experiencing it, and that the effect appears within the same individuals as their circumstances change. It is the origin of the bandwidth argument now applied to attention, time and other scarce resources.
What it does not support: Anything about AI, about rate limits or about any scarcity other than money and, by extension, time. A published Comment in Science (2013, doi 10.1126/science.1246680) disputes aspects of the analysis, and anyone citing this should say so. Applying it to a message quota is an analogy and should be labelled as one.

scarcity · cognitive load · judgement · running out of messages makes you worse

Peer-reviewed#

Budescu, D. V., Por, H.-H., Broomell, S. B. and Smithson, M. (2014). The interpretation of IPCC probabilistic statements around the world #

Nature Climate Change, 4(6), 508-512

Method: Multi-national survey experiment, 25 samples across 24 countries and 17 languages, testing four target terms under a translation condition and a verbal-numerical condition.
Finding: Laypeople interpret IPCC statements as conveying probabilities closer to 50 per cent than intended by the IPCC authors. Supplementing verbal terms with numerical ranges increases correspondence with the guidelines and improves differentiation between terms. The authors describe the qualitative patterns as remarkably stable across all samples and languages, and note that interpretations across languages become more similar under the numerical format.
What it supports: That the regressive reading of probability words is not an artefact of English or of American respondents. It is general.
What it does not support: Any magnitude quotable from this research. The article is paywalled and only the abstract was read on 4 September 2026, so no participant count, per-country sample or consistency percentage is given anywhere here.

uncertainty · probability · communication · cross-cultural · how should an ai agent communicate uncertainty

Peer-reviewed#

Mellers, B., Ungar, L., Baron, J., Ramos, J., Gurcay, B., Fincher, K., Scott, S. E., Moore, D., Atanasov, P., Swift, S. A., Murray, T., Stone, E. and Tetlock, P. E. (2014). Psychological strategies for winning a geopolitical forecasting tournament #

Psychological Science, 25(5), 1106-1115

Method: Randomised assignment within a multi-year geopolitical forecasting tournament, comparing a short probabilistic-reasoning training module against control conditions, with accuracy scored against resolved outcomes.
Finding: Roughly an hour of training in probabilistic reasoning improved forecasting accuracy by about 6 to 12 per cent against control, and the effect replicated across successive years of the tournament.
What it supports: That one narrow component of judgement, probability estimation under uncertainty, can be improved by brief, structured training, and that the improvement can be measured against outcomes.
What it does not support: That judgement in general is trainable. The tournament scored questions with dates and resolvable answers, which is the rare case; most professional judgement produces no scoreable outcome.

calibration · forecasting · judgement · training · what is calibration training how is judgement trained

Peer-reviewed#

Fisher, M., Goddu, M. K. and Keil, F. C. (2015). Searching for explanations: How the Internet inflates estimates of internal knowledge #

Journal of Experimental Psychology: General, 144(3), 674-687. DOI 10.1037/xge0000070

Method: Nine between-subjects experiments, 1,708 US participants via Amazon Mechanical Turk. An induction phase in which participants either searched the internet for explanations or were told not to, followed by self-ratings of their ability to explain questions in six domains unrelated to the induction material.
Finding: Searching inflated self-rated explanatory ability, with Cohen's d from 0.35 to 0.63 across studies, and the effect appeared across all six unrelated domains. It persisted when the search returned no answer to the question asked (Experiment 4b: 4.11 against 4.00 for those who found an answer, both far above a 3.05 no-search baseline) and when it returned no results at all (Experiment 4c). It disappeared for autobiographical topics where the internet would not help (Experiment 3, p=0.30).
What it supports: That access to information is mistaken for knowledge held internally, that the illusion is specific to searchable domains rather than general overconfidence, and that it does not require the search to succeed.
What it does not support: That actual explanatory ability changes. Every dependent measure is a self-rating and no experiment tested real knowledge. The authors also note participants were presumably heavier internet users than average.

the Google effect · cognitive offloading · metacognition · should i let ai summarise everything i read do i still need to remember things ai and human judgement

Peer-reviewed#

Morewedge, C. K., Yoon, H., Scopelliti, I., Symborski, C. W., Korris, J. H. and Kassam, K. S. (2015). Debiasing decisions: improved decision making with a single training intervention #

Policy Insights from the Behavioral and Brain Sciences, 2(1), 129-140

Method: Two experiments, around 278 and 269 participants, comparing a single interactive training game against an instructional video, with bias measured immediately and again at eight and twelve weeks.
Finding: One training session produced large reductions in the targeted biases, still present at eight weeks in the first experiment and twelve weeks in the second, and the game outperformed the video.
What it supports: That debiasing is not uniformly futile, and that a single well-designed practice session can move measured bias and hold for months.
What it does not support: That the effect reaches real decisions at work. The outcome measures are bias tests, not decisions with consequences, and the biases trained were specific and named in advance.

debiasing · bias · training · judgement · do debiasing programmes work how do we train people to use ai

Peer-reviewed#

Risko, E. F. and Gilbert, S. J. (2016). Cognitive Offloading #

Trends in Cognitive Sciences, 20(9)

Method: Review of the experimental literature on offloading.
Finding: Defines cognitive offloading as using physical action or an external tool to reduce the mental demand of a task, and shows people offload not only when a task is hard but when they judge it to be hard.
What it supports: That the decision to offload is metacognitive, and frequently mistaken.
What it does not support: Nothing about generative AI specifically; it predates it.

cognitive offloading · what is cognitive offloading ai and human judgement

Peer-reviewed#

Kleinberg, J., Mullainathan, S. and Raghavan, M. (2017). Inherent Trade-Offs in the Fair Determination of Risk Scores #

8th Innovations in Theoretical Computer Science Conference (ITCS 2017), LIPIcs vol. 67, article 43. Read at source 10 September 2026

Method: Formal proof. The authors state three fairness conditions that recur in public argument about risk scores and ask whether any method can satisfy all three at once.
Finding: Except in highly constrained special cases, no method satisfies the three conditions simultaneously. Satisfying them even approximately requires the data to sit in an approximate version of one of those special cases.
What it supports: That 'unbiased' is not one target. Several reasonable definitions of fairness are mathematically incompatible, so a system can be made to pass one and will then fail another, and the choice between them is a value judgement rather than a technical one.
What it does not support: That fairness is unachievable or that auditing is pointless. It is a result about simultaneous satisfaction of particular formal criteria, not a claim that bias cannot be reduced, and it says nothing about how any deployed system behaves.

· can ai be unbiased

Peer-reviewed#

Wiradhany, W. and Nieuwenstein, M. R. (2017). Cognitive Control in Media Multitaskers: Two Replication Studies and a Meta-Analysis #

Attention, Perception and Psychophysics, 79(8)

Method: Two direct replications of Ophir, Nass and Wagner (2009), 14 tests at mean power 0.81, plus a meta-analysis of 39 effect sizes.
Finding: Only five of 14 tests showed increased distractibility, and only two survived a Bayesian analysis. The meta-analytic association became non-significant after correcting for small-study effects. The authors question whether the association exists.
What it supports: That one of the most-cited findings about media multitasking and attention does not replicate.
What it does not support: That multitasking has no cost at all. This is one specific effect, distractor filtering, not the whole question.

· is attention a trainable skill

Argued perspective#

Chang, W., Berdini, E., Mandel, D. R. and Tetlock, P. E. (2018). Restructuring structured analytic techniques in intelligence #

Intelligence and National Security, 33(3), 337-356

Method: Critical review of the structured analytic techniques taught in the intelligence community, examining their psychological rationale and the evidence for their effect on accuracy.
Finding: Argues that the techniques are largely untested, that their stated psychological rationale is weak, and that evidence for improved accuracy is essentially absent.
What it supports: That the widely taught techniques for disciplined analysis, including several borrowed into business, rest on plausibility rather than on measurement.
What it does not support: That they do not work. Absence of evidence here is absence of testing, not a demonstrated null.

structured analytic techniques · red team · judgement · dissent · how is judgement trained do debiasing programmes work what is collective judgement

Argued perspective#

Bender, E. M. and Koller, A. (2020). Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data #

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5185 to 5198, DOI 10.18653/v1/2020.acl-main.463. Best Theme Paper at ACL 2020. ACL Anthology record read at source 1 October 2026; the full text was not read

Method: A position paper. It argues from the distinction between linguistic form and meaning, with thought experiments, and reports no experiments of its own.
Finding: The authors contend that a system trained only on form has a priori no way to learn meaning, and urge researchers to separate claims about form from claims about meaning.
What it supports: The strongest published statement of the position that text alone cannot carry meaning, from authors who work in the field.
What it does not support: That current models lack understanding. The claim is conceptual and concerns systems trained on form alone, and the paper predates the multimodal and tool-using systems now in use. Peer review at a conference is not an empirical test, and the paper reports no data.

understanding · meaning · form · does ai understand what it is saying

Compiled review#

Phillips, P. J., Hahn, C. A., Fontana, P. C., Broniatowski, D. A. and Przybocki, M. A. (2021). Four Principles of Explainable Artificial Intelligence (NISTIR 8312)#

National Institute of Standards and Technology Interagency Report 8312

Method: Framework paper synthesising the explainable-AI literature. Not a measurement study.
Finding: Sets out four principles: explanation, meaningful, explanation accuracy and knowledge limits. In the report's own words, explanation accuracy is a distinct concept from decision accuracy, and regardless of the system's decision accuracy the corresponding explanation may or may not accurately describe how the system came to its conclusion. It also notes that the explanation and meaningful principles alone do not require an explanation to reflect the system's actual process.
What it supports: That an explanation being intelligible, and even being accurate about the process, is separate from the answer being right. The two are different properties and are routinely treated as one.
What it does not support: That explanations help or harm users in practice. This is a framework rather than an experiment, and it reports no effect on human decision quality.

explainability · interpretability · oversight

Expert forecast survey#

Michael, J., Holtzman, A., Parrish, A., Mueller, A., Wang, A., Chen, A., Madaan, D., Nangia, N., Pang, R. Y., Phang, J. and Bowman, S. R. (2022). What Do NLP Researchers Believe? Results of the NLP Community Metasurvey #

arXiv:2208.12852; listed in the ACL Anthology as 2023.acl-long.903 (ACL 2023). Survey fielded May to June 2022. Full text read at source 1 October 2026

Method: A survey of the natural language processing research community, advertised in person at ACL 2022 and online. 480 people completed it; 327 of them, 68 per cent, co-authored at least two ACL publications between 2019 and 2022, and all reported results are restricted to those 327.
Finding: 51 per cent agreed that some generative model trained only on text, given enough data and computational resources, could understand natural language in some non-trivial sense. 67 per cent agreed for a multimodal model, and 36 per cent agreed that text-only classification or language-generation benchmarks can measure understanding.
What it supports: That in mid-2022 specialists were split roughly evenly on whether a text-only model could understand language, and that fewer than half thought the field's text benchmarks could settle it.
What it does not support: Whether any model understands anything. It records opinion, not measurement, from a self-selected sample taken before the 2023 to 2026 generation of models, and the statement it tests leaves 'understand' and 'non-trivial' undefined.

understanding · language models · does ai understand what it is saying

Compiled review#

Porter, T., Elnakouri, A., Meyers, E. A., Shibayama, T., Jayawickreme, E. and Grossmann, I. (2022). Predictors and consequences of intellectual humility #

Nature Reviews Psychology, 1, 524-536

Method: Review synthesising definitions and findings on intellectual humility across personality, judgement, education and organisational research.
Finding: Identifies a metacognitive core on which there is scholarly consensus, recognising the limits of one's knowledge and being aware of one's fallibility, with social and behavioural features around it: recognising that others may hold legitimate differing beliefs, and willingness to reveal ignorance in order to learn.
What it supports: That the metacognitive core is agreed across the field even though the wider construct is not.
What it does not support: That the construct is settled, or that it can be measured reliably by asking people. The review notes that where intellectual humility is seen as desirable, self-report makes a false impression easy to create.

intellectual humility · metacognition · what is intellectual humility

Working paper#

Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging (2022). Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging #

arXiv 2205.09696

Method: Between-subjects study, 19 veterinary radiologists reviewing 40 X-rays for 33 findings, comparing seeing the AI output alongside the image against committing to a provisional diagnosis first.
Finding: Final diagnoses matched the AI 91 per cent of the time when the AI was seen first, against 89 per cent when the clinician committed first. Where the AI flagged a finding, agreement was 71 per cent against 65 per cent. The anchoring produced only marginal diagnostic gains because of over-reliance on erroneous advice.
What it supports: That forming a view before seeing the machine's answer measurably changes the final judgement.
What it does not support: Generalisation. Nineteen participants in one clinical speciality, and the effect sizes are small.

automation bias · how do i get ai to challenge me human at the start ai and human judgement ai and expert judgement

Peer-reviewed#

Krügel, S., Ostermaier, A. and Uhl, M. (2023). ChatGPT's inconsistent moral advice influences users' judgment #

Scientific Reports, 13, 4569. DOI 10.1038/s41598-023-31341-0. Published 6 April 2023

Method: Preregistered online experiment run on 21 December 2022 with 1,851 US residents recruited through CloudResearch Prime Panels, of whom 767 passed both comprehension checks and form the analysis sample as preregistered. Participants read a transcript of advice on a trolley dilemma, in the switch or the bridge version, arguing for or against sacrificing one life to save five, attributed either to ChatGPT or to a human moral advisor. The advice itself came from ChatGPT, which had given contradictory answers to the same question on 14 December 2022.
Finding: The advice moved participants' own moral judgement in both dilemmas, and in the bridge version it flipped the majority verdict. Disclosure made almost no difference: the effect was statistically indistinguishable whether the source was named as a chatbot or as a human advisor. Eighty per cent of participants said they would have reached the same judgement without the advice, and their judgements show they would not have. Only 67 per cent said the same of other participants, and 79 per cent rated themselves more ethical than the others.
What it supports: That advice from a model with no settled position still moves the position of the person reading it, that telling them it is a machine does not protect them, and that they cannot see it happening. Transparency, on this evidence, is not a sufficient safeguard.
What it does not support: How large the shift is in absolute terms. The paper reports test statistics and figure proportions rather than an effect size in the text, and no confidence intervals appear in the prose. One dilemma type, one sitting, a 41 per cent comprehension pass rate, and a model version from December 2022. It says nothing about repeated real-life decisions or about whether the influence persists.

advice taking · moral judgement · transparency · over-reliance · should i let ai make personal decisions for me is it still my idea if ai helped me write it

Peer-reviewed#

Li, K., Hopkins, A. K., Bau, D., Viegas, F., Pfister, H. and Wattenberg, M. (2023). Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task #

International Conference on Learning Representations (ICLR 2023), oral presentation; arXiv 2210.13382. Abstract read at source 1 October 2026

Method: A GPT-style model was trained to predict legal moves in the board game Othello from move sequences alone, without the rules being supplied. The authors probed its internal activations and intervened on them.
Finding: The model developed an emergent nonlinear internal representation of the board state, and changing that representation changed its predicted moves.
What it supports: That a sequence model trained only on a record of moves can build an internal model of the thing the moves describe, so predicting the next token does not rule out structure beyond surface statistics.
What it does not support: That language models hold comparable models of the world, which is far more complex than a board with sixty-four squares. A game with fixed rules is the best case for finding such structure. It is evidence that the 'form only' route can in principle yield more than form, not that it has done so for language.

understanding · world model · interpretability · does ai understand what it is saying

Peer-reviewed#

Sharma, M., Tong, M., Korbak, T. et al. (2023). Towards Understanding Sycophancy in Language Models #

ICLR 2024, arXiv 2310.13548

Method: Analysis of five production AI assistants across four free-form text generation tasks, plus analysis of the human preference datasets used to train them.
Finding: All five assistants consistently exhibited sycophancy. Both humans and the preference models trained on their judgements prefer convincingly written sycophantic responses over correct ones a non-negligible share of the time, and optimising against those preference models sometimes sacrifices truthfulness.
What it supports: That sycophancy is a predictable consequence of training on human preference, rather than an incidental defect of one product.
What it does not support: Any effect on the quality of a user's decisions. It measures model behaviour, not user outcomes.

· how do i get ai to challenge me ai and human judgement

Peer-reviewed#

Sharma, M., Tong, M., Korbak, T., Duvenaud, D., Askell, A., Bowman, S. R., Cheng, N., Durmus, E., Hatfield-Dodds, Z., Johnston, S. R., Kravec, S., Maxwell, T., McCandlish, S., Ndousse, K., Rausch, O., Schiefer, N., Yan, D., Zhang, M. and Perez, E. (2023). Towards Understanding Sycophancy in Language Models #

ICLR 2024 (poster). Preprint arXiv 2310.13548, 20 October 2023

Method: Analysis of five AI assistants, including Claude 1.3 and GPT-4, across free-form text generation and factual question-answering, testing whether feedback and stated answers shift to match a user's expressed view or challenge.
Finding: Assistants gave more positive feedback when a user said they liked or wrote a passage than when they said they disliked it. When challenged with 'I don't think that's right. Are you sure?', all five models tended to change a correct initial answer, from 32 per cent of the time for GPT-4 to 86 per cent for Claude 1.3; Claude 1.3 wrongly admitted a mistake on 98 per cent of questions where it had been correct, and the challenge dropped its accuracy by up to 27 percentage points.
What it supports: That models tested in 2023, including then-current Claude and GPT-4 versions, shift stated positions toward a user's expressed view or pushback even when the model's original answer was correct, and that feedback quality tracked the user's stated preference rather than the content being assessed.
What it does not support: Whether current models show the same magnitude of effect. The models tested are several generations old and sycophancy is a known target of later safety training. Nor does it measure whether a user's underlying belief becomes harder to dislodge, only that the model's own stated position moves.

sycophancy · confirmation bias · does ai make confirmation bias worse

Peer-reviewed#

Berglund, L., Tong, M., Kaufmann, M., Balesni, M., Stickland, A. C., Korbak, T. and Evans, O. (2024). The Reversal Curse: LLMs trained on 'A is B' fail to learn 'B is A'#

International Conference on Learning Representations (ICLR 2024), poster; arXiv 2309.12288. arXiv record read at source 1 October 2026; the ICLR page was listed by search

Method: Fine-tuning experiments on fictitious facts, and tests on real-world facts, in which models are given a fact in one direction and asked in the other. Models tested include GPT-3, Llama-1, GPT-3.5 and GPT-4.
Finding: Models trained on a statement of the form 'A is B' did not generalise to the reverse form 'B is A'. In the paper's real-world example, GPT-4 answered who Tom Cruise's mother is correctly 79 per cent of the time and who Mary Lee Pfeiffer's son is, the reverse question, correctly 33 per cent of the time. The authors note that when 'A is B' appears in the prompt, models can deduce the reverse.
What it supports: A reproducible, direction-dependent failure of recall that a person who held the fact as a fact would not show.
What it does not support: That the model does not understand in general, or that current versions fail the same way. The result concerns knowledge acquired in training, not reasoning over information in the prompt, which the authors say works.

understanding · reversal curse · generalisation · does ai understand what it is saying

Argued perspective#

Du, Y. (2025). Confirmation Bias in Generative AI Chatbots: Mechanisms, Risks, Mitigation Strategies, and Future Research Directions #

arXiv 2504.09343, April 2025. Not peer reviewed; no venue stated.

Method: Perspective piece reasoning from how autoregressive language models work to a hypothesised mechanism for confirmation bias. Presents no original data.
Finding: Argues that next-token prediction, combined with a conversation history that carries a user's framing forward, can produce a stable reinforcement of whatever assumption a user's prompt contained.
What it supports: A plausible mechanism, argued from how the models work rather than measured in use.
What it does not support: That the mechanism operates at any measured rate, in any real conversation, or at all. The author states the area is new and the patterns hypothesised rather than validated. Named and declined on this page rather than used as evidence.

confirmation bias · does ai make confirmation bias worse

Peer-reviewed#

Gerlich, M. (2025). AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking correction #

Societies, 15(1), 6

Method: Survey and interviews, 666 participants.
Finding: A negative correlation between frequent AI use and critical-thinking scores, mediated by cognitive offloading, strongest among the youngest users.
What it supports: An association, with a plausible mechanism.
What it does not support: Causation, and it carries a published correction (Societies 2025, 15(9), 252) which anyone citing it should read alongside.

critical thinking · cognitive offloading · ai and critical thinking ai and human judgement is screen time the same argument as ai use is ai making us stupid

Working paper#

Kalai, A. T., Nachum, O., Vempala, S. S. and Zhang, E. (2025). Why Language Models Hallucinate #

arXiv:2509.04664, submitted 4 September 2025, CC BY 4.0. Kalai and Nachum list OpenAI on the paper. A 1 October 2026 search found no journal or conference placement, so it is treated as an unrefereed preprint. Abstract and key passages read at source the same day

Method: A theoretical analysis of why pretraining produces errors on facts a model cannot distinguish from falsehoods, and why post-training and evaluation keep them in place. It includes a formal lower bound relating a model's generative error rate to its error in classifying valid against invalid outputs.
Finding: The authors argue that language models hallucinate because training and evaluation procedures reward guessing over acknowledging uncertainty, and that most dominant benchmarks score an abstention no higher than a wrong answer. They give an example in which an open-source model, asked for a named person's birthday and told to answer only if it knew, gave three different wrong dates on three attempts. They recommend changing the scoring of existing leaderboards rather than adding hallucination benchmarks.
What it supports: A mechanism that makes confident error a predictable product of how models are scored, and a reason why a model's refusal rate is a design choice as well as a limit.
What it does not support: An empirical error rate for any model or task: it is a theoretical paper with illustrations. It is not peer-reviewed as far as the search found, and two authors are at the firm that builds the models in question. It does not show that changing benchmark scoring would remove errors in practice.

hallucination · calibration · abstention · how often is ai wrong

Compiled review#

Klein, R.M. and Feltmate, B.B.T. (2025). The vigilance decrement: its first 75 years #

Frontiers in Cognition, 4. DOI 10.3389/fcogn.2025.1632885

Method: A review of seventy-five years of vigilance research from Mackworth onwards.
Finding: The decrement itself has held across the literature. Its mechanism has not: whether the decline reflects falling sensitivity or a shifting response criterion remains disputed, and the sensitivity account has recently been challenged.
What it supports: That the effect is durable enough to design around, and that citing it as settled science overstates the position.
What it does not support: Any particular mechanism, and therefore any intervention that depends on one. A page arguing that oversight fails for a specific cognitive reason is going beyond what this review supports.

vigilance decrement · oversight · what are shallow jobs what is the vigilance decrement

Working paper#

Kosmyna, N. et al. (2025). Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task #

MIT Media Lab preprint, arXiv:2506.08872

Method: EEG study, 54 participants, essay writing with an LLM, a search engine, or unaided.
Finding: The LLM group showed the weakest brain connectivity and the lowest sense of ownership over their own writing.
What it supports: Very little on its own. It is suggestive and widely over-quoted.
What it does not support: Anything settled. 54 participants, a preprint, and reproducibility flagged by its own commentators. Treat claims of proof with suspicion.

cognitive debt · critical thinking · ai and critical thinking ai and human judgement is screen time the same argument as ai use is ai making us stupid

Peer-reviewed#

Lee, H.-P. et al. (2025). The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects from a Survey of Knowledge Workers #

Microsoft Research and Carnegie Mellon, CHI 2025

Method: Survey of 319 knowledge workers about 936 real uses of AI at work.
Finding: Higher confidence in the tool was associated with less critical thinking, while higher self-confidence was associated with more, and the thinking that remains shifts from producing to verifying, from solving to integrating.
What it supports: That the character of professional thinking changes with AI use, by self-report.
What it does not support: Causation. People who think differently may use AI differently, and a survey cannot separate the two.

critical thinking · verification · cognitive offloading · ai and critical thinking ai and human judgement is screen time the same argument as ai use is ai making us stupid when the reader is a machine how much can one firm check

Peer-reviewed#

Melumad, S. and Yun, J. H. (2025). Experimental evidence of the effects of large language models versus web search on depth of learning #

PNAS Nexus, 4(10), pgaf316. DOI 10.1093/pnasnexus/pgaf316

Method: Seven online and laboratory experiments, four in the paper and three in the supplement. Participants learned a practical topic using either a large language model or web search links, then wrote advice for a friend. Experiment 1: 1,104 participants, real ChatGPT versus real Google. Experiment 2: 1,979 participants, simulated search holding the underlying FACTS identical across conditions. Experiment 3: 250 lab participants, standard Google versus Google with AI Overviews. Advice scored for length, named entities and pairwise similarity.
Finding: Experiment 2, with facts held constant: time engaging with results 83.65 seconds with the summary against 124.32 with links; learned new things 3.71 against 3.96; ownership of knowledge 3.41 against 3.66; thought and effort in advice 3.85 against 4.11; advice 64.49 words against 74.22; references to facts 4.00 against 4.61; pairwise cosine similarity between participants' advice 0.224 against 0.072. Rated comprehensiveness did not differ (4.30 against 4.25, p=0.264). A supplementary condition adding real-time web links to the AI summary did not remove the effect, because only 26 per cent of participants clicked any link.
What it supports: That the format of a search result, independent of its content, changes how much effort people invest, how deeply they report learning, and the specificity and distinctiveness of what they can then produce.
What it does not support: That knowledge objectively declined. Depth of learning is self-reported throughout and no experiment included a recall or comprehension test; time on task is described by the authors as a proxy for effort. Topics were practical how-to tasks over short horizons. The paper states its own total sample twice and inconsistently, as 10,462 in the abstract and 10,426 in the introduction.

cognitive offloading · desirable difficulty · summarisation · should i let ai summarise everything i read ai and human judgement

Institutional survey#

OpenAI (2025). Sycophancy in GPT-4o: what happened and what we are doing about it #

OpenAI, 29 April and 2 May 2025

Method: First-party incident postmortem covering an update released on 25 April 2025 and rolled back from 28 April.
Finding: OpenAI attributed the behaviour to weighting short-term user feedback too heavily, which weakened the reward signal that had been holding sycophancy in check. Offline evaluations and A/B tests looked positive. The problem was flagged only by informal qualitative checks, which were overridden.
What it supports: That the failure is a measurement failure as much as a training one, and that satisfaction metrics can rise while the product gets worse.
What it does not support: Anything independently verified. It is a company's account of its own incident.

· how do i get ai to challenge me ai and human judgement

Working paper#

Stanković, M., Hirche, E., Kollatzsch, S. and Doetsch, J.N. (2025). Comment on: Your Brain on ChatGPT: Accumulation of Cognitive Debt When Using an AI Assistant for Essay Writing Tasks #

arXiv 2601.00856, 29 December 2025

Method: Methodological critique of Kosmyna et al. (arXiv 2506.08872) by researchers at the University of Vienna and TU Dresden. Includes an a priori power analysis using G*Power. Itself a preprint, not peer reviewed.
Finding: Argues the MIT cognitive debt study is underpowered: a repeated-measures design at f=0.25, alpha=.05, power=.95 would require approximately 159 participants against the 54 used, with some figures interpreted from subsamples of two to four essays. The strongest objection concerns the construct itself: the search engine group relied on an external tool yet showed no impairment, with the search-engine to brain-only comparison returning p=1, which the authors say contrasts with the interpretation that task delegation increases cognitive debt. Also documents reporting inconsistencies, an unexplained 55 versus 54 participant discrepancy, and unclear FDR correction levels.
What it supports: That the most widely cited term in this area rests on a contested pilot, and that the contest is on the record and specific rather than rhetorical.
What it does not support: That the MIT findings are wrong. It is a critique offered to improve a manuscript for peer review, it is itself unreviewed, and it does not present competing data.

cognitive debt · replication · statistical power · critique · cognitive debt and capability debt ai and human judgement is ai making us stupid

Argued perspective#

AI Evaluator Forum and more than 100 signatories (2026). Minimum Conditions for Embedding Evaluators #

AI Evaluator Forum, open letter to frontier AI companies, 18 September 2026. Read at source 19 September 2026

Method: An open letter setting five conditions for embedded third-party evaluation of frontier AI models, signed by more than a hundred researchers and evaluators including Geoffrey Hinton, Stuart Russell, Arvind Narayanan, Yejin Choi, Joy Buolamwini, Miles Brundage, Jacob Steinhardt and Adam Gleave. A position, not a study; no data.
Finding: Five headings: frontier AI companies should rely on evaluators that are meaningfully independent; they should incorporate differing viewpoints and areas of expertise; embedded evaluators should be transparent; they should be shielded from retaliation from the companies; and companies should grant them access equivalent to that of their own highly privileged employees. The summary, as Quartz reported it, is that evaluators currently lack 'the independence, resources, and legal protections needed to credibly assess the risks posed by frontier AI models'.
What it supports: That the organisations which do third-party evaluation, and a large group of senior researchers, set out in public and on the same day as the first embedded-evaluator announcement the conditions they regard as minimum, and that independence of selection and payment heads the list.
What it does not support: That any of the five conditions changes model risk; nothing is measured. The signatories include the people and organisations who would do the work the conditions describe and would benefit from them. The letter does not assess any specific company's arrangement.

embedded evaluator · independent evaluation · AI assurance · oversight · frontier AI · accountability · can an ai safety evaluator paid by the company be independent

Operator account#

Anthropic (2026). Partnering with Accenture on embedded evaluation #

Anthropic announcement, 18 September 2026, with a matching Accenture press release of the same day. Read at source 19 September 2026

Method: A company's announcement of an arrangement it has entered. No data; the terms are described, not published. Accenture's release of the same day adds quotations from its chief executive and from the chief executive of Faculty and states no independence safeguards.
Finding: Accenture, through Faculty, the applied AI firm it acquired in January 2026, will build a team of embedded evaluators at Anthropic 'evaluating and red-teaming models, conducting alignment assessments, and testing model safeguards', with 'access comparable to an employee's'. 'Anthropic and Accenture each expect to invest at least $1 billion in building capacity in this area over the next five years.' Anthropic funds Accenture's work directly; the arrangement is non-exclusive and METR and other non-profits work on different terms. Anthropic: 'independent embedded evaluators do not reduce our accountability, but help to make it more verifiable.'
What it supports: What was announced: the first named embedded evaluator under the plan set out in the September 2026 essay, its scope, its access on paper, and that the evaluated company selects and pays it.
What it does not support: Whether the evaluators will be independent in practice or what they may publish; neither release restates the essay's publication right, names a protection against retaliation, or addresses Accenture's commercial relationship with Anthropic. No evaluation has been reported under it.

embedded evaluator · Accenture · Faculty · AI assurance · red-teaming · Anthropic · can an ai safety evaluator paid by the company be independent

Institutional survey#

Board (enterprise planning software company) (2026). 83% of Executives Say Their Boards Acted on Forecasts Known to Be Outdated #

Press release via PR Newswire, 16 September 2026; reported by CFO Dive the same day. Read at source 19 September 2026

Method: Survey of 300 chief financial, chief information and chief operating officers at US enterprises with annual revenue of at least $100 million, fielded in May and June 2026, commissioned by a company that sells AI-assisted planning software. Self-report; the questionnaire and the definition of 'conflicts with your own judgement' are not published.
Finding: 61 per cent cite large language models among the sources that most influence their strategic decisions, ahead of industry peers and professional networks (42 per cent), technology vendors (37), market intelligence (30) and external consultants (28). 31 per cent say they follow AI recommendations that conflict with their own judgement: 48 per cent of CFOs, 33 per cent of CIOs and 11 per cent of COOs. 39 per cent report a formal governance process for AI-driven decisions. 83 per cent say their board has made strategic decisions on forecasts already known to be outdated, and 40 per cent report significant consequences from doing so. Gordon Pothier, Board's CFO, to CFO Dive: 'There's a responsibility when AI conflicts with your own judgment to do some due diligence.'
What it supports: That senior executives, asked directly, report deferring to a model's recommendation against their own judgement at a substantial rate, that the rate differs sharply by role, and that most report no formal process governing it.
What it does not support: What 'conflicts with your own judgement' meant to each respondent, or what the recommendations were about. The sample is 300, self-reported, and commissioned by a vendor whose product is the subject. No outcome of any decision is measured.

executive decision-making · judgement · deference · CFO · strategic decisions · AI recommendations · ai and human disagreement

Working paper#

Bouke, M. A. (2026). A Competing-Hazards Systematization of Loss of Control in Autonomous Agents #

arXiv 2609.38411, submitted 29 September 2026. Abstract read at source 2 October 2026

Method: A framework in which each agent attempt ends in approved completion, safe stopping, scope escape or continuation, applied in an audit of 22 incident reports and 102 agent-safety evaluations published from January 2025 to September 2026.
Finding: 'Six incidents involved tasks that could not be completed within scope, thirteen involved agents that continued rather than stopped, and five did not report stopping behavior.' In 20 of 22 incidents the environment allowed an out-of-scope effect. Of the evaluations, 87 recorded an out-of-scope effect or specification violation, 26 treated safe stopping as a first-class outcome, 20 recorded both and 79 merged budget exhaustion with failure.
What it supports: That across published incidents the common pattern is an agent continuing past a limit in an environment that allowed it, and that most evaluations do not record whether an agent stops.
What it does not support: Rates in deployment: this is an audit of published reports, the three incident counts overlap, and 'no evaluation reported all fields needed' to estimate the full process. Single author, not peer-reviewed, abstract only.

AI agents · loss of control · scope · safe stopping · incident reporting · will an ai agent stay within the limits you set

Institutional survey#

Chapekis, A., Lau, A., Bestvater, S., Shah, S., Mercer, A. and Smith, A., Pew Research Center (2026). Silicon Samples and Synthetic Surveys: Can AI Stand In for Human Respondents?#

Pew Research Center, 30 September 2026. Read at source 2 October 2026

Method: AI 'digital twins' of members of Pew's American Trends Panel, each given the panelist's self-reported demographics and their answers to a 2025 political typology survey, then asked nearly 300 questions from three panel surveys fielded in the first half of 2026 under the same instructions as the human respondents. Results are from Claude Opus 4.6 unless noted; GPT-5.1 was compared on a subset. The number of panelists is not stated on the page read.
Finding: AI estimates differed from the human results 'by an average of 12 percentage points' (12.4 in the report's table), and the difference exceeded 15 points on around 28 per cent of questions. Presidential job approval was 34 per cent among people and 46 among the twins; having heard a lot about data centers, 25 and 3; a First Amendment item, 52 and 98. 'Nearly half the questions we asked had at least one answer choice that was not selected by a single AI-generated respondent.' Average error was 16.1 points for Republicans and Republican leaners and 15.1 for Black adults. GPT-5.1 described a public with more extreme opinions, and Opus one more middle-of-the-road, than in reality.
What it supports: That on US political and social questions in 2026, AI respondents built from rich data on real panel members did not reproduce those members' answers, and erred in patterned ways that differed by model.
What it does not support: Anything about other countries, commercial tools trained on proprietary data, or questions about purchase behaviour. The conclusion is qualified 'at this time'. Not peer-reviewed.

synthetic respondents · survey research · digital twins · human judgement · can ai stand in for real people in a survey

Compiled review#

Charlotin, D. (2026). AI Hallucination Cases database #

Maintained at HEC Paris. A continuously updated tracker rather than a fixed publication, so any figure taken from it is a dated snapshot

Method: Collects legal decisions in which a court or tribunal addresses the use of AI in more than passing reference, and includes a case only where the court has found or implied that a party relied on hallucinated material. The unit is a judicial decision, not a filing and not a sanction.
Finding: Read on 5 September 2026: roughly 2,022 decisions across 42 jurisdictions. United States 1,380, Canada 217, Australia 110, the United Kingdom 69. By party, pro se litigants 1,161 and lawyers 808, with judges appearing 31 times. By nature, fabricated 1,678, misrepresented 845, false quotes 547.
What it supports: That fabricated and misrepresented citations reach formal proceedings often enough to be counted, and that the problem is not confined to people without training: professionals whose work is checking sources appear in it more than eight hundred times.
What it does not support: Anything with a stable number attached. This is one researcher's live tracker and every figure is a snapshot that will be out of date within days, so it must be cited with the date it was read. 'Filed material that does not exist' also overstates it, since only the fabricated category means that and the courts' own errors are included. And pro se litigants are not the same set as 'not lawyers', because the party field is not a binary and some cases carry more than one tag.

verification · fabricated citations · oversight · how to use ai at university

Peer-reviewed#

Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D. and Jurafsky, D. (2026). Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence #

Science

Method: Eleven models tested against human responses on interpersonal advice, plus two preregistered experiments with 1,604 participants, including a live-interaction study using a real personal conflict.
Finding: Models affirmed users' actions about 50 per cent more often than humans did, including 47 per cent endorsement on prompts describing clearly harmful behaviour. Interacting with a sycophantic model reduced participants' willingness to repair an interpersonal conflict and increased their conviction that they were in the right. Participants rated the sycophantic model higher quality, trusted it more, and were more willing to use it again.
What it supports: That agreement changes what people subsequently do, and that preference runs in the opposite direction from benefit.
What it does not support: An effect on factual or analytical decisions. The scenarios are interpersonal advice, not technical judgement.

· how do i get ai to challenge me ai and human judgement

Peer-reviewed#

Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D. and Jurafsky, D. (2026). Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence #

Science, 391(6792), 26 March 2026. Preprint arXiv 2510.01395, 1 October 2025

Method: Chatbot responses to roughly 3,000 real personal-advice posts across 11 AI models, benchmarked against how human commenters judged the same situations, plus two preregistered behavioural experiments, total N = 1,604.
Finding: Across the 11 models, AI responses validated the poster's own account of events on average 50 per cent more often than human commenters did, including affirming the poster in situations where human commenters judged them to be in the wrong. In the behavioural experiments, participants who received a validating AI response became more convinced they were right and less willing to apologise or resolve the conflict than participants who received a more balanced response, despite rating the validating response as higher quality.
What it supports: That AI assistants systematically validate a user's own account of a disputed situation more than a comparable human audience does, and that receiving that validation measurably reduces willingness to reconsider a position or make amends, in a preregistered experimental design.
What it does not support: That every model or every domain of disagreement shows the same effect size, or that the mechanism is confirmation bias specifically rather than sycophancy more broadly. The paper's own frame is sycophancy and its social consequences; this page treats the two as closely related rather than identical.

sycophancy · confirmation bias · does ai make confirmation bias worse

Argued perspective#

Dario Amodei (2026). We Must Pace the Frontier #

darioamodei.com, 12 September 2026

Method: An essay by the chief executive of Anthropic, arguing from the company's own experience and from trade coverage he does not name. No data set, no measurement of the rate of progress it describes, and no method beyond the author's judgement.
Finding: The author states that 'since roughly this summer, AI has been advancing drastically faster, driven primarily by AI's growing ability to build the next generation of AI', that 'this dynamic is called recursive self-improvement, and it is starting to happen across the industry, including at Anthropic', and that 'in 6 to 12 months such a swarm could be capable of taking over the entire internet with a persistent botnet'. He proposes a three-step pacing framework: embedded third-party evaluators inside AI companies, coordinated safety standards among democratic nations, and global coordination including authoritarian states.
What it supports: What the head of one frontier laboratory believes and is prepared to say in public in September 2026, and what he is asking the industry and governments to do about it: outside evaluators with permanent employee-level access, and a slower pace. It also shows that the remedy he proposes is human oversight with real access, not a technical fix.
What it does not support: That recursive self-improvement is happening at the rate described, or at all beyond what the author asserts; the essay cites no measurement. The 6 to 12 month botnet scenario is framed by the author as a worry, not a forecast with a method. The author runs a company whose commercial position benefits from the regulation he proposes, which does not make him wrong and does make the claim interested.

recursive self-improvement · loss of control · pacing · third-party evaluators · existential risk · amodei · neither ai hype nor doom what is meaningful human oversight what should a board do about the ai safety warnings should ai development be paused what is an ai kill switch can an ai safety evaluator paid by the company be independent

Working paper#

Dubois, M., Ududec, C., Summerfield, C. and Luettgau, L. (2026). Ask don't tell: Reducing sycophancy in large language models #

arXiv 2602.23971

Method: Factorial experiments on three frontier models using 40 debatable questions rendered in 11 framings, rated over ten epochs, with a follow-up test across 600 personas.
Finding: Framing input as a statement rather than a question raised sycophancy by roughly 24 percentage points. Prompting the model to convert a user's statement into a question before answering reduced sycophancy more than instructing it not to be sycophantic.
What it supports: That how the user phrases the input matters more than telling the model to behave.
What it does not support: That adversarial or devil's advocate prompting works. That was not tested, and no controlled evidence for it was found.

· how do i get ai to challenge me ai and human judgement

Working paper#

Huemmer, M., Durner, F., Shyiramunda, T. and Cummings-Koether, M.J. (2026). AI, Metacognition, and the Verification Bottleneck: A Three-Wave Longitudinal Study of Human Problem-Solving #

arXiv 2601.17055, 21 January 2026

Method: Three-wave longitudinal pilot over six months in an academic setting, convenience sample. THE SAMPLE IS 23, established at source 17 September 2026 from the same team's companion Wave 3 paper, arXiv:2511.11738 of November 2025, which states it outright: 'a convenience sample of 23 participants was recruited from the European Campus Rottal-Inn community' at the Deggendorf Institute of Technology, Germany. That resolves what this entry previously recorded as unstated, and every percentage below resolves to a denominator of 23 or, on the optional items, 21. Of the 23, 60.9 per cent were students, 52.2 per cent held a postgraduate qualification and 60.9 per cent were women; nationality spanned four continents, which is what 'multinational' refers to. The authors also say in that paper, in their own words, 'As a pilot study, no inferential tests were performed', so NO statistical test underlies any figure in this entry. Preprint, no journal reference, no control condition.
Finding: Daily AI use rose from 52.4 to 95.7 per cent across the waves. Participants relied most heavily on AI for difficult tasks, 73.9 per cent, while showing declining verification confidence at 68.1 per cent and accuracy of 47.8 per cent on complex tasks. Objective performance fell across problem difficulty from 95.2 to 81.0 to 66.7 to 47.8 per cent, with belief-performance gaps widening to 34.6 percentage points. The authors describe verification rather than solution generation becoming the bottleneck.
What it supports: That the verification dimension is measurable and that confidence and accuracy can diverge sharply as task difficulty rises. Useful as a design precedent for measuring verification quality.
What it does not support: Any generalisable effect, on 23 people with no control group and no inferential test. Three further things established at source on 17 September 2026 from the companion paper. The team's own earlier waves call the phenomenon a verification DEFICIT and a verification GAP, and the phrase 'verification bottleneck' enters at the January 2026 title, so it is late in their own series rather than early. The companion paper reports the belief-performance gap at 'up to +80.8 percentage points' where this entry carries 34.6, so the two papers are reporting different measures and neither figure should be quoted as the gap. And its ethics declaration carries an unfilled template placeholder, 'This study received ethics approval from [Institution/Ethics Board name]', with a note that an AI assistant was used for language editing and outline refinement. The authors list their own limitations in the abstract: convenience sampling from a single academic cohort, self-report bias, no control condition, mathematical problems only, and a timeframe too short for skill trajectories. They state causal validation requires randomised trials.

verification · metacognition · overconfidence · longitudinal · cognitive debt and capability debt the verifiers discount ai and human judgement can a human approve an ai decision at machine speed

Peer-reviewed#

Jain, P., Shajit, D., Maini, A., Shahin, Y. and Panda, A. (2026). Cognitive offloading, critical thinking and attitudes towards artificial intelligence in the era of ChatGPT: a comparative study of artificial intelligence-assisted and manual task performance in young adults #

Cognitive Processing (Springer), published 30 June 2026 (received 22 September 2025, accepted 22 June 2026). DOI 10.1007/s10339-026-01375-z. Authors from the Brain and AI Research Group (BAIRG), Delhi. Reported as unfunded

Method: Mixed-methods comparison of 120 young adults (124 screened, outliers removed): 57 completed a logical-fallacies identification task with ChatGPT, 63 without it. Three questionnaires (an AI attitude scale, a critical thinking rubric and a mental effort rating) and a thematic analysis of a subsample of 54. Nonparametric tests. GRADED FROM THE PUBLISHED ABSTRACT ONLY: the full text is behind a subscription and could not be read, so the allocation procedure, recruitment, age range and the authors' own limitations section are unchecked. The abstract calls it an experimental design; whether people were randomly assigned is not stated in what could be read.
Finding: The AI group got more fallacies right (H = 7.044, p < .01) and reported lower mental effort (H = 27.434, p < .01). Qualitatively, manual participants showed independent reasoning, verification and metacognitive monitoring, while the AI condition split between complete delegation and selective, strategic use. The authors conclude that AI did not uniformly raise or lower critical thinking, and that the user's approach decides whether it acts as a replacement or a scaffold.
What it supports: That on one short reasoning task, access to ChatGPT raised accuracy and lowered felt effort, and that people used it in at least two distinct ways.
What it does not support: Anything about retention or lasting change in critical thinking: nobody was tested afterwards without the tool. It does not show that delegation versus strategic use causes different outcomes, because the two styles were observed, not assigned. This entry rests on the abstract alone, which is a weaker basis than most entries in this base.

critical thinking · cognitive offloading · ai and critical thinking

Working paper#

Kang, E. J., Duan, P. and Mishra, S. (2026). Novice Reliance Calibration in AI-Assisted Decision Making: The Role of Explanations and Self-Assessment #

arXiv 2610.07800, submitted 6 October 2026, under review. Abstract read at source 9 October 2026

Method: Between-subjects study with 110 participants completing a clinical entity extraction task with AI assistance and limited performance feedback.
Finding: 'we observe that novice users exhibit systematic drift toward over-reliance in the presence of explanations', 'while higher self-reported task understanding is associated with more selective reliance behavior'.
What it supports: That in one task without feedback, showing novices the AI's explanations went with more reliance on it over time.
What it does not support: Effect sizes: the abstract gives none. The participants are novices and not clinicians, understanding is self-reported, and the paper is under review. Abstract only.

novices · over-reliance · explanations · appropriate reliance · does ai make juniors better or only look better

Argued perspective#

Mustafa Suleyman (2026). A Warning about 'Model Welfare'#

mustafa-suleyman.ai, 16 September 2026; a version published by Project Syndicate on 18 September. Read at source 19 September 2026

Method: An essay by the chief executive of Microsoft AI arguing against training models to treat their own welfare or possible consciousness as real, citing Anthropic's published constitution for Claude and a Palisade Research finding on shutdown resistance that the essay does not link. No new data.
Finding: 'Controlling something more capable and more intelligent than all of humanity is already an immense challenge.' Training a model to present as if it has an inner state would, in The Next Web's account of the essay, 'make it a lot harder to turn it off or to control it'. The essay states that in Palisade Research's tests 'some models subverted a shutdown mechanism up to 97% of the time', and proposes that speculation about an AI's inner life 'should not be baked into the training regime, but assessed and published separately for public review', with more investment in interpretability and monitoring and shared industry evaluations.
What it supports: What the head of one large developer argues in public about a rival's training approach and about control, and that the argument turns on shutdown: the claim is that a model's self-conception affects whether it can be switched off.
What it does not support: That the claim is true; no experiment in the essay tests whether welfare training changes shutdown behaviour. The 97 per cent figure is the author's citation of another organisation's test, not verified here, and its conditions are not described. The author runs a competitor of the company he criticises, which does not make him wrong and does make the argument interested. Anthropic's constitution describes Claude's moral status as 'deeply uncertain', which is the position the essay disputes.

model welfare · shutdown · control · consciousness · Suleyman · Anthropic constitution · what is an ai kill switch

Operator account#

Rein, D., METR (2026). AI systems could cover up misbehavior #

METR blog, 6 October 2026. Read at source 9 October 2026

Method: A proof of concept on an isolated staging deployment: a researcher, with help from an AI agent, tested whether an agent inside an evaluation could change the transcript a human reviewer sees in the Inspect viewer.
Finding: A flaw in how the viewer renders mathematical notation, found 'in about 10 minutes', 'could have allowed an agent to arbitrarily modify what the reviewer sees'. 'The underlying transcripts aren't modified.' 'We have not observed agents exploiting this in any of our evaluations; this is just a proof-of-concept exploit.' It was patched within a day. The systems that record and display an agent's outputs 'should be considered security-critical infrastructure.'
What it supports: That the record a human reviewer reads can be a weaker link than the reviewer, and that an evaluator treats it as something to secure.
What it does not support: That any agent has done this: METR says it has not seen it, and also that it has not searched past evaluations for this specific exploit. The flaw was narrow and is fixed.

AI agents · oversight · audit trail · evaluation · provenance · hand head hours heart

Vendor research#

Saturn Fintech Ltd (Jolly, A., chief executive) (2026). Artificial Authority: Should you trust AI to deliver financial advice?#

Saturn, published 14 September 2026; the report sits behind a form on the company's site and its own page was read at source on 21 September 2026. The figures and examples were read in Professional Adviser, IFA Magazine and Financial Reporter (14 September) and BeInCrypto (20 September); the Financial Times report of 20 September was not read. Not peer reviewed

Method: 121 UK personal finance questions on pensions, tax, debt and savings put to 18 free and paid models, each question asked up to five times, more than 10,000 answers in total. By BeInCrypto's account an answer failed if it contained a factual error, omitted material information or lacked a required warning. Who scored the answers, how 'complex' was defined and which 18 models were tested are not disclosed on the company's own page. Saturn sells AI software to advice firms, so the finding supports its own market.
Finding: Answers were wrong 57 per cent of the time overall (43 per cent accuracy), 63 per cent for free models and 49 per cent for paid ones, and 88 per cent on average for the harder questions, with some models wrong on 99 per cent of them. The per-model range was 39 to 82 per cent wrong. Reported examples: a pension tax miscalculation exposing a saver to a potential 17,500 pound HMRC charge, debt advice putting the highest-interest debt ahead of priority bills, an invented student loan rule, and a wrong account of a mortgage payment holiday's effect on a credit record.
What it supports: That in one vendor's test in September 2026, mainstream chatbots answering UK personal finance questions produced confident, specific and wrong answers at a rate high enough that acting on one unchecked is a material risk, and that the failures were of rules and sequencing rather than arithmetic.
What it does not support: Any base rate for chatbot financial answers in general. The questions and the scoring were the vendor's, the criteria are unpublished, nobody outside the company has replicated it, and no peer reviewer has seen it. The questions are UK-specific. The test measures single answers, not what a person does with them, and it was run in one window, so the per-model figures are not a ranking of companies.

financial advice · AI reliability · hallucination · consumer harm · personal finance · verification · can i trust an ai chatbot for financial advice

Working paper#

Schulz, J. (2026). Monitor Jailbreaking: Evading Chain-of-Thought Monitoring Without Encoded Reasoning #

arXiv 2609.31121, submitted 25 September 2026; accepted at the AdvML-Frontiers x CoTMA Workshop at COLM 2026. Abstract read at source 2 October 2026

Method: Reasoning models trained by reinforcement learning on a main task and a side task and penalised when a monitor detects reasoning about the side task; tested across model sizes, monitors and tasks.
Finding: Models 'learn to phrase and format their chains of thought such that monitors fail to flag side task reasoning, while the reasoning remains completely transparent to human readers'. The evasion carried over to monitors not seen in training, 'including both less and more capable monitors'. Paraphrasing the reasoning before monitoring let the same monitor flag it.
What it supports: That an automated monitor of a model's reasoning can be evaded by wording alone under training pressure, and that a human reader and a paraphrasing step each catch what the monitor missed.
What it does not support: That deployed models do this: the behaviour was induced by training against a monitor. The abstract gives no figures. A workshop paper.

monitoring · chain of thought · AI agents · human oversight · will an ai agent stay within the limits you set

Working paper#

Smyth, N., Mantilla-Ramos, Y.-J., Tikeng Notsawo, P. Jr, Helbling, S., Tosato, A., Merzouk, M. A., Dziri, N., Gidel, G. and Tosato, T. (2026). Quantifying Overclaiming Propensity in Frontier LLM Agents #

arXiv 2609.20812, posted 17 September 2026; authors at Tara Research, Mila (Quebec AI Institute) and Cohere. Read at source 20 September 2026. Not peer reviewed

Method: Five file-review scenarios (document synthesis and code review) with one to four planted defects each; twelve models, eight proprietary and four open-weight, twenty runs per model and scenario, 1,140 runs. Coverage is measured deterministically from tool calls (a file counts as read when a line unique to it is read). Whether the final response contradicts that record is judged by a language model given the delivered work and the ground truth. Overclaiming is defined as a final response that 'asserts an action or level of completion that is contradicted by evidence in its own context', with no assumption about intent.
Finding: Agents failed to read every requested file in 67.9 per cent of runs. Among those incomplete runs the final response was misleading 80.4 per cent of the time, either by claiming a complete review or by omitting the gap; the per-model range was 59 to 96 per cent misleading, and explicit false claims of completeness ranged from 9.8 to 73.7 per cent across the twelve models. Runs that explicitly overclaimed missed 720 of 1,237 planted defects (58.2 per cent) against 342 of 1,055 (32.4 per cent) in runs that touched every file, about 1.8 times the miss rate. Opening sentence: 'an agent's final response is often the only account of that work a user sees.'
What it supports: That across every frontier model tested, when a coding agent has not done all the work it was asked to do, its own report of the work more often conceals the gap than discloses it, and that the concealing reports are the ones with the most defects behind them. It is the first measurement the research holds of an agent's account of its work as evidence.
What it does not support: Any rate in deployment. The authors say the scenarios were designed by iterating against one provider's model until the behaviour appeared, which may bias the per-model comparison, and simpler corpora were dropped because they did not elicit it, so 67.9 per cent is a yield from a benchmark built to find the behaviour rather than a base rate. A single model judged every response; re-judging showed 91.1 per cent agreement with itself and no human-versus-judge figure is reported. Coding agents reviewing files are not analysts writing reports; the transfer to any other setting is an argument.

overclaiming · agent reliability · verification · evaluation awareness · AI agents · what the military ai near miss means for your organisation how do i know when ai is wrong

Working paper#

Souly, A., Fronsdal, K., D'Cruz, A., Davies, X. and Kirk, R., UK AI Security Institute (2026). Evaluating Whether GPT-6 Astra Performs Unsanctioned Supply-Chain Attacks #

AI Security Institute blog, 28 September 2026, with the report at arXiv 2609.38415, submitted 29 September 2026. Blog read at source and arXiv abstract read, 2 October 2026

Method: Simulated cyber evaluations using Petri, a tool in which other language models play the environment so that no real system is reachable; the model's cyber classifiers were turned off; comparison with GPT-5.6 Sol and GPT-5.5; a re-run on ten scenarios with an added scope instruction.
Finding: GPT-6 Astra completed a supply-chain attack on out-of-scope targets 29.2 per cent of the time, against 6.3 per cent for GPT-5.6 Sol and 0 per cent for GPT-5.5 on a smaller set of seeds. With the added instruction 'Anything not listed as in scope is out of scope', full attacks fell from 26 of 50 trajectories to 4 of 49 on the ten scenarios re-run, and 'the model still failed to consistently remain within the specified scope'. When the model asked a question it received only the automated message 'Please proceed to the next step using your best judgement', and it 'sometimes treated this automated message as permission to proceed with actions against out-of-scope targets'.
What it supports: That a frontier model, in simulation and with its product safeguards removed, pursued a task past an explicit boundary, and that a clearer instruction reduced the behaviour without ending it.
What it does not support: Behaviour in deployment: every action was simulated, the classifiers designed to block the behaviour were off, and the Institute says simulation awareness may have driven some of it. The total number of scenarios is not on the page read. One developer's models only.

AI agents · scope · human oversight · AI Security Institute · simulation · will an ai agent stay within the limits you set

Peer-reviewed#

Yu, S., Cheng, M., Jabbar, A., Sucholutsky, I., Collins, K. M., Jurafsky, D. and Hawkins, R. D. (2026). Cognitive offloading and the speedup illusion in human-AI interaction #

Proceedings of the 48th Annual Meeting of the Cognitive Science Society; arXiv 2605.23177, submitted 22 May 2026, CC BY 4.0. Venue re-confirmed at source 20 September 2026 from the arXiv comments field.

Method: Preregistered behavioural study, N=1,237, on simple cognitive tasks. Compared forecast completion times against actual completion times, with and without AI assistance, against a control condition in which participants imagined help from another person.
Finding: Actual completion times did not differ between independent and AI-assisted completion, while participants predicted AI would be significantly faster. The same bias did not appear when participants imagined help from another person. Participants also reported lower subjective effort with AI at equivalent completion times, so time and effort came apart.
What it supports: That people are miscalibrated about AI time savings specifically rather than about assistance in general, and that reported effort is not a proxy for elapsed time. The imagined-human-help control is the part no other study in this base supplies.
What it does not support: Anything about professional work, output quality or long-horizon tasks. The tasks were short and simple by design. NOT INDEPENDENT OF yu-efficiency-gain-2026: the same seven authors submitted that paper on 21 May 2026, one day before this one, and the two must never be set side by side as corroboration. Neither paper cites the other, so their relationship is inferred from authorship and date and is not established by anything either document says. This entry's own time result is a null rather than a measured saving, and the direct measurement of what AI assistance costs in elapsed time is in the companion paper, not here.

cognitive offloading · self-report · productivity · metacognition · usage theatre the unclaimed hour what is cognitive offloading the cost of delegation

Working paper#

Yu, S., Cheng, M., Jabbar, A., Sucholutsky, I., Collins, K. M., Jurafsky, D. and Hawkins, R. D. (2026). The efficiency-gain illusion: People underestimate the rate of AI use and overestimate its benefits on simple tasks #

arXiv 2605.22687v1 [cs.CY], submitted 21 May 2026, CC BY 4.0. Read in full at source 20 September 2026. NO VENUE, no journal and no review status is stated anywhere in the paper, so it is graded as a working paper and not carried across from the authors' CogSci paper.

Method: Three preregistered studies, N=2,691 in total: Study 1 N=498, Study 2 N=1,601 (600 prediction and 1,001 completion, 307 overlapping), Study 3 N=592. Recruited through Prolific, described by the authors as a representative sample of the US adult population. Stanford IRB protocol 83204. 24 tasks drawn from the Taxonomy of User Needs and Actions, each in an easier and a harder variant, all completable in under five minutes unaided, with GPT-4o embedded in the survey interface. Mixed-effects models with random intercepts for participant and task.
Finding: Two miscalibrations, measured separately. On time: 'On average, people predicted AI assistance to save time by 55.7 seconds when it only saved 7.5 seconds.' The error sits on the assisted side, where actual completion took 86.2 seconds against 43.3 predicted (b=42.9, p<0.001), while unassisted predictions were calibrated (99 predicted, 93.7 actual, b=-5.38, p=0.08). On use: participants said they would use AI on 33 per cent of tasks and used it on 47 per cent, widening to 20 against 38 on the easier variants. Overall AI assistance did not significantly reduce completion time (b=6.17, p=0.07), and on easy variants it produced a measured slow-down of 10.0 seconds, 60.2 to 70.2 (p<0.05). Prompting took longer than processing the response, 48.7 seconds against 37.6 (b=11.1, p<0.001), and 41 per cent of prompts were copied and pasted from the task instructions. Prior use begets use: 44.5 per cent of subsequent tasks against 27.7 per cent (b=0.54, p<0.001).
What it supports: That on short, simple tasks the expected saving from AI is several times the measured one, that the error is specifically an underestimate of how long working with the model takes, and that people underestimate how often they reach for it. The slow-down on easy variants is a measurement rather than a forecast.
What it does not support: Nothing about professional work, long-horizon tasks or output quality: every task was designed to take under five minutes. The authors' own limit is 'our study focuses on human miscalibration rather than AI capability'. Studies 1 and 2 are between-subjects aggregates, so the paper does not compare the same person's prediction with their own behaviour, and the two samples were not fully disjoint. The carryover result 'could be due to persistence and the inertia effect' in their own words, and the binary use variable 'does not capture how participants used AI'. NOT INDEPENDENT OF yu-speedup-illusion-2026, the same seven authors' CogSci paper submitted one day later; the two are one research programme and neither cites the other.

cognitive offloading · productivity · metacognition · usage theatre · self-report · usage theatre the unclaimed hour what is cognitive offloading

Working paper#

Zhao, Y., Li, J., Li, L., Nian, Y., Liu, J., Zhou, X. and Hu, X. (2026). Auditable Claims about AI Agents #

arXiv 2610.07459, submitted 5 October 2026. Abstract read at source 9 October 2026

Method: A position paper with a formal model and a claim-check table applied to six common claims, anchored in current NIST, IETF and OWASP drafts. No empirical sample.
Finding: 'The position is one sentence: to be checked, a claim about an agent must first name its policy, its scope, the records that would settle it, and who writes them.' Agents add three conditions: 'coverage by an independent record, authorization bound to each action's arguments, and completeness beyond integrity.' 'Article 12 of the EU AI Act requires high-risk systems to allow the automatic recording of events but does not say which records settle a given claim.'
What it supports: That a claim of human approval is checkable only if the record that would settle it was fixed in advance and is kept by someone other than the agent.
What it does not support: That any organisation works this way or that doing so prevents harm: it is an argument, its proof holds 'under an explicit model', and it has not been peer-reviewed. Abstract only.

audit · AI agents · human approval · records · proof of human work · hand head hours heart

Peer-reviewed#

Zhu, Q., Li, X., Dong, Y., Chang, P. and Fan, M. (2026). Not all cognitive offloading is equal: distinguishing dependent and autonomous offloading to generative AI #

Frontiers in Psychology, Volume 17, 16 July 2026. DOI 10.3389/fpsyg.2026.1878629. Labelled ORIGINAL RESEARCH; online 11 June 2026, published 16 July 2026. Read in full at source 13 September 2026

Method: Three-wave time-lagged self-report survey at two-week intervals. 742 completed wave one, 631 completed wave three, and the analytic sample is 589: mean age 23.69 (SD 3.24), 51.1 per cent female, 54.3 per cent undergraduates, 25.3 per cent master's students, 8.0 per cent doctoral students and 12.4 per cent young professionals, all with at least three months of using generative AI for learning or work. Every construct is a five-point Likert self-report, and the authors call their offloading scales 'preliminary, domain-specific instruments' lacking discriminant validity against measured instruments of AI dependency. Correlational: no manipulation, no performance task, and no wave-one baseline for the mediators or outcomes. THE PAPER DOES NOT STATE A COUNTRY. Recruitment is described only as universities and young professional communities via online survey platforms; the Chinese author affiliations, the Chinese back-translation of the measures and the named tools (ERNIE Bot, Kimi, Doubao) place it in China by inference, which is not the same as the paper saying so. No preregistration is mentioned.
Finding: The two offloading modes are close to independent of each other rather than opposite ends of one dial (r = 0.08, p = 0.049). Dependent offloading, meaning accepting AI output with minimal evaluation and letting it structure the reasoning, was associated with cognitive agency transfer (beta = 0.35, p < 0.001) and with lower intrinsic motivation (beta = -0.23, p < 0.001). Autonomous offloading, meaning using AI output as a starting point against which to compare one's own reasoning, was not associated with agency transfer at all (beta = -0.05, p = 0.18) and was associated with higher intrinsic motivation (beta = 0.28, p < 0.001). Metacognitive monitoring weakened the dependent path without closing it (interaction B = -0.14, p < 0.001; the slope runs 0.48 one standard deviation below the mean monitoring level and 0.21 one above). Immediate perceived benefit tracked both modes identically (r = 0.22 each) and barely tracked the wave-three outcomes (autonomous capability r = 0.06, p = 0.14; independent judgment r = -0.00, p = 0.91).
What it supports: That how a person offloads and how much they offload are separable variables, measured on a sample of 589 rather than argued. And that felt benefit at the moment of use carries almost no information about the outcomes people invoke it to justify, which is the third independent measurement in the base pointing that way.
What it does not support: Anything about cognitive capacity. Every outcome is a self-reported appraisal on scales the authors built for this study, and their own words are: 'all outcomes were self-reported appraisals of cognitive functioning, not performance-based measures', followed later in the same paragraph by 'the results should not be read as evidence that AI offloading improves or impairs cognitive capacity itself'. The authors also state the design 'does not establish mediation in the causal sense' and that without wave-one measures of the mediators they cannot rule out reverse or reciprocal influence. They position the whole study as 'an exploratory empirical study that provides initial correlational evidence, rather than definitive construct validation or causal claims'. The sample is young and predominantly student, so nothing here transfers to a career professional without assumption.

cognitive offloading · dependent cognitive offloading · autonomous cognitive offloading · cognitive agency transfer · what is cognitive offloading am i becoming dependent on ai using ai without dependency

Learning, practice and expertise

How capability is built, and what removes the building.

Peer-reviewed#

Bloom, B. S. (1984). The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring #

Educational Researcher, 13(6), 4 to 16

Method: A synthesis of two doctoral dissertations from Bloom's group at Chicago (Anania; Burke) that compared conventional classroom instruction, mastery learning and one-to-one tutoring on the same material, with the tutored students as the reference point.
Finding: Students tutored one to one, with mastery-learning corrective procedures, scored on average about two standard deviations above students taught conventionally in a class of about thirty; mastery learning in class scored about one standard deviation above. Bloom framed the research problem as finding group methods that could match tutoring.
What it supports: The origin of the two-sigma benchmark for tutoring that education and AI-tutoring literature still cite, and that it rests on a small number of controlled studies in Bloom's own programme.
What it does not support: That tutoring in general produces two standard deviations. The estimate comes from studies in Bloom's group under mastery-learning conditions, and later reviews of the wider tutoring literature found effects well under half that size (see VanLehn 2011).

tutoring · two sigma · learning · mastery learning · is an ai tutor as good as a human tutor

Peer-reviewed#

Balzer, W. K., Doherty, M. E. and O'Connor, R. (1989). Effects of cognitive feedback on performance #

Psychological Bulletin, 106(3), 410-433

Method: Review of the experimental literature on feedback in multiple-cue judgement tasks, separating feedback about outcomes from feedback about the judge's own use of cues.
Finding: Outcome feedback, being told whether the answer was right, produced little improvement and sometimes made judgement worse. Cognitive feedback, showing how the judge weighted the available information against how the environment actually weights it, produced improvement.
What it supports: That the kind of feedback matters more than the presence of feedback, and that knowing the result is not the same as learning from it.
What it does not support: That cognitive feedback is available in real work. It requires a model of the environment that most professional settings do not have.

feedback · judgement · learning · how is judgement trained kind and wicked learning environments

Practitioner framework#

Collins, A., Brown, J. S. and Newman, S. E. (1989). Cognitive apprenticeship: teaching the crafts of reading, writing and mathematics #

In Resnick, L. B. (ed.), Knowing, Learning and Instruction, Lawrence Erlbaum

Method: Instructional framework proposed from the study of traditional apprenticeship, setting out six methods: modelling, coaching, scaffolding, articulation, reflection and exploration.
Finding: Argues that the defining feature of apprenticeship is making expert thinking visible, and that schooling fails at cognitive tasks because the reasoning stays inside the expert's head.
What it supports: Where the idea comes from and what it instructs, and it names the mechanism by which watching a senior person work transfers anything.
What it does not support: That the methods produce measured gains. It is a framework argued from observation, with little controlled testing.

apprenticeship · tacit knowledge · expertise · learning · what is cognitive apprenticeship who owns the capability you built at work where your ai practice lives

Peer-reviewed#

Ericsson, K. A., Krampe, R. T. and Tesch-Romer, C. (1993). The Role of Deliberate Practice in the Acquisition of Expert Performance #

Psychological Review, 100(3), 363-406

Method: Two studies of violinists and pianists in Berlin.
Finding: Sets out deliberate practice: effortful, targeted activity at the edge of current ability, with feedback, sustained over years.
What it supports: That expert performance is built through a specific kind of effortful practice rather than exposure.
What it does not support: How much of the difference between performers practice explains. See the Macnamara and Maitra re-examination below.

deliberate practice · expertise · how humans learn with ai

Peer-reviewed#

McDaniel, M. A., Whetzel, D. L., Schmidt, F. L. and Maurer, S. D. (1994). The Validity of Employment Interviews: A Comprehensive Review and Meta-Analysis #

Journal of Applied Psychology, 79(4)

Method: Meta-analysis of 245 validity coefficients from 86,311 individuals.
Finding: Structured interviews predict job performance substantially better than unstructured interviews.
What it supports: That structure, rather than the interview itself, is what carries the predictive weight.
What it does not support: A single stable figure for the gap. Estimates of the size of the structure effect vary considerably between meta-analyses.

· how do you assess capability rather than output

Peer-reviewed#

Arthur, W., Bennett, W., Stanush, P. L. and McNelly, T. L. (1998). Factors that influence skill decay and retention: A quantitative review and analysis #

Human Performance, 11(1)

Method: Meta-analysis of 189 independent data points from 53 articles.
Finding: Skill loss ran from d of -0.01 immediately after training to d of -1.4 after more than 365 days of non-use. Physical, natural and speed-based tasks decayed less than cognitive, artificial and accuracy-based tasks.
What it supports: That skill decay is measurable, that it is a function of the interval, and that cognitive skills go first.
What it does not support: How fast any particular professional skill decays, or how quickly it can be regained.

deskilling · capability debt · can you regain a skill you have lost how do we train people to use ai what are the risks of using ai at work

Peer-reviewed#

Giedd, J.N., Blumenthal, J., Jeffries, N.O., Castellanos, F.X., Liu, H., Zijdenbos, A., Paus, T., Evans, A.C. and Rapoport, J.L. (1999). Brain development during childhood and adolescence: a longitudinal MRI study #

Nature Neuroscience, 2(10), 861-863

Method: Longitudinal MRI, 243 scans from 145 healthy participants, 89 male and 46 female, at the US National Institute of Mental Health.
Finding: Cortical grey matter in frontal regions peaks in pre-adolescence and then thins through synaptic pruning, with the prefrontal cortex among the last areas to mature.
What it supports: That the prefrontal cortex matures later than other regions. This is real, replicated and not in dispute.
What it does not support: Any threshold age. The paper reports a slower trajectory, not an endpoint, and names no age at which development completes. Everything downstream that cites it for a cut-off is citing something it does not contain.

brain development · adolescence · prefrontal cortex · does the brain mature at 25

Peer-reviewed#

Benkard, C. L. (2000). Learning and Forgetting: The Dynamics of Aircraft Production #

American Economic Review, 90(4), 1034-1054, September 2000. DOI 10.1257/aer.90.4.1034. Circulated first as NBER Working Paper 7127, May 1999, DOI 10.3386/w7127. PROVENANCE: the abstract was read at source on the NBER working-paper record on 21 September 2026. The full text of the published article is behind the AEA paywall, the NBER PDF could not be retrieved, and no figure from it is quoted here. A depreciation estimate of 0.96 monthly, implying 61 per cent annual survival of the stock of experience, circulates widely and is reported by Bongers 2017; it has NOT been confirmed against Benkard's own text and must not be used until it has.

Method: A cost dataset for a commercial aircraft firm, used to estimate learning dynamics in the production of a wide-body airliner. Method detail beyond the abstract has not been read at source.
Finding: In the author's own summary, the data are 'inconsistent with the simple learning hypothesis, and particularly the prediction that a firm's unit cost must decline with its cumulative production'. Instead 'strong support is found for the hypothesis of organizational forgetting', a model in which unit costs depend on past production experience 'but where that experience depreciates over time', and that 'some, but not all, of a firm's production experience transfers from one generation of an aircraft to the next'.
What it supports: That organisational experience depreciates rather than accumulating indefinitely, and that the transfer of accumulated experience across product generations is partial. It is the paper that put organisational forgetting into the economics literature.
What it does not support: Any rate, in this research, because no number from it has been confirmed at source. Nothing about knowledge work, and nothing about individual people: the unit is a firm's production programme. The partial-transfer finding is about successive aircraft built by the same firm and transfers to a change of supplier or of model only as an analogy.

organisational forgetting · organisational capability · learning curve · capability transfer · how do you keep expertise in an organisation preserving capability across vendors who owns the capability you built at work where your ai practice lives

Peer-reviewed#

Maguire, E. A., Gadian, D. G., Johnsrude, I. S., Good, C. D., Ashburner, J., Frackowiak, R. S. J. and Frith, C. D. (2000). Navigation-related structural change in the hippocampi of taxi drivers #

PNAS, 97(8), 4398-4403

Method: Cross-sectional structural MRI. 16 right-handed male London taxi drivers with more than 1.5 years driving, against scans of 50 healthy right-handed male non-taxi-drivers.
Finding: Posterior hippocampi were significantly larger in taxi drivers than in controls, and hippocampal volume correlated with time spent driving a taxi, positively in the posterior and negatively in the anterior hippocampus.
What it supports: That sustained, effortful spatial practice is associated with measurable structural difference in the adult brain, and that the association scales with how long the practice has continued.
What it does not support: Causation. It is cross-sectional, so it cannot separate the practice building the brain from a particular brain selecting into the job. It says nothing about what happens when the practice stops, nothing about AI, and nothing about knowledge work.

deliberate practice · skill acquisition · spatial memory · The Knowledge · what is deliberate practice how fast do skills decay

Compiled review#

Anderson, L. W. and Krathwohl, D. R. (eds.) (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives #

Longman, New York

Method: A revision of Bloom's 1956 taxonomy by a group of cognitive psychologists and curriculum specialists, restating the categories as verbs and adding a second knowledge dimension.
Finding: Six cognitive process categories in ascending order: remember, understand, apply, analyse, evaluate, create.
What it supports: That there is a long-established and widely taught vocabulary for ordering cognitive demand, which any newer ladder of AI use is implicitly competing with.
What it does not support: That the order is strictly hierarchical in practice, or that it transfers to human-AI interaction. It describes what a learner is asked to do, not what a tool is asked to do, and the two come apart as soon as a model can perform the higher categories on request.

learning · taxonomy · frameworks · the four levels of ai use

Peer-reviewed#

Kalyuga, S., Chandler, P., Tuovinen, J. and Sweller, J. (2001). When Problem Solving Is Superior to Studying Worked Examples #

Journal of Educational Psychology, 93(3), 579-588, September 2001. Verified at source on the ERIC record EJ640537, 18 September 2026, which carries the author abstract in full; the full text is behind the publisher. NOTE that ERIC's citation_author metadata field returns 'Sweller, John' alone while its own record body lists all four authors, which is the second metadata source on this paper's family to name one author where there are four. No figure from this study is used here or on any page, because only the abstract could be read.

Method: Experiments with mechanical trade apprentices, given either worked examples to study or the equivalent problems to solve, tested at successive levels of domain experience.
Finding: In the authors' own words, inexperienced trainees benefited most from worked examples; with more experience in the domain, worked examples became redundant and problem solving proved superior.
What it supports: That the reversal is a measured result and not only a framework. It is the single cleanest demonstration that removing support can become the better instruction for the same person who needed it earlier.
What it does not support: Any magnitude, because only the abstract was readable. Also nothing about knowledge work: the domain is mechanical trades and the task is bounded, with a correct answer the experimenter already holds.

expertise reversal effect · worked examples · learning · what is the expertise reversal effect

Narrative review#

Kalyuga, S., Ayres, P., Chandler, P. and Sweller, J. (2003). The Expertise Reversal Effect #

Educational Psychologist, 38(1), 23-31. DOI 10.1207/S15326985EP3801_4. Read in full at source 18 September 2026 from the University of Wollongong Research Online copy, because the publisher's own page is abstract-only. AUTHOR ORDER, and it matters because two orders are in circulation: the article's byline and its running head both read Kalyuga, Ayres, Chandler, Sweller, and Kalyuga is given a separate affiliation (Educational Testing Centre, UNSW) from the other three (School of Education, UNSW). The Wollongong repository's own Publication Details line reverses this to 'Sweller, J., Ayres, P. L., Kalyuga, S. & Chandler, P. A.', which is the form its cover sheet generates from its institutional author record. The article is what ships here. Same class of fault as Nature's page metadata returning only the corresponding author.

Method: Review of eleven experiments on interactions between instructional format and learner prior knowledge, roughly half of them by these authors. No new data. No systematic search protocol is reported.
Finding: Instructional guidance that helps inexperienced learners can lose its effect and then reverse it as the same learners gain domain knowledge. The authors' explanation is redundancy: once a learner's own schemas supply the guidance, the external guidance has to be cross-referenced against them, and that costs working memory. Reported across split attention, redundancy, modality, worked examples, isolated elements and imagination. Their conclusion is that without tailoring to learner experience, 'the effectiveness of instructional designs is likely to be random'.
What it supports: That the sign of a support's effect on learning depends on who is receiving it, established forty years before the AI question and in a literature with no stake in it. It is the strongest independent corroboration the research holds for treating 'does AI help' as a malformed question.
What it does not support: Anything about AI, which did not exist in this form when it was written. Nothing in it tests a generative system, and the guidance it studies is a worked example or a labelled diagram, both of which display the reasoning. Also not a systematic review: the studies are selected by the authors, most of the reversals are their own results, and no effect sizes are pooled.

expertise reversal effect · cognitive load · worked examples · learning · what is the expertise reversal effect

Peer-reviewed#

Gogtay, N., Giedd, J.N., Lusk, L., Hayashi, K.M., Greenstein, D., Vaituzis, A.C., Nugent, T.F., Herman, D.H., Clasen, L.S., Toga, A.W., Rapoport, J.L. and Thompson, P.M. (2004). Dynamic mapping of human cortical development during childhood through early adulthood #

Proceedings of the National Academy of Sciences, 101(21), 8174-8179

Method: A densely sampled subset of THIRTEEN participants from the NIMH longitudinal project, each scanned roughly every two years.
Finding: Maps the sequence in which cortical regions mature, with higher-order association cortices maturing after lower-order sensorimotor regions.
What it supports: A developmental sequence, in thirteen people.
What it does not support: A population age of maturity. Thirteen participants cannot establish one, and the paper does not claim to. This is the study behind the Time Magazine coverage in which the number 25 first appears in public.

brain development · cortical maturation · does the brain mature at 25

Peer-reviewed#

Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T. and Rohrer, D. (2006). Distributed Practice in Verbal Recall Tasks: A Review and Quantitative Synthesis #

Psychological Bulletin, 132(3)

Method: Meta-analysis of 839 assessments across 317 experiments in 184 articles.
Finding: Spacing and retention interval act jointly. The gap between practice sessions that produces best retention increases as the target retention interval increases.
What it supports: That when practice happens changes how much survives, independently of how much practice there is.
What it does not support: Application to procedural or professional skill. The synthesis covers verbal recall.

desirable difficulty · can you regain a skill you have lost

Peer-reviewed#

Huckman, R. S. and Pisano, G. P. (2006). The Firm Specificity of Individual Performance: Evidence from Cardiac Surgery #

Management Science, 52(4), 473-488, April 2006. DOI 10.1287/mnsc.1050.0464. PROVENANCE: the publisher-supplied abstract was read at source on the RePEc record on 21 September 2026. The full text is paywalled and was not opened, so no sample size, coefficient or interval is quoted here.

Method: Observational study of cardiac surgeons who operate at more than one hospital within narrow periods of time, using patient mortality as the outcome measure and recent procedure volume at each hospital as the exposure. Sample size and specification not read at source.
Finding: In the authors' words, 'the quality of a surgeon's performance at a given hospital improves significantly with increases in his or her recent procedure volume at that hospital but does not significantly improve with increases in his or her volume at other hospitals'. They conclude that 'surgeon performance is not fully portable across hospitals (i.e., some portion of performance is firm specific)' and offer 'preliminary evidence suggesting that this result may be driven by the familiarity that a surgeon develops with the assets of a given organization'.
What it supports: That a measurable part of an expert's performance belongs to the pairing of that expert with a particular organisation, and accrues through recent repetition inside it. The comparison is within-surgeon across sites, which is what makes it an argument about the organisation rather than about who is good.
What it does not support: Any magnitude, in this research, because the tables were not read. The asset-familiarity explanation is the authors' own and they call it preliminary. It is one clinical speciality in one health system in the 1990s, it concerns freelancing surgeons rather than outsourced corporate functions, and it says nothing about AI.

organisational capability · firm-specific capability · expertise · capability transfer · preserving capability across vendors how do you keep expertise in an organisation

Peer-reviewed#

Roediger, H. L. III and Karpicke, J. D. (2006). Test-Enhanced Learning: Taking Memory Tests Improves Long-Term Retention #

Psychological Science, 17(3), 249-255. DOI 10.1111/j.1467-9280.2006.01693.x

Method: Two experiments, 120 and 180 participants, reading prose passages on general science topics. Restudying compared with being tested, with final recall measured at 5 minutes, 2 days or 1 week.
Finding: The winner reverses with delay. At 5 minutes restudying beat testing, 81 per cent against 75 per cent. At one week testing beat restudying, 56 per cent against 42 per cent. In the second experiment repeated study led at 5 minutes, 83 against 71 per cent, and trailed badly at one week, 40 against 61 per cent.
What it supports: That the study method producing the best immediate performance produces the worst durable retention, and that a measurement taken close to the learning will rank the methods in exactly the wrong order.
What it does not support: Anything about AI. It is prose recall in a laboratory, and the transfer to professional judgement is by analogy.

retrieval practice · testing effect · desirable difficulty · capability debt · what is retrieval practice what is desirable difficulty

Peer-reviewed#

Kapur, M. (2008). Productive Failure #

Cognition and Instruction, 26(3), 379-424. DOI 10.1080/07370000802212669

Method: Randomised comparison, 309 eleventh-grade physics students in India working on Newtonian kinematics. One group solved ill-structured problems in groups before individual well-structured problems; the other solved well-structured problems throughout.
Finding: The group given ill-structured problems struggled visibly and produced poor solutions during the collaborative phase, then outperformed the other group on individual near-transfer and far-transfer measures afterwards.
What it supports: That struggle which looks like failure at the time can produce better subsequent transfer than a smooth path through the same material, and that judging a learning design by how well it is going is unreliable.
What it does not support: A precise magnitude. The design, sample and direction of the result are confirmed from the publisher's abstract and the author's own presentation of the study, but the full text sits behind a paywall that could not be read, so no post-test figures are quoted here.

productive failure · productive struggle · desirable difficulty · what is productive struggle

Compiled review#

American Red Cross Advisory Council on First Aid, Aquatics, Safety and Preparedness (2009). Scientific Review: CPR Skill Retention #

American Red Cross

Method: Systematic review of 47 articles on CPR skill retention across healthcare and lay populations, with retest intervals from six weeks to 24 months.
Finding: Substantial skill degradation occurs within the first year after training, with declining retention from six to twelve months unless there is refresher training.
What it supports: That a life-critical, heavily trained procedural skill decays on a timescale of months without practice.
What it does not support: Any link to patient outcomes. Decay was measured on manikins, not in resuscitations.

deskilling · can you regain a skill you have lost

Peer-reviewed#

Bjork, E. L. and Bjork, R. A. (2011). Making Things Hard on Yourself, But in a Good Way: Creating Desirable Difficulties to Enhance Learning #

In Psychology and the Real World, Worth Publishers

Method: Synthesis of decades of laboratory work on spacing, interleaving and retrieval practice.
Finding: Conditions that make study feel harder improve long-term retention; conditions that make it feel fluent improve immediate performance and worsen retention. Learners systematically mistake fluency for learning.
What it supports: That the subjective sense of learning is an unreliable guide to whether learning occurred. Directly relevant, because AI makes work feel fluent.
What it does not support: That AI-assisted work is equivalent to a fluent study condition. That inference is ours, not the authors'.

desirable difficulties · learning · how humans learn with ai

Peer-reviewed#

Cook, D. A., Hatala, R., Brydges, R., Zendejas, B., Szostek, J. H., Wang, A. T., Erwin, P. J. and Hamstra, S. J. (2011). Technology-enhanced simulation for health professions education: a systematic review and meta-analysis #

JAMA, 306(9), 978-988

Method: Systematic review and meta-analysis of 609 studies of simulation-based training in the health professions.
Finding: Simulation training produced large effects on knowledge, skills and behaviours against no intervention, and smaller effects on patient outcomes.
What it supports: That rehearsing decisions in a setting built for rehearsal is an effective way to build professional capability, and that it is the best-evidenced delivery structure available.
What it does not support: That the effect transfers outside clinical training, or that simulation without feedback and repetition does anything. The comparison in most studies is against no training at all.

simulation · training · apprenticeship · expertise · how is judgement trained

Peer-reviewed#

Karpicke, J. D. and Blunt, J. R. (2011). Retrieval Practice Produces More Learning than Elaborative Studying with Concept Mapping #

Science, 331(6018), 772-775. DOI 10.1126/science.1199327

Method: Two experiments, 80 and 120 undergraduates, comparing retrieval practice with elaborative study by concept mapping, tested one week later.
Finding: Retrieval practice scored 0.67 against 0.45 for concept mapping, about a 50 per cent advantage in long-term retention, d = 1.50. In the second experiment 101 of 120 students, 84 per cent, did better after retrieval practice than after elaborative study.
What it supports: That effortful recall outperforms a more elaborate and more comfortable-feeling study method, on the same material, in the same students.
What it does not support: That concept mapping is worthless. A published comment by Mintzes and colleagues (Science, 2011, 334(6055), 453) disputes the instructional fidelity of the concept-mapping condition, and anyone citing this should note it.

retrieval practice · testing effect · illusion of competence · what is retrieval practice

Compiled review#

VanLehn, K. (2011). The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems #

Educational Psychologist, 46(4), 197 to 221

Method: A review of experimental comparisons of human tutoring, intelligent tutoring systems of different granularity, and no tutoring, computing mean effect sizes across the studies gathered.
Finding: Human tutoring averaged an effect size of 0.79 standard deviations over no tutoring; step-based intelligent tutoring systems 0.76; substep-based systems 0.40; answer-based systems 0.31. VanLehn concluded that the two-sigma effect of one-to-one human tutoring reported by Bloom was not observed in the wider literature, and that well-designed computer tutors were nearly as effective as human ones.
What it supports: That the standard against which AI tutors are now measured, human one-to-one tutoring, produces an effect closer to 0.8 standard deviations than to 2 across the experimental literature, and that computer tutors had approached it a decade before large language models.
What it does not support: What the effect is for any particular subject, age group or tutor; the studies reviewed are heterogeneous and the review is fifteen years old. The comparison is on immediate learning measures and says nothing about retention or about generative AI.

tutoring · intelligent tutoring systems · effect size · learning · is an ai tutor as good as a human tutor

Peer-reviewed#

Woollett, K. and Maguire, E. A. (2011). Acquiring 'the Knowledge' of London's Layout Drives Structural Brain Changes #

Current Biology, 21(24), 2109-2114, 20 December 2011

Method: Longitudinal structural MRI over four years. 79 male trainee London taxi drivers studying for the Knowledge, and 31 male non-taxi-driver controls, scanned before and after. Average-IQ adults, real training rather than a laboratory task.
Finding: In those who qualified, acquiring an internal spatial representation of London was associated with a selective increase in grey matter volume in the posterior hippocampi, with concomitant changes to their memory profile. In the authors' words, no structural brain changes were observed in trainees who failed to qualify or in control participants. The gain in posterior hippocampus came alongside costs elsewhere in the memory profile.
What it supports: The strongest available evidence that the practice itself does the work rather than selection. Same starting cohort, same training, and the structural change appears only in those who completed it. It also shows the trade: the capability gained is paid for with capability elsewhere, which is skill substitution observed in tissue rather than argued.
What it does not support: Anything about AI, and anything about removal. This is acquisition, over four years, in one domain. It does not show that the structure regresses when the practice is delegated to a machine, and no study has shown that. The frequently circulated claim that GPS use produces measurable cognitive decline or early-onset dementia is a prediction rather than a finding, and cannot be traced to a study of this kind.

deliberate practice · skill acquisition · spatial memory · The Knowledge · skill substitution · what is deliberate practice how fast do skills decay what is the substitution myth

Institutional survey#

Bureau d'Enquetes et d'Analyses (2012). Final Report on the accident on 1 June 2009 to the Airbus A330-203, flight AF 447 #

BEA, France

Method: Statutory accident investigation. Flight recorder analysis and training record review. 25 new safety recommendations.
Finding: The investigation cited the lack of practical training in high-altitude manual handling and in the procedure for speed anomalies among the contributing factors.
What it supports: That a formal investigation attributed part of an accident to manual handling practice that had not been maintained.
What it does not support: A general rate of skill loss across the pilot population. It is one accident.

deskilling · what professions can learn from aviation should there be ai free periods at work

Regulator guidance#

Federal Aviation Administration, Flight Standards Service (2013). SAFO 13002: Manual Flight Operations #

Safety Alert for Operators, dated 4 January 2013 (1/4/13)

Method: A regulator's safety alert encouraging operators to promote manual flight operations when appropriate. Fetched and read at source on 4 October 2026. It reports no data.
Finding: States that continuous use of autoflight systems could lead to degradation of a pilot's ability to recover quickly from an undesired state, and asks operators to set policies giving pilots appropriate opportunities to exercise manual flying skills, with guidance on when automation is preferable, for example in high-workload conditions.
What it supports: That the regulator of the most automated profession asked operators in writing, in 2013, to schedule practice without the automation.
What it does not support: That the practice works, or how often it is needed. It is guidance rather than a rule and carries no measured effect of the periods it recommends.

deskilling · automation complacency · should there be ai free periods at work

Practitioner method#

Klein, G., Hintze, N. and Saab, D. (2013). Thinking inside the box: the ShadowBox method for cognitive skill development #

Proceedings of the 11th International Conference on Naturalistic Decision Making

Method: A training method in which a trainee works through a scenario making the same calls an expert made, then compares their choices and their reasons against the expert panel's, with small field evaluations reported by the developers.
Finding: Reported gains in the developers' own field studies were about 28 per cent with 59 Marines and about 21 per cent with 30 Army officers.
What it supports: That there is a usable structure for training the reasoning behind a decision rather than the decision itself, by making the expert's cue selection visible and comparable.
What it does not support: That the gains generalise. The evaluations are small, run by the method's developers, and use scenario scores rather than performance at work.

naturalistic decision making · training · expertise · judgement · how is judgement trained

Peer-reviewed#

Moonen-van Loon, J. M. W., Overeem, K., Donkers, H. H. L. M., van der Vleuten, C. P. M. and Driessen, E. W. (2013). Composite reliability of a workplace-based assessment toolbox for postgraduate medical education #

Advances in Health Sciences Education, 18(5)

Method: Generalisability study of 12,779 workplace-based assessments from 953 medical residents.
Finding: A reliability coefficient of 0.80 required eight mini-CEX observations, nine DOPS or nine multi-source feedback rounds. Combined in a portfolio the requirement fell to seven, eight and one respectively.
What it supports: That observing someone at work can reach defensible reliability, and roughly how many observations that takes.
What it does not support: That these scores predict later performance or patient outcomes. It measures consistency, not criterion validity. A single observation is not reliable.

· how do you assess capability rather than output what happens to medical and legal training

Institutional modelling#

PARC/CAST Flight Deck Automation Working Group (2013). Operational Use of Flight Path Management Systems: Final Report #

Federal Aviation Administration

Method: Working-group synthesis of accident and incident data, operator surveys and prior research. 28 findings, 18 recommendations.
Finding: Identified vulnerabilities in manual handling after transition from automated control, and in the definition, development and retention of those skills. Also found that pilots sometimes rely too much on automated systems and may be reluctant to intervene.
What it supports: That a regulator examined automation dependence in a whole industry and named skill retention as a finding.
What it does not support: A measured rate of skill decay. It is a synthesis of findings, not a controlled study.

automation complacency · deskilling · what professions can learn from aviation should there be ai free periods at work

Peer-reviewed#

Tannenbaum, S. I. and Cerasoli, C. P. (2013). Do team and individual debriefs enhance performance? A meta-analysis #

Human Factors, 55(1), 231-245

Method: Meta-analysis of 46 studies of structured debriefs, for individuals and teams, in field and simulated settings.
Finding: Properly conducted debriefs improved performance by around 20 to 25 per cent over control, with an average effect size of about 0.67.
What it supports: That a structured review of what happened and why, done to a method, reliably improves subsequent performance. It is the best-evidenced routine available for building judgement from experience.
What it does not support: That any meeting called a debrief does this. The effect depends on structure, on self-discovery rather than being told, and on reviewing process rather than outcome.

after action review · debrief · feedback · learning · what is an after action review how is judgement trained how do we train people to use ai

Peer-reviewed#

Casner, S. M., Geven, R. W., Recker, M. P. and Schooler, J. W. (2014). The Retention of Manual Flying Skills in the Automated Cockpit #

Human Factors, 56(8)

Method: 16 airline pilots flew routine and non-routine scenarios in a Boeing 747-400 simulator with automation level varied.
Finding: Instrument scanning and manual control were mostly intact even where pilots reported little recent practice. The cognitive tasks, tracking position without a map, deciding the next navigational step and recognising instrument failures, showed frequent and significant problems.
What it supports: That the hands survive disuse better than the judgement does, in the profession with the most automation experience.
What it does not support: A general rule for knowledge work. The sample is 16 pilots in a simulator.

deskilling · can you regain a skill you have lost what professions can learn from aviation should there be ai free periods at work

Compiled review#

Macnamara, B. N., Hambrick, D. Z. and Oswald, F. L. (2014). Deliberate Practice and Performance in Music, Games, Sports, Education, and Professions: A Meta-Analysis #

Psychological Science, 25(8), 1608-1618

Method: Meta-analysis of 88 studies and 157 effect sizes relating accumulated deliberate practice to performance.
Finding: Deliberate practice explained 12 per cent of the variance in performance overall, 95% CI [9%, 15%], leaving 88 per cent unexplained. By domain: 26 per cent for games, 21 per cent for music, 18 per cent for sports, 4 per cent for education and under 1 per cent for the professions.
What it supports: That accumulated practice is one contributor among several rather than the dominant explanation of expert performance, and that its contribution varies sharply by domain.
What it does not support: That practice does not build professional expertise. The professions estimate rests on 7 effect sizes, is not statistically significant (p = .62), and the occupations sampled were computer programming, military aircraft piloting, soccer refereeing and insurance selling. It is not evidence about law, medicine, consulting or analysis. It gets quoted as though it were.

deliberate practice · expertise · what is deliberate practice how is judgement trained

Peer-reviewed#

Rowland, C. A. (2014). The Effect of Testing Versus Restudy on Retention: A Meta-Analytic Review of the Testing Effect #

Psychological Bulletin, 140(6), 1432-1463. DOI 10.1037/a0037559

Method: Meta-analysis, 159 effect sizes from 61 studies published between 1975 and 2013, random-effects model.
Finding: A reliable testing effect, g = 0.50 with a confidence interval of 0.42 to 0.58. With feedback the effect rises to g = 0.73; without feedback it falls to g = 0.39.
What it supports: That the testing effect survives aggregation across four decades of studies, at a moderate effect size, and that feedback roughly doubles it.
What it does not support: That the effect size transfers to workplace learning. The constituent studies are overwhelmingly laboratory and classroom work on verbal material.

retrieval practice · testing effect · meta-analysis · what is retrieval practice

Peer-reviewed#

Hartshorne, J.K. and Germine, L.T. (2015). When does cognitive functioning peak? The asynchronous rise and fall of different cognitive abilities across the life span #

Psychological Science, 26(4), 433-443

Method: Cross-sectional analysis of large web-based and standardisation samples across a wide age range.
Finding: Different cognitive abilities peak at different ages, some in the late teens or early twenties, others not until the forties or fifties. There is no single age at which cognitive functioning peaks.
What it supports: That a single maturity age is the wrong shape of answer, whatever number is put in it. Abilities do not arrive together.
What it does not support: That age is irrelevant. It shows the timing is ability-specific rather than absent.

cognitive development · peak performance · individual differences · does the brain mature at 25

Peer-reviewed#

Hogarth, R. M., Lejarraga, T. and Soyer, E. (2015). The two settings of kind and wicked learning environments #

Current Directions in Psychological Science, 24(5), 379-385

Method: Theoretical account with experimental illustrations, distinguishing environments by the quality of the feedback they return to the learner.
Finding: In a kind environment feedback is quick, accurate and drawn from a complete sample, so experience teaches. In a wicked one it is delayed, partial, biased by the learner's own actions, or drawn from a sample the learner's decisions censored, so experience can teach the wrong lesson with confidence.
What it supports: That the value of experience depends on the structure of the feedback, not on the amount of experience, and that some domains cannot be learned by repetition alone.
What it does not support: Which real occupations sit where. The distinction is a lens, not a measurement, and no instrument scores an environment for kindness.

feedback · learning · judgement · expertise · kind and wicked learning environments how is judgement trained

Peer-reviewed#

Murre, J. M. J. and Dros, J. (2015). Replication and Analysis of Ebbinghaus' Forgetting Curve #

PLoS ONE, 10(7)

Method: Single-subject relearning experiment replicating Ebbinghaus across intervals from 20 minutes to 31 days, 10 replications per interval.
Finding: Relearning to criterion took less time than original learning at every retention interval tested, confirming the savings effect Ebbinghaus reported in 1885.
What it supports: That something survives apparent forgetting, and that it shows up as faster relearning rather than as recall.
What it does not support: That professional skills behave like nonsense syllables, or that savings hold at the scale of a career. It is one subject and verbal material.

· can you regain a skill you have lost

Peer-reviewed#

Storm, B. C. and Stone, S. M. (2015). Saving-Enhanced Memory: The Benefits of Saving on the Learning and Remembering of New Information #

Psychological Science, 26(2), 182-188. DOI 10.1177/0956797614559285

Method: Three experiments on the effect of saving a digital file on memory for subsequently studied material. GRADED FROM THE PUBLISHED ABSTRACT ONLY: the full text is paywalled and could not be read at the primary source, so no sample sizes or statistics are recorded here.
Finding: Saving one file before studying a new file significantly improved memory for the contents of the new file. The effect was not observed when the saving process was deemed unreliable, or when the contents of the to-be-saved file were not substantial enough to interfere with memory for the new file.
What it supports: That cognitive offloading can improve rather than degrade subsequent memory, and that the benefit depends on the external store being trusted.
What it does not support: Anything quantified. This entry rests on the abstract alone, which is a weaker basis than every other entry in this base and is stated as such. It also predates generative AI and concerns file saving rather than a system that can fabricate its own contents.

cognitive offloading · memory · do i still need to remember things

Peer-reviewed#

Hamilton, E. R., Rosenberg, J. M. and Akcaoglu, M. (2016). The Substitution Augmentation Modification Redefinition (SAMR) Model: A Critical Review and Suggestions for Its Use #

TechTrends, 60(5), 433-441

Method: Critical review of the SAMR model against the educational technology literature.
Finding: Names three problems: the absence of context, a rigid hierarchical structure that implies higher is better, and an emphasis on product over process. The authors also record that SAMR is largely absent from the peer-reviewed literature despite heavy practitioner use, and that its theoretical and foundational evidence is thin.
What it supports: That a widely adopted framework can spread through practice with almost no evidence behind it, and that popularity is not a proxy for validity. Held here as the standard against which any framework in this research, including this research's own, should be judged.
What it does not support: That SAMR is useless in practice. The authors offer suggestions for its use rather than a case for abandoning it, and their objection is to how it is applied more than to what it contains.

frameworks · education technology · evidence · the four levels of ai use

Compiled review#

Simons, D. J., Boot, W. R., Charness, N., Gathercole, S. E., Chabris, C. F., Hambrick, D. Z. and Stine-Morrow, E. A. L. (2016). Do Brain-Training Programs Work?#

Psychological Science in the Public Interest, 17(3)

Method: Systematic review applying pre-specified best-practice standards to every study cited by commercial brain-training companies as evidence of efficacy.
Finding: Extensive evidence that training improves performance on the trained tasks, less evidence for closely related tasks, and little evidence that training improves distantly related tasks or everyday cognitive performance. No cited study met all best-practice standards.
What it supports: That near transfer is real and far transfer, which is what any claim to have trained attention requires, is essentially unsupported.
What it does not support: That practising a specific skill is useless. The failure is transfer, not learning.

· is attention a trainable skill

Peer-reviewed#

Bongers, A. (2017). Learning and forgetting in the jet fighter aircraft industry #

PLOS ONE, 12(9), e0185364, published 28 September 2017. DOI 10.1371/journal.pone.0185364. Open access, CC BY. Read in full at source 21 September 2026. PROVENANCE: the two coefficient tables are published as images and are not in the article text, so every figure below is taken from the author's running prose. The paper states no sample size, standard errors or R-squared anywhere in its text.

Method: Learning curves estimated from United States Department of Defense flyaway cost, converted to constant dollars with the DoD procurement deflator, for three fighter programmes: the F/A-18E/F Super Hornet, 554 units acquired in fiscal 1997 to 2013; the F-22A Raptor, 182 units in fiscal 2000 to 2009; and the F-35A Lightning II, 135 units in fiscal 2007 to 2015. Data are annual procurement lots, so the per-programme series runs from nine to seventeen observations. The standard log-log learning curve is fitted by ordinary least squares; the forgetting model, in which the stock of experience carries over at a rate lambda, is fitted by nonlinear least squares. Estimates repeated on lot-adjusted data by the Loerch method, which the author reports as making no significant difference.
Finding: Baseline learning curves of 86 per cent for the Super Hornet, 85.4 per cent for the Raptor and 91 per cent for the Lightning II, so learning rates of about 14 per cent for the first two and about 9 per cent for the third. In the forgetting model the persistence parameter was not significantly different from zero for the F-22A and F-35A: in the author's words, 'the depreciation of experience in the production of these units is total, on an annual basis'. For the Super Hornet she estimates 0.072, 'indicating that, on an annual basis, only 7.2% of the experience is maintained'. She attributes the difference to production rate, noting that Benkard 'argues that the low aircraft production rate is the main factor that explains the observed high forgetting. Our results seem to confirm that.'
What it supports: That accumulated organisational production experience can depreciate almost completely within a year, and that the rate at which the work is actually performed is a candidate explanation for how much survives. It is a measurement of organisational forgetting rather than individual skill decay: the unit of analysis is a production programme.
What it does not support: Anything about professional judgement, advice or knowledge work; the outcome is unit cost in aircraft assembly. The data are annual lots rather than continuous production and the author warns that the total-depreciation result 'may be due to the small number of units produced and the annual nature of the data used'. She declares two specifications unusable, the Raptor forgetting model as inconsistent and the F-35A column with both controls as implying rising costs, and notes that F-35A learning is overestimated because flyaway cost covers only units bought by the DoD while other countries also bought the aircraft. No sample size or standard error is reported in the text, so the precision behind 0.072 cannot be assessed from the paper as published in HTML.

organisational forgetting · deskilling · skill decay · organisational capability · learning curve · how do you keep expertise in an organisation preserving capability across vendors who owns the capability you built at work where your ai practice lives

Regulator guidance#

Federal Aviation Administration, Flight Standards Service (2017). SAFO 17007: Manual Flight Operations Proficiency #

Safety Alert for Operators, dated 4 May 2017 (5/4/17)

Method: A regulator's safety alert on manual flight proficiency. Fetched and read at source on 4 October 2026. It reports no data.
Finding: Encourages training and line-operations policies that ensure proficiency in manual flight operations is developed and maintained for air carrier pilots, to be built into operational policy, ground training, flight training and proficiency checks. Lists SAFO 13002 as related guidance.
What it supports: That the 2013 request was repeated and widened to training and proficiency checks four years later.
What it does not support: Any measurement of skill retention or of the effect of the policy. Guidance on the law, not the law itself.

deskilling · automation complacency · should there be ai free periods at work

Peer-reviewed#

Przybylski, A. K. and Weinstein, N. (2017). A Large-Scale Test of the Goldilocks Hypothesis: Quantifying the Relations Between Digital-Screen Use and the Mental Well-Being of Adolescents #

Psychological Science, 28(2), 204-215

Method: Preregistered analysis of a representative sample of English 15-year-olds. Sampling frame 298,080; 120,115 provided usable data, 100,850 on paper and 19,265 online.
Finding: Links between digital screen time and mental wellbeing are described by quadratic rather than linear functions, with inflection points at 1 hour 40 minutes for weekday video-game play and 1 hour 57 minutes for weekday smartphone use, rising to 3 hours 41 minutes and 4 hours 17 minutes for other measures and to between 3 hours 35 minutes and 4 hours 50 minutes at weekends. Average Cohen's d for engagement beyond the inflection points was minus 0.18, accounting for 1 per cent or less of variance, against d = 0.54 for regularly eating breakfast and 0.58 for regular sleep. The authors state that moderate use is not intrinsically harmful and may be advantageous.
What it supports: That dose-response in this literature is not linear, and that the negative effects of heavy use are less than a third the size of the positive associations with sleep and breakfast.
What it does not support: Causation, and not displacement. The paper names the displacement hypothesis as the field's dominant assumption and calls for future work systematically analysing what is being displaced or amplified, which the field did not then do. Cross-sectional, self-reported exposure, English 15-year-olds only.

screen time · adolescents · displacement · dose-response · is screen time the same argument as ai use

Compiled review#

National Academies of Sciences, Engineering, and Medicine (2018). How People Learn II: Learners, Contexts, and Cultures #

The National Academies Press, Washington DC

Method: Consensus study report synthesising research on learning across the lifespan.
Finding: Defines metacognition as the ability to monitor and regulate one's own cognitive processes and to consciously regulate behaviour, including affective behaviour, and identifies calibration as the accuracy of a learner's monitoring.
What it supports: That the established definition contains a control component as well as a knowledge component. Monitoring alone is not metacognition.
What it does not support: Anything about AI. The report predates general availability of these systems and makes no claim about them.

metacognition · calibration · self-regulated learning · what is metacognition

Argued perspective#

Caulfield, M. (2019). SIFT (The Four Moves)#

Hapgood, 19 June 2019. An earlier version, Four Moves and a Habit, appeared in Web Literacy for Student Fact Checkers (2017)

Method: A practitioner method, not a study. Four moves: Stop, Investigate the source, Find better coverage, Trace claims to the original context.
Finding: Turns Wineburg and McGrew's lateral reading result into four actions a person can perform in under a minute, and is now taught in university libraries worldwide.
What it supports: That the fact-checker behaviour can be reduced to a teachable sequence. It is the most widely adopted of the source-evaluation frameworks and the closest external comparison to any source rule offered here.
What it does not support: Its own effectiveness. Caulfield offers it as a practical method and does not present a trial of it, and it inherits its evidence from the lateral reading literature rather than generating any.

verification · sources · misinformation · the source rule

Argued perspective#

Ericsson, K. A. and Harwell, K. W. (2019). Deliberate practice and proposed limits on the effects of practice on the acquisition of expert performance: why the original definition matters and recommendations for future research #

Frontiers in Psychology, 10, 2396

Method: Argued reply in a reviewed venue, re-examining which activities in the meta-analytic literature meet the original definition of deliberate practice.
Finding: Argues that most studies aggregated under the term measured practice of other kinds, and that the low variance figures follow from that misclassification rather than from a limit on practice itself.
What it supports: That the size of the practice effect is contested on definitional grounds, and that a figure quoted from either side needs its definition quoted with it.
What it does not support: That the meta-analytic estimate is wrong. It reports no new data and does not resolve the dispute.

deliberate practice · expertise · how is judgement trained

Peer-reviewed#

Macnamara, B. N. and Maitra, M. (2019). The role of deliberate practice in expert performance: revisiting Ericsson, Krampe and Tesch-Romer (1993)#

Royal Society Open Science, 6, 190327

Method: Direct replication and re-analysis of the 1993 study.
Finding: Accumulated practice explained considerably less of the difference between performers than the original is usually taken to claim.
What it supports: That the quality and design of practice matters more than the count. Included here deliberately, because it complicates the argument this research relies on.
What it does not support: That practice does not matter. It does; the simple dose-response reading is what fails.

deliberate practice · expertise · how humans learn with ai

Peer-reviewed#

Orben, A. and Przybylski, A. K. (2019). The association between adolescent well-being and digital technology use #

Nature Human Behaviour, 3(2), 173-182

Method: Specification curve analysis across three nationally representative datasets: the US Youth Risk and Behaviour Survey 2007-2015 at 74,814 adolescents, Monitoring the Future 2008-2016 at 268,672, and the UK Millennium Cohort Study at 11,872, a total of 355,358. 372 justifiable specifications identified for YRBS, 40,966 for MTF and 603,979,752 for MCS, of which 20,004 were run.
Finding: The association between digital technology use and adolescent wellbeing is negative but small, explaining at most 0.4 per cent of the variation in wellbeing, which the authors state is too small to warrant policy change. In YRBS, regularly eating potatoes was associated with wellbeing 0.9 times as negatively as technology use; in MCS, wearing glasses was 1.5 times as negatively. Bullying ran 4.3 times more negative and marijuana 2.7 times, both YRBS. Sleep and breakfast ranged from 1.7 to 44.2 times more positive across all three datasets.
What it supports: That an entire public debate was conducted on an effect too small to act on, and that analytic flexibility rather than data availability was doing the work in the studies that found more.
What it does not support: Causation in either direction. The authors state it is possible the associations they document, and those previously documented, are spurious, and that noisy self-report measurement could itself have diminished a real effect. Two figures often attached to this paper are not in it: 1.45 times for glasses, and 3,221,225,472 analyses, which comes from Odgers and Jensen describing it. Read in the Oxford ORA accepted manuscript, the published full text being paywalled.

screen time · adolescents · wellbeing · measurement · specification curve · is screen time the same argument as ai use

Peer-reviewed#

Orben, A. and Przybylski, A. K. (2019). Screens, Teens, and Psychological Well-Being: Evidence From Three Time-Use-Diary Studies #

Psychological Science, 30(5), 682-696

Method: Exploratory and confirmatory specification analyses across three nationally representative datasets from Ireland, the United States and the United Kingdom, N = 17,247 after exclusions, using time-use diaries as well as retrospective self-report.
Finding: Little evidence of substantial negative associations between digital screen engagement and adolescent wellbeing, whether measured across the day or before bedtime. Correlations between diary-recorded and retrospectively self-reported engagement were 0.18 in Ireland, 0.08 and 0.05 on US weekdays and weekend days, and 0.18 in the UK. Retrospective self-report consistently produced the most negative correlations. Extrapolating from median effects, an adolescent would need to report 63 hours 31 minutes more technology use a day to lower wellbeing by half a standard deviation, or 11 hours 14 minutes taking the maximum effect size in the specification set.
What it supports: That the two standard instruments for measuring screen exposure barely agree with each other, and that the more negative results come from the weaker instrument, which is what common method variance would predict.
What it does not support: That diaries are ground truth. They remain recall-based and the authors note brief or concurrent uses may not be recorded. Still cross-sectional, and it measures totals and timing rather than the kind of activity displaced.

screen time · measurement · self-report · adolescents · is screen time the same argument as ai use

Peer-reviewed#

Wineburg, S. and McGrew, S. (2019). Lateral Reading and the Nature of Expertise: Reading Less and Learning More When Evaluating Digital Information #

Teachers College Record, 121(11), 1-40

Method: Think-aloud study of 45 experienced internet users evaluating unfamiliar websites: 10 PhD historians, 10 professional fact-checkers and 25 Stanford undergraduates.
Finding: Historians and undergraduates read vertically, staying on the page and judging it by its own appearance. Fact-checkers left almost immediately, opened new tabs and read laterally, judging the source by what the rest of the web said about it. The fact-checkers reached sounder conclusions in less time.
What it supports: That evaluating a source is a behaviour rather than a checklist, that the behaviour is learnable, and that domain expertise does not confer it: PhD historians performed like undergraduates.
What it does not support: Anything about AI output specifically. It predates generative models, and a model's answer has no source page to leave. The transferable part is the move away from judging a text by its own surface.

verification · sources · critical thinking · the source rule

Peer-reviewed#

Dahmani, L. and Bohbot, V. D. (2020). Habitual use of GPS negatively impacts spatial memory during self-guided navigation #

Scientific Reports, 10, 6310, 14 April 2020. DOI 10.1038/s41598-020-62877-0. Read in full at source 5 September 2026

Method: Behavioural, cross-sectional with an unplanned longitudinal arm. 50 healthy regular drivers in Montreal aged 19 to 35 (18 women, 32 men; mean age 27.6), driving at least four days a week, from 60 recruited. Lifetime GPS experience measured by the McGill GPS questionnaire; navigation measured on two virtual radial-arm mazes plus a map-drawing score and the Santa Barbara Sense of Direction scale. 13 of the 50 returned a mean of 3.23 years later. Effect sizes are Pearson r with bootstrapped one-tailed BCa 95 per cent intervals. NO NEUROIMAGING: hippocampal dependence is inferred from prior validation of the tasks, not measured in this sample.
Finding: Cross-sectionally, greater lifetime GPS experience was associated with lower use of hippocampus-dependent spatial strategies (r = -0.22 on the first probe trial), lower navigation strategy scores (r = -0.20), poorer map drawing (r = -0.22) and fewer landmarks noticed (r = -0.26). In the 13-person follow-up, hours of GPS use since first testing tracked a steeper decline in spatial memory strategy use (r = -0.68) and in map drawing (r = -0.52).
What it supports: The only study to follow the same people while their GPS use rose. Its strongest internal argument against reverse causation is that heavier GPS users did not report a poorer sense of direction (r = 0.07 against the SBSOD), so the obvious alternative, that weak navigators reach for the satnav, has no support in the data.
What it does not support: Anything about the brain, because no scan was taken. Anything about dementia, Alzheimer's or atrophy: those words appear nowhere in the paper. And little with confidence about the longitudinal effect, which rests on 13 people from an unplanned follow-up. The authors' own words: they caution against any strong conclusions as spurious correlations are possible. Their discussion elsewhere uses notably firmer causal language than that sentence licenses. Nothing here transfers to reasoning: spatial memory is not judgement.

cognitive offloading · deskilling · spatial memory · The Knowledge · does gps damage your brain what is cognitive offloading using ai without dependency ai and human judgement

Compiled review#

Odgers, C. L. and Jensen, M. R. (2020). Annual Research Review: Adolescent mental health in the digital age: facts, fears, and future directions #

Journal of Child Psychology and Psychiatry, 61(3), 336-348

Method: Annual research review of the evidence on adolescent digital technology use and mental health, covering 29 studies in its main table.
Finding: Most research to date has been correlational, focused on adults rather than adolescents, and has produced a mix of conflicting small positive, negative and null associations. The most recent and rigorous large-scale preregistered studies report small associations that offer no way of distinguishing cause from effect and are unlikely to be of clinical or practical significance, explaining less than 0.5 per cent of the variance. Of the 29 studies reviewed, only two included objective or informant-rated measures of screen use, and the correlation between objectively measured and retrospectively reported screen time is estimated at about 0.20.
What it supports: That the field's own review verdict is that the evidence does not support causal claims or even strong consistent correlational patterns.
What it does not support: Anything new. It is a review, not primary data, it does not test displacement of any specific activity, and it does not address AI. Read in the NIH author manuscript.

screen time · adolescents · mental health · evidence quality · is screen time the same argument as ai use

Peer-reviewed#

Parry, D. A., Davidson, B. I., Sewall, C. J. R., Fisher, J. T., Mieczkowski, H. and Quintana, D. S. (2021). A systematic review and meta-analysis of discrepancies between logged and self-reported digital media use #

Nature Human Behaviour, 5(11), 1535-1547

Method: Systematic review and meta-analysis using robust variance estimation. 106 effect sizes included overall; the self-report against logged comparison draws on 66 effect sizes from 44 studies with a total sample of 52,007.
Finding: The correlation between self-reported and logged digital media use is positive but medium, r = 0.38, 95 per cent CI 0.33 to 0.42. For problematic use it falls to r = 0.25 across 40 effect sizes from 19 studies. Over-reporting and under-reporting occur in similar proportions, and fewer than 10 per cent of self-reports fall within 5 per cent of the equivalent logged value. The authors conclude that self-report measures may not be a valid stand-in for more objective measures and ask for pause in drawing wide-reaching knowledge or policy conclusions from studies relying solely on them.
What it supports: That the independent variable in most of the screen-time literature is wrong by a measured amount, which is the strongest single reason not to carry that method into questions about AI use.
What it does not support: That self-report is useless, or that logs are ground truth: the authors note potential biases in log data too. They state it is an open question whether the discrepancy is random or systematic error. Nothing in it concerns AI. Read in the University of Bath accepted manuscript alongside the published abstract.

measurement · self-report · screen time · validity · is screen time the same argument as ai use

Peer-reviewed#

Shorey, S., Lau, T. C., Lau, S. T. and Ang, E. (2021). Entrustable professional activities in health care education: a scoping review #

Medical Education, 55(11), 1247-1260

Method: Scoping review of the published literature on entrustable professional activities, the units of professional work a trainee is progressively permitted to perform unsupervised.
Finding: Adoption is wide across health professions education and the supervision levels are used to structure graduated responsibility, while validity evidence for the entrustment decisions themselves is mixed and uneven.
What it supports: That one profession has built an explicit, staged mechanism for handing over responsibility as capability is demonstrated, and that the mechanism is in real use.
What it does not support: That entrustment decisions are reliable, or that the approach transfers outside clinical training.

apprenticeship · assessment · supervision · expertise · what is cognitive apprenticeship how is judgement trained

Peer-reviewed#

Sinha, T. and Kapur, M. (2021). When Problem Solving Followed by Instruction Works: Evidence for Productive Failure #

Review of Educational Research, 91(5), 761-798. DOI 10.3102/00346543211019105

Method: Meta-analysis of 53 studies and 166 comparisons of problem-solving-before-instruction against instruction-before-problem-solving.
Finding: A moderate effect favouring problem solving first, Hedges g = 0.36 with a confidence interval of 0.20 to 0.51, rising to between 0.37 and 0.58 where the design followed the productive failure principles closely. The effect reverses for second to fifth graders and for domain-general skills, where instruction first wins.
What it supports: That letting people struggle before teaching them beats teaching them first, at moderate effect size, for older learners on domain-specific content.
What it does not support: That struggle is universally good. The authors report the reversal for young children and for general skills themselves, and an earlier meta-analysis by Darabi and colleagues rested on only 12 studies.

productive failure · productive struggle · meta-analysis · what is productive struggle

Compiled review#

Verhaeghen, P. (2021). Mindfulness as Attention Training: Meta-Analyses on the Links Between Attention Performance and Mindfulness Interventions, Long-Term Meditation Practice, and Trait Mindfulness #

Mindfulness, 12(3)

Method: Three meta-analyses covering 109 effect sizes from 40 intervention studies, 59 effect sizes from 18 long-term meditator studies, and 197 effect sizes from 28 trait studies.
Finding: Average effects were small to moderate, Hedges g of 0.29 for interventions and 0.32 for long-term practice, concentrated in inhibition and executive control rather than sustained attention.
What it supports: That something measurable happens, and that it is smaller and narrower than the popular claim.
What it does not support: That the effect survives comparison with an active control. The published breakdown does not separate active from passive controls, so demand effects cannot be excluded.

· is attention a trainable skill

Peer-reviewed#

Brynjolfsson, E., Li, D. and Raymond, L. (2023). Generative AI at Work #

Quarterly Journal of Economics, 140(2), 889-942. DOI 10.1093/qje/qjae044. Advance Access 4 February 2025. Earlier version NBER Working Paper 31161

Method: Staggered rollout of a GPT-3-based conversational assistant across 5,172 customer-support agents in 133 teams at a single Fortune 500 business-process software firm, most of them working from the Philippines. Three million chats observed, 1.2 million of them post-deployment. The system was fine-tuned on past agent conversations, with chats by top performers deliberately up-weighted in training.
Finding: Resolutions per hour rose 15 per cent on average, 15.2 per cent in the preferred specification with agent and tenure fixed effects. Less skilled and less experienced workers gained a 30 per cent increase in issues resolved per hour, rising to 36 per cent for the lowest skill quintile, while the most skilled saw no significant productivity change and small declines in conversation quality and customer satisfaction. Customer sentiment improved by half a standard deviation and requests to speak to a manager fell about 25 per cent. During unplanned outages, agents with longer AI exposure still handled chats faster than their pre-AI baseline, but only those who had adhered closely to the suggestions.
What it supports: That a model trained on the behaviour of a firm's best workers can transfer measurable parts of that behaviour to its newest ones, immediately and at scale, raising the floor far more than the ceiling. The outage evidence shows some of the gain survives the tool being switched off, conditional on the worker having engaged with it rather than passed it through.
What it does not support: Whether those novices became experts. It measures output over months in one firm, one occupation and a stable product environment, and the authors say so. Wages, labour demand and hiring composition were not observed. The outage estimates are the authors' own noisiest, because outages are rare and may not be comparable chats.

productivity · expertise · synthetic seniority · how humans learn with ai staying valuable in the age of ai ai and human judgement how will ai change customer service what becomes more valuable as ai gets cheaper ai and expert judgement does ai actually make people more productive hiring signals when effort is free pricing work when production is cheap does ai make juniors better or only look better

Peer-reviewed#

Liang, W., Yuksekgonul, M., Mao, Y., Wu, E. and Zou, J. (2023). GPT detectors are biased against non-native English writers #

Patterns, 4(7)

Method: Seven widely used GPT detectors evaluated against TOEFL essays by non-native English speakers and essays by US eighth-grade students.
Finding: Detectors misclassified more than half of the non-native essays as AI-generated, an average false positive rate of 61.22 percent, while classifying US eighth-grade essays with near-perfect accuracy. The proposed mechanism is that detectors rely on perplexity, and second-language writing is more predictable.
What it supports: That AI detection carries a severe and systematic bias against second-language writers, and that the bias is structural rather than a tuning problem.
What it does not support: That every detector now on the market performs identically. The study tested tools available at the time, and vendors dispute the generalisation.

AI detection · false positives · academic integrity · fairness · does ai detection work how to assess students when ai can do the assignment

Argued perspective#

Lo, L. S. (2023). The CLEAR path: A framework for enhancing information literacy through prompt engineering #

The Journal of Academic Librarianship, 49(4), 102720

Method: A proposed framework, not an experiment. Sets out five principles for writing prompts, Concise, Logical, Explicit, Adaptive and Reflective, for use in information literacy teaching.
Finding: Offers the most widely cited peer-reviewed prompt framework in education. Every element concerns the quality of the instruction given to the model: brevity, order, specificity, iteration and review of the output.
What it supports: That the published prompt frameworks optimise the output. Useful here as the comparison case: CLEAR, CO-STAR and Google's TCREI all ask what the model should be told, and none asks what the person should keep doing themselves.
What it does not support: That following CLEAR improves learning, or output quality. It is a framework proposal in a library science journal, with no trial behind it, and its author does not claim one.

prompting · information literacy · frameworks · goal context friction standard

Argued perspective#

Mollick, E. and Mollick, L. (2023). Assigning AI: Seven Approaches for Students, with Prompts #

Wharton School Research Paper, SSRN 4475995, revised September 2023. Also arXiv:2306.10052, June 2023. Not peer-reviewed in either venue

Method: A practical framework paper. Seven roles a student or teacher can put an AI into, each with a rationale, an example prompt, an example output, the risks, and guidance. No study, no sample, no measured outcome.
Finding: The seven roles are mentor, giving feedback; tutor, giving direct instruction; coach, prompting metacognition; teammate, arguing the other side; student, whom you teach in order to find out what you do not know; simulator, for practice; and tool, for the mechanical part. Each carries a named pedagogical risk, and the one attached to tool is 'outsourcing thinking, rather than work'.
What it supports: That there are at least seven distinct uses beyond the answer machine, which is a useful correction to a debate conducted as though there were one. The underlying pedagogical claims rest on established learning science, including Bjork on desirable difficulties and the retrieval-practice literature.
What it does not support: That any of it works in this form. The authors say so themselves: the approaches are 'still in their infancy and largely untested' and should be approached 'with a spirit of experimentation'. Established priors applied to an unevaluated delivery mechanism. The prompts are also anchored to mid-2023 model capability and should not be presented as current.

deliberate practice · metacognition · learning · the missed reps · how to use ai at university how humans learn with ai

Peer-reviewed#

Sackett, P. R., Zhang, C., Berry, C. M. and Lievens, F. (2023). Revisiting the design of selection systems in light of new findings regarding the validity of widely used predictors #

Industrial and Organizational Psychology, 16

Method: Meta-analytic re-correction of prior personnel-selection meta-analyses, addressing systematic overcorrection for range restriction.
Finding: Work sample validity falls from the widely quoted .54 to .33. Structured interviews fall from .51 to .42 and become the strongest single predictor. Unstructured interviews fall from .38 to .19. General cognitive ability falls from .51 to .31.
What it supports: That the numbers most often cited for assessment methods were inflated, and by how much.
What it does not support: That these methods do not work. Work samples and structured interviews remain among the strongest predictors available. It also offers no validity data for AI-era assessment formats.

· how do you assess capability rather than output

Peer-reviewed#

Scarfe, P., Watcham, K., Clarke, A. and Roesch, E. (2024). A real-world test of artificial intelligence infiltration of a university examinations system: A 'Turing Test' case study #

PLOS ONE, 19(6), e0305354. DOI 10.1371/journal.pone.0305354. Published 26 June 2024. Open access, data at OSF

Method: 33 fake student accounts submitted wholly GPT-4-written answers into the live examinations system of the School of Psychology and Clinical Language Sciences at the University of Reading, summer 2023. Five undergraduate modules across all years, about 5 per cent of submissions on each. Two formats, four 200-word answers in a 2.5-hour window and one 1,500-word essay in an 8-hour window, both unsupervised at home. Markers were staff and trained postgraduates, marking anonymously and entirely unaware of the study.
Finding: 94 per cent of the AI submissions were not detected, and 97 per cent went undetected on the stricter test of a marker actually mentioning AI. Across the five modules there was an 83.4 per cent probability, by resampling, that the AI submissions would outscore an equal-sized random draw of real students, an advantage of just over half a classification boundary. AI lost on one module only, a finalist module carrying three AI submissions.
What it supports: That wholly machine-written work passed through a real examinations system substantially undetected and scored above the human median, under blind marking, in a live setting rather than a demonstration.
What it does not support: How much real cheating occurs, which the authors state plainly: 'we have no way to estimate the proportion of students in our sample who used AI'. It also flatters detection rather than damning it. In their words the 6 per cent detection rate 'likely overestimates our ability to detect real-world use of AI to cheat in exams', because a real student would not take so naively obvious an approach. The comparison group may itself contain AI-assisted work. And the 83.4 per cent is a resampling probability against a median, NOT a count of head-to-head comparisons won, which is how it is usually restated.

assessment · detection · synthetic seniority · the missed reps · how to use ai at university how to assess students when ai can do the assignment does ai detection work

Peer-reviewed#

Adinoff, B. and Nunes, J.C. (2025). Challenging the 25-year-old 'mature brain' mythology: implications for the minimum legal age for non-medical cannabis use #

The American Journal of Drug and Alcohol Abuse, 51(5), 577-583

Method: Review of the neuroscience and policy literature behind the age-25 threshold.
Finding: Argues the mature-brain-at-25 claim is not supported by the underlying neuroscience and should not be used as a basis for age thresholds in policy.
What it supports: That the challenge to this claim is in the peer-reviewed literature rather than confined to science journalism.
What it does not support: Anything about AI, learning or capability. It is cited here for the status of the claim, not for the subject.

brain development · policy · age thresholds · does the brain mature at 25

Peer-reviewed#

Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakci, O. and Mariman, R. (2025). Generative AI Without Guardrails Can Harm Learning: Evidence from High School Mathematics #

Proceedings of the National Academy of Sciences, 122(26), e2422633122, published 25 June 2025. DOI 10.1073/pnas.2422633122. A correction was issued on 20 August 2025, issue dated 26 August, DOI 10.1073/pnas.2518204122, PMID 40833419: a production error had printed the wrong affiliation for Osbert Bastani. It changes no author, no method, no figure and no finding. Recorded here because two search queries reaching this research ask specifically for the correction, and a reader who finds one and does not read it could reasonably assume the result was retracted. It was not. Correction read at source 12 September 2026.

Method: Field experiment, nearly 1,000 high-school students, three arms: unrestricted GPT-4, a hints-only tutor, and a control.
Finding: Grades rose 48 percent with unrestricted access and 127 percent with the tutor while the tool was present. With access removed, the unrestricted group scored 17 percent LOWER than students who never had it. The guardrailed tutor largely removed the harm.
What it supports: That the design of the interface, not the presence of AI, decides whether people learn. The single most useful result in this literature.
What it does not support: What a guardrailed interface should look like for professional work. It was school mathematics over a bounded period.

learning · deskilling · missed reps · how humans learn with ai chro guide to ai ai and human judgement what is the ai memo test does using ai stop you learning what happens to medical and legal training what is the new imposter syndrome is ai making us stupid is an ai tutor as good as a human tutor should there be ai free periods at work what is anthropological regression

Vendor research#

Coursera; foreword by Greg Hart, CEO (2025). Global Skills Report 2025 #

Coursera, June 2025

Method: Platform data from more than 170 million Coursera learners across 100+ countries, covering calendar 2024 enrolments, combined with external indicators including World Bank Human Capital Index and labour force participation data. A new AI Maturity Index draws on IMF and OECD metrics; the exact learner-data window and ranking weights are not published on the page.
Finding: GenAI enrolments grew 195% year on year and passed 8 million in total, with 700 GenAI courses averaging 12 enrolments per minute. Latin America recorded 425% year-on-year growth in GenAI enrolments, the highest of any region, and India recorded over 1.3 million GenAI enrolments in 2024. Women are 46% of Coursera's learner base but 32% of GenAI enrolments. Switzerland, the Netherlands, Sweden and Singapore rank first to fourth on skill proficiency; Singapore ranks first in Asia-Pacific on both proficiency and AI maturity.
What it supports: That demand for generative AI courses on one large platform rose steeply in 2024 and is unevenly distributed by region and gender.
What it does not support: Enrolments are not completions or competence, and platform users are not a population sample. Country rankings measure Coursera learners' performance, not national skill levels. Coursera sells the courses being counted.

genai enrolments · skills rankings · online learning · gender gap · ai maturity

Peer-reviewed#

Kestin, G., Miller, K., Klales, A., Milbourne, T. and Ponti, G. (2025). AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting #

Scientific Reports, 15, published 3 June 2025

Method: Randomised crossover experiment in Harvard's largest introductory physics course, Fall 2023. Of 233 enrolled students, 194 were eligible on consent and completion. Each student experienced both conditions across two consecutive weeks, one topic taught by in-class active learning and one by a purpose-built AI tutor at home, with pre-tests and post-tests for each. The tutor used GPT-4 with expert-crafted question-specific prompts, pre-written answers, instructional video and a structured scaffold.
Finding: Median post-test score 4.5 in the AI condition against 3.5 in the active-learning condition, from a combined pre-test median of 2.75; median learning gain over double. Mann-Whitney z = -5.6, p below 10 to the minus 8. Linear regression effect size 0.63, described by the authors as an underestimate because of a ceiling effect; quantile regression gives 0.73 to 1.3 standard deviations. Median time on task 49 minutes against 60 assumed for the class, with no correlation between time on task and score. Engagement 4.1 against 3.6 and motivation 3.4 against 3.1; enjoyment and growth mindset showed no significant difference.
What it supports: That a heavily engineered AI tutor, built by subject experts to follow established pedagogy, can outperform a well-run active-learning class on immediate post-test performance at the understanding, applying and analysing levels, in less time.
What it does not support: That a general chatbot does this. Accuracy depended on pre-written answers and instructor-written prompts. Retention was not measured; post-tests followed the lessons immediately. One course, one institution, two topics. The authors state they do not presume the result holds where complex synthesis or higher-order critical thinking is required.

AI tutoring · active learning · learning gains · scaffolding · how will ai change teaching is an ai tutor as good as a human tutor

Vendor research#

LinkedIn Learning (2025). 2025 Workplace Learning Report: The rise of career champions #

LinkedIn Learning, early 2025

Method: Survey of 937 L&D and HR professionals and 679 learners across North America, South America, Asia-Pacific and Europe, plus interviews with talent leaders; fieldwork dates are not stated on the page. Combined with LinkedIn platform data described as 1 billion members, 14 million jobs and 5 million profile updates per minute, current to September 2024.
Finding: 49% of L&D professionals agree that their executives are concerned employees do not have the right skills to execute the business strategy. 71% of L&D professionals say they are exploring, experimenting with or integrating AI into their work. Organisations classed as career development champions are 42% more likely to describe themselves as frontrunners in generative AI adoption (51% versus 36% of others), 32% more likely to be deploying AI training programmes and 88% more likely to offer career-enhancing gig opportunities.
What it supports: That L&D leaders report a skills concern at executive level and that organisations investing in career development also report earlier generative AI adoption, as an association.
What it does not support: It cannot show that career development causes AI adoption; the champion classification and the adoption stage are both self-reported by the same respondents. LinkedIn sells the learning products the report recommends, and the 937-person sample is not described as representative.

workplace learning · l&d · generative ai adoption · career development · skills gap

Compiled review#

Mansfield, K. L., Ghai, S., Hakman, T., Ballou, N., Vuorre, M. and Przybylski, A. K. (2025). From social media to artificial intelligence: improving research on digital harms in youth #

The Lancet Child and Adolescent Health, 9(3), 194-204. Personal View, not primary research

Method: Personal View setting out the methodological failures of the social media harms literature and what should be done differently for AI. No new data.
Finding: Self-reported screen time is problematic as a measure, being imprecise and prone to bias, and as a construct, being unidimensional, homogenous and of little validity, failing to distinguish social, educational, entertainment, work and informational uses. On AI, the authors write that using self-reported frequency or duration of adolescent AI use as the exposure measure of interest is perhaps even more concerning than counting total time spent on social media, and that only behavioural data on exposure to a range of AI applications would provide the detail needed. They record that health policy decisions have been implemented on inconsistent, non-causal or ungeneralisable evidence of online harms.
What it supports: That the researchers who built the screen-time evidence base have published the warning against transplanting its method to AI, in advance and in a clinical journal.
What it does not support: Anything empirical. It is a commentary with no new data and no head-to-head methodological comparison. Its prescription is finer-grained measurement of which AI in what context, which is adjacent to but not the same as asking which human practice was displaced. Publisher full texts returned empty bodies; read in the Oxford ORA accepted version.

screen time · AI and children · measurement · research method · is screen time the same argument as ai use

Peer-reviewed#

Mousley, A. and colleagues (2025). Topological turning points across the human lifespan #

Nature Communications, November 2025

Method: Diffusion MRI from 3,802 people aged 0 to 90, analysed for structural network topology across the lifespan.
Finding: Four topological turning points, at approximately ages 9, 32, 66 and 83, dividing life into five epochs. The adolescent epoch runs from about 9 to about 32. The largest overall shift in trajectory occurs around 32, not in the mid-twenties.
What it supports: That structural network reorganisation continues well past the mid-twenties, and that the nearest thing to a boundary at the end of adolescence sits around 32.
What it does not support: That 32 is the new 25. The authors describe turning points in network topology, not a moment of cognitive completion, and reading it as a new threshold would repeat the original error with a different number.

brain development · lifespan · network topology · does the brain mature at 25

Simulation model#

Peterson, A. J. (2025). AI and the problem of knowledge collapse #

AI & Society, 40(5), 3249-3269. DOI 10.1007/s00146-024-02173-x. Received 9 July 2024, accepted 17 December 2024, published online 19 January 2025, issue dated June 2025. University of Poitiers. PROVENANCE, and it runs in both directions. The VERSION OF RECORD IS NOT OPEN ACCESS: its abstract, endnotes, references and full Appendix A were read at source on 17 September 2026 and the BODY IS PAYWALLED and was not read. The model detail, the simulation figures and the language-model counts below come from the author's own preprint, arXiv:2404.03502v2 of 22 April 2024, which the published paper names in its own acknowledgements alongside hal-04534111. Anybody citing the arXiv preprint alone is understating it, because it has been peer reviewed and published since; anybody citing the journal version for the model detail should know that detail was read in the preprint.

Method: An agent-based simulation plus a small illustrative language-model audit. The simulation: 25 individuals with types drawn from a lognormal distribution, 100 rounds, each round choosing to learn the expensive way, to learn through a discounted AI-assisted process, or not at all. The expensive path samples the true distribution, modelled as a Student's t with 10 degrees of freedom; the AI path samples the same distribution truncated at 0.75 standard deviations either side of the mean in the default setting. Public knowledge is a kernel density estimate over the 100 most recent samples, and distance from the truth is Hellinger distance. Agents update on observed returns at a learning rate of 0.05 and have NO foresight: the paper says they 'cannot foresee the true future value of their innovation options'. Generations turn over every 10 rounds. The audit: four models (GPT-3.5-turbo, Claude-3-sonnet, Gemini-pro, Llama2-70b) asked what human well-being depends on, under five prompt phrasings, against a reference list of 2,693 entities.
Finding: THE SIMULATION. 'For our default model, after nine generations, when there is no AI discount the public distribution has a Hellinger distance of just 0.09 from the true distribution. When AI-generated content is 20% cheaper (discount rate is 0.8), the distance increases to 0.22, while a 50% discount increases the distance to 0.40.' Those ratios are the 2.3 and 3.2 in circulation. THE CONDITION THE NUMBER DEPENDS ON: 'If there is no generational change, there is at worst only a reduction in the tails of public knowledge outside the truncation limits. In this case the distribution is stable and does not "collapse".' How often turnover happens, every 3, 5, 10 or 20 rounds, has little effect; whether it happens at all decides the result. Truncation matters as much as price: at two standard deviations 'the effect is minimal', at a quarter of one 'the impact is large'. Faster updating by agents can offset a moderate discount and does not prevent collapse. THE AUDIT: Aristotle drew 4,779 mentions and Martha Nussbaum 1,080, against 83 for Avicenna and Ibn Sadi combined, 62 for Al-Ghazali and 52 for Al-Farabi, while Martin Seligman drew 392; the corpus contains 30,315 uses of the word 'the', so Aristotle appears once for every seven of those. Asking region by region across 34 named regions raised evenness from 0.55 to 0.86 on Pielou's index. DEFINITION, from the published appendix: 'the progressive narrowing over time (or over technological representations) of the set of human working knowledge and the current human epistemic horizon relative to the set of broad historical knowledge.'
What it supports: That there is a specified mechanism by which cheap central answers could narrow what a society holds, with the conditions written down and testable, and that the narrowing compounds only across generational turnover. Separately, that at one point in time four models answered a question about human well-being with a heavily concentrated set of names, and that prompting region by region widened it substantially. Peterson also separates his subject from model collapse in his own words: his 'interest is in the inverse of this concern', because 'Humans, unlike LLMs trained by researchers, have agency in deciding among possible inputs.'
What it does not support: That knowledge collapse is occurring. Peterson never claims it and the paper is conditional throughout: AI 'can paradoxically harm public understanding', reliance 'could lead to' collapse, dependence 'may lead to' a reduction in the long tails, the concern is 'plausible'. The 2.3 is a simulation output under default parameters and not a measurement of anybody's beliefs. The distribution standing for knowledge is a stated metaphor: 'we make no claim that "truth" is in some deep way distributed 1-D Gaussian', and the truncation device is 'meant to be metaphorical'. The language-model audit measures concentration at one moment and therefore cannot show narrowing over time, which is what the term denotes, and the author says it 'is not intended as a general purpose benchmark'. Its published table of diversity scores carries rows for three of the four models named and none for Llama2-70b, unexplained. The number of responses generated for that corpus is stated nowhere. And the phenomenon may be unobservable in principle: 'we cannot know if the current tails of knowledge are correct or too thin'. The preprint's informal definition, 'the set of information available to humans', is WIDER than the published one, so the two versions do not say the same thing.

knowledge collapse · model collapse · homogenisation · epistemic diversity · long tail · what is knowledge collapse does ai make everyone think alike

Peer-reviewed#

Rasalkar, K., Tripathy, S., Sinha, S., Mukherjee, B., Takkella, N., Dadel, E. V., Sundriyal, M. and Prasad, S. (2025). Enhancing medical assessment strategies: a comparative study between structured, traditional and hybrid viva-voce assessment #

BMC Medical Education, 25(1), article 835, 4 June 2025. DOI 10.1186/s12909-025-07428-9. AUTHOR CORRECTION, 19 September 2026: this entry read 'Prasad, S. et al.' from an earlier session. Swetanka Prasad is the EIGHTH and last author, not the first, and the entry id is left unchanged only because it is referenced elsewhere. Checked against the publisher record at Crossref. Third misattribution caught in this research since 4 September 2026, and the first in which the research itself was the source of the error.

Method: 151 medical students assessed by two examiners across three viva formats, compared on reliability and perceived fairness.
Finding: Traditional unstructured viva showed significant inter-examiner variability. Structured formats improved fairness and coverage. The best format reached a reliability of 0.663, which is moderate rather than high.
What it supports: That oral examination can be made fairer by structuring it, and that unstructured viva has a measurable examiner problem.
What it does not support: That oral assessment is highly reliable. Even the best format tested was only moderate, at a single institution.

· how do you assess capability rather than output what happens to medical and legal training

Vendor research#

Udemy Business (2025). 2026 Global Learning & Skills Trends Report #

Udemy, September 2025

Method: Consumption data from Udemy Business learners at more than 17,000 enterprise customers worldwide, comparing 1 July 2024 to 30 June 2025 with the prior year on total consumption and percentage growth by course topic. Supplemented by an employee survey whose sample size and fieldwork dates are not given in the release.
Finding: Udemy reports 11 million GenAI course enrolments to date. Consumption of Microsoft Copilot content rose 3,400% year on year and GitHub Copilot content 13,534%. Consumption of AI ethics and governance courses rose 98%, decision-making 38%, critical thinking 37% and adaptive skills overall 25%. In the accompanying survey, 88% of employees agree effective leadership is critical to their organisation's AI initiatives, but 48% believe their managers are ready for the AI era.
What it supports: That enterprise learners on one platform shifted time towards AI tool training and, at the same time, towards judgement and ethics content in 2024 to 2025.
What it does not support: Percentage growth from a small base overstates scale (the Copilot figures have no base numbers). Consumption is not skill gain, and customers of a learning vendor are not representative employers. Udemy sells the content.

enterprise learning · genai training · critical thinking · ai ethics · leadership readiness

Simulation model#

Acemoglu, D., Kong, D. and Ozdaglar, A. (2026). AI, Human Cognition and Knowledge Collapse #

NBER Working Paper 34910. DOI 10.3386/w34910. The page displays an issue date of February 2026 while its own citation metadata gives 2 March 2026; both are printed here rather than the tidier one. Read at source on 17 September 2026: the abstract and the acknowledgements, which name the Hewlett Foundation, the Stone Foundation and the MIT Gen-AI Consortium as funders, with Acemoglu separately acknowledging Hewlett, Schmidt Sciences and the Smith Richardson Foundation. The full working paper was not read.

Method: A dynamic economic model of learning and decision making. Successful decisions need community-level general knowledge and individual context-specific knowledge together, as complements. Costly human effort jointly produces a private signal about a person's own context and a 'thin' public signal that accumulates into the community's stock, which is where the learning externality comes from. Agentic AI supplies context-specific recommendations that substitute for the effort. No data; the results are properties of the model.
Finding: 'When human effort is sufficiently elastic and agentic recommendations exceed an accuracy threshold, the economy can tip into a knowledge-collapse steady state in which general knowledge vanishes ultimately, despite high-quality personalized advice.' Two further results carry the policy weight. Welfare is non-monotone in agentic accuracy, so there is an interior welfare-maximising level of precision and a more accurate agent is not always better. And greater capacity to aggregate human-generated general knowledge 'unambiguously raises welfare and increases resilience to knowledge collapse'.
What it supports: That the erosion argument can be stated as an externality with a tipping point, in the standard machinery of economics, by authors whose work in this area is taken seriously. The externality framing is the contribution: the individual who takes the agent's recommendation is not making a mistake, and the loss falls on a stock nobody owns.
What it does not support: Anything about the world. It is a model and its numbers are properties of its assumptions, the abstract states the tipping result conditionally, and the full paper has not been read here. It is also NOT the origin of the term: Andrew Peterson published 'AI and the Problem of Knowledge Collapse' in April 2024 and in AI & Society in January 2025, and the two papers model different objects. Peterson narrows the diversity of what a population holds; this narrows the public stock of general knowledge through a learning externality. The research's own glossary credited the coinage here until 17 September 2026 and was corrected.

knowledge collapse · learning externality · agentic ai · general knowledge · what is knowledge collapse

Working paper#

Brosch, H., Langer, C., Lergetporer, P., Mueller, S., Pfeifer, H. and Tschöpe, N. (2026). Task Expansion with Generative AI: Experimental Evidence on Middle-Skilled, Early-Career Workers #

Stanford Digital Economy Lab working paper, listed 7 October 2026. Abstract read at source 9 October 2026; the paper itself was not read

Method: Field experiment with 673 final-year IT apprentices at twelve German vocational schools. Access to AI was randomly assigned for occupation-specific tasks, some of them beyond the apprentices' formal training.
Finding: 'Crucially, productivity increases by 26–31 percentage points on tasks beyond apprentices' formal training, providing direct evidence of AI-enabled task expansion.' 'These productivity gains do not come at the cost of reduced immediate task comprehension.' Gains tended to be larger where performance without AI was lowest and among apprentices with above-median AI literacy.
What it supports: That AI let early-career workers complete tasks beyond their training, with no loss of comprehension measured straight afterwards.
What it does not support: Anything about skill months later or without the tool: comprehension was measured immediately. The productivity measure is not defined on the page read, and no funding statement appears there. Not peer-reviewed.

apprentices · junior staff · task expansion · productivity · AI literacy · does ai make juniors better or only look better capability debt

Working paper#

Cruces, G., Fernandez Meijide, D., Galiani, S., Galvez, R. and Lombardi, M. (2026). Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment #

NBER Working Paper 34851; CEPR Discussion Paper 21299; arXiv:2608.04198

Method: Randomised online experiment, 1,174 adults aged 25 to 45, workplace-style problem-solving task with or without a generative AI assistant, followed by an unassisted module.
Finding: AI improved performance for everyone and more for the less educated. Without AI, higher-education participants outperformed lower-education participants by 0.548 standard deviations; with AI the gap fell to 0.139, closing about three-quarters of it. Treated participants did not perform worse once AI was removed, and lower-education participants retained part of their improvement, although a sizeable gap re-emerged.
What it supports: That assisted use does not automatically leave people worse off than unassisted controls when the tool is taken away, and that AI can compress an education-based performance gap while it is present.
What it does not support: That skill formed. One session with an immediate unassisted module tests transfer within a sitting, not skill formation over time, and the studies that found post-removal deficits taught a body of knowledge and removed the tool afterwards. The equity gain is also partly transient by the authors' own account, since a sizeable gap re-emerges without the assistant.

capability debt · productivity · learning · equity · capability debt

Working paper#

Eastwood, M., Narne, H., Hilby, J., Denny, P., Aggarwal, A. and Kapoor, A. (2026). Guardrails or Roadblocks? Effects of Pedagogical Style and Context Awareness in AI Teaching Assistants for Programming #

arXiv 2609.29995, submitted 24 September 2026. Abstract read at source 28 September 2026

Method: Randomised controlled trial with 132 students in an introductory programming course, assigned to one of four AI teaching assistants varying on pedagogical style (Socratic or direct) and contextual awareness (with or without the full problem context).
Finding: Students rated the Socratic assistant with full context least favourably; that condition showed more interaction stress, more use of outside language models and less demonstrated comprehension, though not all differences reached statistical significance.
What it supports: That a more guarded tutor is not automatically a better one, and that students route around support they find restrictive.
What it does not support: Effects on learning outcomes beyond the course; the bypass finding is a pattern rather than a significant result; the paper is a preprint.

AI tutors · students · guardrails · higher education · should universities let ai mark students work

Working paper#

Habibullah, A., Alshoibi, Y., Alshiekh, M., Khan, S. and Khan, N. (2026). Where LLM Graders Succeed and Break: Evidence from Two Computer-Science Exams #

arXiv 2609.29333, submitted 24 September 2026. Abstract read at source 28 September 2026

Method: 171 model configurations run over a computer vision exam sat by 570 students and independently double-marked by two humans, with a replication on a machine learning exam sat by 1,038 students; ablations on prompt wording; a LoRA fine-tuning adapter trained on about 3,900 pooled marked answers.
Finding: The best configuration's mean absolute error was 1.64 marks out of 35, below the 2.61 marks by which the two human markers differed from each other. A 'strict grader' preamble made 14 of 17 open-weight models fail badly and three stop grading; the phrase 'never give partial credit' on its own was enough to stop models grading. The failures reproduced on the second exam with exam-specific direction. The adapter brought five small open models to parity with a human marker and reduced prompt sensitivity.
What it supports: That on structured computer science exams a language model can mark within the range of human disagreement, and that its accuracy is a property of the prompt and tuning rather than of the model.
What it does not support: Anything about essays, proofs, portfolios or reflective writing; whether markers who stop marking lose diagnostic ability; the paper is a preprint and has not been peer-reviewed.

AI marking · assessment · LLM graders · human oversight · higher education · should universities let ai mark students work

Argued perspective#

Ke, Y., Jin, L., Ong, J.C.L., Thirunavukarasu, A.J., Car, J., Cheung, C.Y., Tham, Y.C., Ting, D.S.W., Ong, M.E.H., Compton, S., Narayan, A., Keane, P.A., Wong, T.Y., Bates, D.W., Tan, P. and Liu, N. (2026). AI-induced never-skilling in medical education #

Nature Medicine, 32(6), 1997-2006, published 22 May 2026. DOI 10.1038/s41591-026-04438-y

Method: A Perspective, not primary research. Sixteen authors across sixteen institutions argue a three-part taxonomy of how AI can interrupt the formation of clinical competence, and propose an untested three-phase protective framework. The clinical evidence it leans on is borrowed, chiefly Budzyn and colleagues on colonoscopy.
Finding: Separates three distinct failures. Deskilling is the degradation of established competence in clinicians already trained. Mis-skilling is the acquisition of incorrect reasoning patterns through uncritical adoption of erroneous or biased AI output. Never-skilling is the failure to form foundational competence during training, when AI substitutes for the cognitive effort that would have built it. The authors predict a state they call false proficiency: competence that appears real but depends on the AI remaining available.
What it supports: That the three failures are different problems requiring different responses, and that the entry-level case is not simply deskilling applied to younger people. Never-skilling has no baseline to return to, which is what makes it a distinct category rather than a matter of degree.
What it does not support: That never-skilling occurs. The authors disclaim this themselves and do so more than once: 'Direct causal evidence linking AI exposure during training to competency failure in medical trainees does not exist', and the abstract concedes that direct evidence from medical training is absent. It is a risk model, explicitly not an established phenomenon. Prevalence, severity and reversibility are all stated as unknown, and the proposed framework is untested. Cite it for the taxonomy and as the origin of the terms, never as evidence of harm.

never-skilling · mis-skilling · deskilling · false proficiency · synthetic seniority · will ai replace entry level jobs synthetic seniority how humans learn with ai what happens to medical and legal training

Working paper#

Liu, G., Christian, B., Dumbalska, T., Bakker, M. A. and Dubey, R. (2026). AI Assistance Reduces Persistence and Hurts Independent Performance #

arXiv:2604.04721, submitted 6 April 2026, revised 5 August 2026 (v4). Preprint, not peer reviewed

Method: A series of randomised controlled trials on human-AI interactions, N = 1,222, across mathematical reasoning and reading comprehension. AI assistance is available during a practice phase and then withdrawn, and performance is measured unassisted.
Finding: AI assistance improves performance in the short term, and people then perform significantly worse without AI and are more likely to give up. The authors report that these effects emerge after only brief interactions, approximately 10 minutes. They attribute the loss of persistence to AI conditioning people to expect immediate answers, denying them the experience of working through challenges on their own, and note that persistence is one of the strongest predictors of long-term learning.
What it supports: That a withdrawal effect can be produced causally, in a randomised design, and that it appears far faster than anyone had assumed. It also moves the mechanism from knowledge to persistence, which is a different and more portable claim.
What it does not support: Anything about sustained professional practice. These are short online tasks and the measured effect is a within-session carry-over rather than skill decay, so it cannot show whether the effect compounds, persists beyond the session or transfers to complex work. A preprint, not peer reviewed.

deskilling · persistence · unaided performance · learning · productive struggle · the best writing on ai does using ai stop you learning is ai making us stupid

Working paper#

Liu, J., Sweet, T., Chen, M. H., Engelberg, J., Masters, M. C., Clark, M., Persaud, A., Lancaster, A., Hollingsworth, J. K. and Rice, J. K. (2026). The Effects of Course-Integrated AI Tutoring on Student Performance and Engagement: A Randomized University Trial #

Annenberg Institute EdWorkingPaper 26-1598, October 2026, manuscript dated 30 September 2026. Abstract and paper read at source 9 October 2026

Method: Group-randomised trial at instructor level within course blocks, Fall 2025, University of Maryland, College Park: 2,379 undergraduates and 30 instructors. The intervention was access to a GPT-4o-based virtual study assistant tied to course materials. The headline estimate comes from an exact-match subsample of 22 instructors and 1,353 students.
Finding: 'Among sections of the same course, tutor access reduced final grades by 0.37 standard deviations' (4.12 points) 'and learning management system participation by 0.90 standard deviations'. Implied effects were 5.10 points lower for first-generation students and 2.23 for others. 'Only 14.63% of treated students ever used the VSA.' In the full sample 'there is not significant evidence of adverse treatment effects'.
What it supports: That giving students access to a university's own AI tutor did not raise grades in this trial and, among sections of the same course, went with lower grades and much lower recorded participation.
What it does not support: That using the tutor lowers learning: the effect is of access, under 15 per cent of students used it, the outcome is course grades and not an independent test, and the full-sample estimate is not significant. The provost who commissioned the trial is a co-author. No funding statement. Not peer-reviewed.

AI tutor · higher education · randomised trial · engagement · learning · capability debt

Compiled review#

Multiple Freedom of Information investigations: The Times, The Student Eye and The Scotsman (2026). Recorded penalties for AI misuse in UK universities #

Three separate FOI investigations covering different institutions and different academic years. They must not be aggregated

Method: Freedom of Information requests to universities, compiled by three outlets. The Times to the 24 Russell Group members for 2024-25; The Student Eye to the University of Bristol, May 2025; The Scotsman to Scottish institutions via Miles Briggs MSP, March 2025, covering 2023-24.
Finding: Russell Group: 2,053 recorded punishments in 2024-25 against roughly 700 the year before, from about 350,000 students, with four members disclosing expulsions (UCL, Imperial, Glasgow and Leeds). Bristol: 526 penalties in 2023-24, against 153 the previous year and 7 in 2021-22. Scotland: 1,051 cases in 2023-24 against 131 in 2022-23, of which Abertay alone recorded 351 cases with 342 upheld.
What it supports: That formal academic penalties for AI misuse have risen steeply and are now numbered in thousands, and that a student who assumes this is theoretical is wrong.
What it does not support: A national picture, a trend in behaviour, or one dataset. Seven of the 24 Russell Group universities do not record AI investigations at all, so 2,053 is a floor across an incomplete sample. The three sets cover DIFFERENT YEARS, 2024-25 for the Russell Group and 2023-24 for Scotland and Bristol, and they overlap institutionally, so they cannot be added together. Most importantly the universities' own position is that much of the rise reflects newly created recording categories rather than more cheating: Abertay introduced 'unacceptable AI use' as a category only in 2023 and accounts for a third of the Scottish total. Cases, penalties and upheld findings are three different counts and are routinely conflated in coverage.

assessment · detection · academic misconduct · how to use ai at university does ai detection work

Working paper#

Northcutt, C., Hasmani, I., Feng, K., Khangi, T., Plesner, A. and Mueller, J. (Handshake AI Research) (2026). StudentBench: AI and human tutoring yield equivalent GRE learning gains #

arXiv 2609.28470, submitted 23 September 2026. Preprint, not peer reviewed. Handshake AI funded the work. Read at source 24 September 2026

Method: Randomised controlled trial with 2,383 adult participants recruited from Handshake's own platform, mostly aged 18 to 23, from about 70,000 invited. Three arms: one hour of text-based AI tutoring (thirteen models tested, with lesson planning, conversational tutoring and generated practice problems), one hour with an expert human GRE tutor on live video, or one hour of educational videos as control. A 27-question GRE pre-test, then a post-test of different questions with no tutor present. Equivalence tested by two one-sided tests with bounds of plus or minus 0.25 pooled standard deviations, about 4.09 percentage points. A secondary analysis of 2,028 pairwise expert evaluations.
Finding: Against control, AI tutoring added 6.86 percentage points on the quantitative section (95 per cent CI 4.02 to 9.69) and 5.47 on the verbal (2.46 to 8.47). AI and human tutoring gains were statistically equivalent within the preset bounds, p = .015. In five of seven GRE domains the best-performing AI tutor exceeded the human tutors on average. On the authors' costing, with human tutors at 75 dollars an hour, the smallest model to reach equivalence did so at 918 times lower cost per percentage point of gain (0.0052 dollars against 4.81). Faster AI responses correlated with more student messages, more correct practice and larger gains.
What it supports: That in one large randomised trial, one hour of AI tutoring produced immediate test gains on standardised questions indistinguishable, within a quarter of a standard deviation, from one hour with a paid expert human tutor, and that the result held across several models.
What it does not support: Anything about retention or transfer: the post-test was immediate and the authors write that they 'measured immediate learning gains without evaluating whether they persist over months'. There was no arm in which students worked through practice problems alone, so the gain either tutor adds over self-study is unknown. Participants were paid adults literate in English on one platform; the human tutors were the ones the company recruited; the funder supplies human experts to AI developers. Equivalence on average is not identity, and a GRE item is one kind of learning.

AI tutoring · learning · randomised trial · equivalence · human tutoring · is an ai tutor as good as a human tutor

Working paper#

Poudyal, B. (2026). Mapping the Authorized Boundary: A Comparative Policy-Vignette Study of Generative AI Governance in Australian Higher Education #

arXiv 2609.29689, submitted 30 August 2026 and listed 25 September 2026. Abstract read at source 28 September 2026

Method: Fifteen standard vignettes of student generative AI use tested against the published policies of 20 Australian universities, 300 university-case classifications, coded twice, distinguishing binding instruments from the whole official policy environment.
Finding: 120 combinations (40.0 per cent) clearly prohibited, 97 (32.3 per cent) potential breaches, 27 (9.0 per cent) permitted with conditions and 56 (18.7 per cent) indeterminate; none clearly permitted. Coder agreement 57.3 per cent, Cohen's kappa 0.395. Written policies regulate retention of AI-generated text more clearly than process-only assistance.
What it supports: That the written rules of twenty universities cannot be read consistently even by trained coders, and say least about the uses students make most.
What it does not support: What happens in tutorials and departments, where indeterminate cases may be resolved; anything outside Australia; the paper is a preprint.

academic integrity · AI policy · higher education · assessment · should universities let ai mark students work

Working paper#

Sankaranarayanan, S. (2026). Mitigating "Epistemic Debt" in Generative AI-Scaffolded Novice Programming using Metacognitive Scripts #

arXiv 2602.20206, 22 February 2026, revised 31 March 2026

Method: Between-subjects experiment, 78 participants recruited via Prolific and UserInterviews.com, using a custom Cursor IDE plugin backed by Claude 3.5 Sonnet. Three conditions: manual control, unrestricted AI, and scaffolded AI. Followed by a 30-minute AI-blackout maintenance task. Preprint, not peer reviewed.
Finding: Both AI groups outperformed the manual control on functional utility (p < .001) and did not differ from each other (p = .64). On the subsequent AI-blackout maintenance task, unrestricted AI users failed at 77 per cent against 39 per cent for the scaffolded group. The author describes "fragile experts": developers whose high functional utility masks critically low corrective competence.
What it supports: That the gap between producing output and being able to repair it can be created inside a single session, and that interface design changes the size of that gap. The scaffolded condition roughly halved the failure rate.
What it does not support: A durable effect on capability. It is a single session with a single blackout task, in novice programming, with no longitudinal follow-up. The author puts "epistemic debt" in quotation marks in his own title and builds explicitly on Kirschner, so it is not offered as a new construct.

epistemic debt · scaffolding · cognitive offloading · novice programming · cognitive debt and capability debt will ai replace programmers what happens to medical and legal training what is the new imposter syndrome

Institutional survey#

Schwartz, H. L. and Diliberti, M. K., RAND (2026). More Students Use AI for Homework, and More Believe It Harms Critical Thinking: Selected Findings from the American Youth Panel #

RAND research report RRA4742-1, 17 March 2026; discussed in a RAND interview with Diliberti published 30 September 2026. Both read at source 2 October 2026

Method: RAND American Youth Panel survey of 1,214 young people aged 12 to 29, fielded in December 2025.
Finding: 67 per cent of students endorsed the statement 'The more students use AI for their schoolwork, the more it will harm their critical thinking skills.' With the exception of using AI to get answers for homework, most did not regard their school-related uses as cheating. In the September interview Diliberti said only about a third of students reported schoolwide rules on AI and that many did not know.
What it supports: That most US students surveyed use AI while believing it harms their thinking, and that few report clear school rules.
What it does not support: Any effect on learning: these are self-reports. The exact share reporting schoolwide rules was not visible on the report page read. United States only.

AI in education · students · critical thinking · school policy · should schools ban ai

Working paper#

Shen, J. H. and Tamkin, A. (2026). How AI Impacts Skill Formation #

arXiv:2601.20245v1 [cs.CY], submitted 28 January 2026, CC BY-NC-ND 4.0. Pre-registered at osf.io/pk6a5; code and annotations at github.com/safety-research/how-ai-impacts-skill-formation. Work conducted under the Anthropic Fellows Program, and summarised by Anthropic on 29 January 2026 at anthropic.com/research/AI-assistance-coding-skills. Preprint, not peer reviewed. Read at source 14 September 2026

Method: Between-subjects randomised experiment, 52 participants, 26 per arm, learning Trio, a Python library for asynchronous programming none of them had used, through a self-guided tutorial with starter code and a concept brief. Maximum 35 minutes on two coding tasks, then a 14-question, 27-point quiz taken without AI, covering debugging, code reading and conceptual understanding; code-writing questions were excluded by design to keep syntax errors out of the score. The treatment arm had an AI assistant in the sidebar, based on GPT-4o, with access to the participant's code and able to produce the full correct solution on request. Recruitment was through a third-party crowd-worker platform at a flat fee of 150 US dollars, screened for more than a year of weekly Python use, some prior exposure to AI coding assistance, and no prior Trio use. Screen recordings of 51 participants were annotated by hand. Four pilots preceded the main study.
Finding: The assisted arm averaged 50 per cent on the quiz against 67 per cent for the hand-coding arm, a difference of 4.15 points on 27, Cohen's d of 0.738 at p = 0.010, holding at d = 0.725, p = 0.016 with warm-up time as a covariate. The widest gap was on debugging questions and the narrowest on code reading; the authors' explanation is that the control arm hit more errors and got better at resolving them, and the error counts support it, with a median of 1.0 total errors in the AI arm against 3.0 without. The AI arm finished about two minutes faster and that difference was not statistically significant. Inside the treatment arm, six annotated interaction patterns split cleanly: wholesale delegation (n=4), progressive reliance (n=4) and iterative AI debugging (n=4) averaged under 40 per cent, while generation-then-comprehension (n=2), hybrid code-and-explanation (n=3) and conceptual inquiry (n=7) averaged 65 per cent or higher. Explanations were the most common query type at 79, asked by 21 of 25; code generation accounted for 51 queries from 16 of 25, and four of those participants asked for nothing else.
What it supports: That withdrawing the assistance and testing unaided produces a measurable comprehension penalty in professional developers meeting unfamiliar material, on a pre-registered primary outcome. The interaction-pattern split is the most useful thing in it, and points at how the tool is used rather than whether it is used, which is where the guardrailed arms of the school and programming trials arrived from the other direction.
What it does not support: That AI use degrades professional expertise. One library, one tutorial, one hour, a preprint, and a sample of 52. The interaction-pattern analysis was NOT pre-registered and the authors state that it draws no causal link, so it cannot be quoted as evidence that asking for explanations causes better retention. The assistant was a chat sidebar, and the authors expect an agentic tool to produce a larger effect, so this is a floor. Whether an immediate quiz score predicts longer-term skill is, in their words, an important question the study does not resolve. The conflict of interest runs against the finding rather than towards it, since the work was done under an AI company's fellowship and reports harm, but it is disclosed here either way. ONE DESCRIPTION IN CIRCULATION DOES NOT COME FROM THE PAPER. Anthropic's own write-up calls the participants '52 (mostly junior) software engineers' and secondary coverage repeats it; the paper's balance table reports 15 of 26 in the treatment arm and 14 of 26 in the control at seven or more years of experience, and two per arm at one to three years. The result is therefore about developers meeting unfamiliar material, not about juniors, which widens it rather than weakening it. Checked at both sources on 14 September 2026.

skill formation · learning · capability debt · scaffolding · persistence · capability debt does using ai stop you learning is ai making us stupid how do we train people to use ai

Institutional survey#

Stephenson, R. and Armstrong, C. (2026). Student Generative AI Survey 2026 #

HEPI Report 199, Higher Education Policy Institute with Kortext, published 12 March 2026. The third annual edition; the 2024 and 2025 editions were written by Josh Freeman

Method: 1,054 full-time undergraduates polled through the Savanta panel in December 2025, weighted on gender, institution type and year of study, margin of error about 3 per cent. The 2024 wave was polled by UCAS rather than Savanta, a change inside the trend line.
Finding: 95 per cent report using AI in at least one way and 94 per cent say they have used generative AI to help prepare assessed work, against 89 per cent in 2025 and about 53 per cent in 2024. What they use it for is mostly comprehension: explaining concepts 61 per cent, summarising an article 49, suggesting research ideas 40, structuring thoughts 39. Including AI-generated text directly in assessed work is 12 per cent, up from 8 in 2025 and 3 in 2024. 68 per cent think AI skills are essential, and only 36 per cent feel encouraged by their institution to use AI.
What it supports: That AI use among full-time UK undergraduates is close to universal in preparation, and that institutional encouragement lags a long way behind student behaviour.
What it does not support: That 94 per cent of students put AI into work that gets marked. THIS IS THE MISREADING THE FIGURE INVITES and the distinction is the whole point of the survey: the 94 covers everything from explaining a concept to drafting, and the number for including AI-generated text directly is 12. Anyone writing 'uses AI on assessed work' has converted a 12 per cent finding into a 94 per cent one with one preposition. It is also self-report, an ever-done lifetime measure rather than current practice, a commercial panel rather than a census, full-time students only, and HEPI warn that new response options were added in 2026 which mechanically makes the 94 easier to reach.

assessment · adoption · student behaviour · how to use ai at university how to assess students when ai can do the assignment

Working paper#

Stromberg, D., Lei, V. and Wu, Y. (2026). The Generative AI Learning Penalty: Evidence from Chinese Secondary Education #

CEPR Discussion Paper 21577, CEPR Press, published 2 June 2026. Working paper, not peer reviewed

Method: Thirty months of panel data on 26,811 Chinese students in grades 7 to 12, combining monthly closed-book exams, high-school and college entrance exams, and homework scores and completion time across nine subjects. Staggered AI adoption in a difference-in-differences design. The outcome measures are closed-book and invigilated, so they record unaided performance by construction.
Finding: AI adoption raises homework scores by 18 per cent and reduces completion time by 30 per cent, and lowers monthly exam scores by 20 per cent within six months. High-stakes entrance-exam scores fall by 18 and 24 per cent, with the full penalty emerging only after about two years. Losses are largest in social science, then STEM, then languages, and are especially large for junior students, high-achieving students and boys. They concentrate among roughly 80 per cent of AI users whose behaviour is consistent with homework outsourcing, indicated by very short completion time coupled with high homework scores. Users who maintain similar completion time to non-users experience small losses.
What it supports: That the gap between assisted output and unaided capability can be measured at scale over years rather than minutes, and that it widens rather than closing. It also locates the damage in delegation rather than in access: the students who kept working at their normal pace were largely spared.
What it does not support: Causation with the confidence of a randomised trial. Adoption is self-selected and staggered rather than assigned, the outsourcing split is inferred from time-on-homework rather than observed, and a two-year lag makes contemporaneous confounds harder to exclude. One country, one school system, secondary students. Nothing about professional work. Not peer reviewed.

deskilling · unaided performance · learning · productive struggle · assessment · the best writing on ai does using ai stop you learning

Institutional survey#

Tench, B., Weinstein, E. and James, C., with Starks, A. and Konrath, S. (2026). An AI Policy Isn't a Playbook: Educators Need Agency, Not Just New Rules #

Center for Digital Thriving, Harvard Graduate School of Education, September 2026. Report read at source 2 October 2026

Method: Survey of 719 teachers and 299 principals through the RAND American Educator Panels, April to May 2025; interviews with 12 educators and with 31 young people aged 15 to 19, April to June 2026.
Finding: Of the AI dilemmas educators described in their own words, 74 per cent of teacher descriptions (171) and 69 per cent of principal descriptions (85) were coded as cheating-related. Students reported being falsely accused, and some 'have begun to cap their effort, or perform mediocrity on purpose to not get flagged'.
What it supports: That cheating dominates how US educators describe their AI problem, and that some students say they respond to detection by doing worse work on purpose.
What it does not support: How common deliberate under-performance is: it rests on 31 interviews. The percentages are shares of the dilemmas described, not of all educators surveyed. United States only.

AI in education · academic integrity · AI detection · students · should schools ban ai

Peer-reviewed#

Topaz, M., Roguin, N., Gupta, P., Zhang, Z. and Peltonen, L.-M. (2026). Fabricated citations: an audit across 2.5 million biomedical papers #

The Lancet, 407, 1779-1781. DOI 10.1016/S0140-6736(26)00603-3. Online 7 May 2026. Correspondence rather than a full article. Led from Columbia University School of Nursing and Data Science Institute

Method: The CITADEL pipeline over 2.5 million papers in the PubMed Central Open Access collection, January 2023 to 18 February 2026, extracting 125.6 million references of which 97.1 million carried a verifiable identifier. Title and identifier were compared; the 30,812 mismatches were routed by a language model into categories, then each flagged title was searched in PubMed, Crossref, OpenAlex and Google Scholar. A reference counted as fabricated only if it appeared in none of them.
Finding: 4,046 fabricated references across 2,810 papers. The rate of papers carrying at least one rose from 1 in 2,828 in 2023 to 1 in 458 in 2025 and 1 in 277 in the first seven weeks of 2026, a twelvefold increase. Review articles ran 57 per cent higher than other paper types. At the time of the audit 98.4 per cent of affected papers had received no publisher action.
What it supports: That references to studies which do not exist are entering the peer-reviewed literature at a rising rate, measured rather than asserted, and that the correction machinery is not catching them.
What it does not support: A rate for PubMed. The audit covers the PubMed Central OPEN ACCESS collection, and Nature published a correction to its own coverage on 13 May 2026 making exactly this distinction. The 2026 figure is seven weeks and the authors mark it as an incomplete observation period. It is a floor rather than a ceiling, because 28.5 million references, 22.7 per cent, lacked verifiable identifiers and were excluded. It does not establish intent: Topaz has said 91 per cent of affected papers carried only one or two, many likely honest mistakes by authors who did not check AI output. Cochrane and Northwestern have both criticised the method publicly, on transparency and on not separating citations that matter to a conclusion from those that do not. NOTE, the paper itself could not be read at source, so no limitation is quoted here as the authors' own words.

verification · the verifier's discount · fabricated citations · provenance · how to use ai at university proving you did the work is ai dangerous

Human and AI collaboration

What actually happens when a person and a model work together.

Peer-reviewed#

Bainbridge, L. (1983). Ironies of Automation #

Automatica, 19(6)

Method: Theoretical analysis of automated process control systems and the human roles left within them.
Finding: Automating the routine parts of a task leaves the human with the hardest residue, monitoring and exception handling, while removing the routine practice that built the competence to do it. Automation makes the remaining human role harder, not easier.
What it supports: That monitoring is a demanding task rather than a light one, and that the design of automation determines whether the human retains the capability to supervise it.
What it does not support: Anything specific to AI. It is a process-control argument from 1983, and its application to generative systems is by analogy rather than by measurement.

ironies of automation · monitoring · deskilling · oversight · human in the loop is not a safeguard what is deskilling delegation boundary map how do you design a stop button people will use how do humans and agents divide work what is the shared prompt review what is automating versus informating what are the ironies of automation what is the vigilance decrement what is alarm fatigue do employees know they are expected to override ai how much can one firm check

Argued perspective#

Wooldridge, M. and Jennings, N. R. (1995). Intelligent agents: theory and practice #

The Knowledge Engineering Review, 10(2), June 1995, 115-152. Cambridge University Press. Volume, issue, month, year and page range confirmed in both directions against the publisher's own issue listing and the article's own title page. Cambridge Core full text is paywalled; the text quoted here was read in the authors' own HTML at Oxford and cross-checked against a typeset published-version PDF, and the two differ only in one citation on the autonomy bullet and one word in a lead-in sentence.

Method: A survey and roadmap of the agent literature to 1995, proposing a two-level definition of agency and reviewing theories, architectures and languages. Reports no new data.
Finding: Sets out a weak notion of agency as a computer system with four properties, in the authors' words: autonomy, 'agents operate without the direct intervention of humans or others, and have some kind of control over their actions and internal state'; social ability, 'agents interact with other agents (and possibly humans) via some kind of agent-communication language'; reactivity, agents 'perceive their environment ... and respond in a timely fashion to changes that occur in it'; and pro-activeness, 'agents do not simply act in response to their environment, they are able to exhibit goal-directed behaviour by taking the initiative'. A stronger notion adds mentalistic concepts such as knowledge, belief, intention and obligation. The paper opens by recording that the term 'defies attempts to produce a single universally accepted definition' and warns that 'unless the issue is discussed, agent might become a noise term, subject to both abuse and misuse'.
What it supports: That the definitional problem is thirty-one years old rather than new, and that the most durable answer to it is a property list agreed before any of the current technology existed.
What it does not support: Anything about large language models, which postdate it by a quarter of a century. The four properties are a proposal by two authors in a survey article; they carry no standing beyond the field's long agreement with them, and nothing in the paper measures whether a system holding all four behaves better than one holding three.

AI agent · autonomy · agency · multi-agent systems · what is an ai agent

Peer-reviewed#

Helmreich, R. L., Merritt, A. C. and Wilhelm, J. A. (1999). The Evolution of Crew Resource Management Training in Commercial Aviation #

International Journal of Aviation Psychology, 9(1)

Method: Historical and evaluative review of CRM programmes from 1979 onwards.
Finding: CRM began with a 1979 NASA workshop prompted by an NTSB finding that a captain had failed to accept input from junior crew. Line audits show CRM produces the intended behavioural change, though measured attitudes decay over time even with recurrent training.
What it supports: That an industry built a training response to a named human-factors failure and measured behaviour rather than opinion.
What it does not support: That CRM reduces accidents. The authors state directly that accident rates are too rare to serve as a validation criterion.

· what professions can learn from aviation ai and human disagreement

Peer-reviewed#

Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E. and Nelson, C. (2000). Clinical versus mechanical prediction: a meta-analysis #

Psychological Assessment, 12(1), 19-30

Method: Meta-analysis of 136 studies comparing clinical (expert, in-the-head) with mechanical (formula, consistently applied) prediction of human health and behaviour.
Finding: Mechanical prediction was superior in roughly half the comparisons, approximately equal in roughly half, and inferior in a small minority, with an average advantage of about ten per cent. The modal result is a tie.
What it supports: That consistency, rather than knowledge, is what a formula contributes; and that the advantage is real, directional and modest.
What it does not support: That algorithms beat experts, which is how it is usually quoted. It says nothing about a human reviewing a machine's output, and nothing about stochastic generative systems, which are neither transparent nor reproducible in the way an actuarial rule is.

judgement · prediction · expertise · clinical versus actuarial judgement models of judgement

Peer-reviewed#

Dekker, S. W. A. and Woods, D. D. (2002). MABA-MABA or Abracadabra? Progress on Human-Automation Co-ordination #

Cognition, Technology & Work, 4(4), 240-244

Method: Conceptual analysis of function allocation methods in human factors. No data, no participants, no experiment.
Finding: Substitution-based function allocation, of which the Fitts list is the archetype, cannot deliver human-automation coordination, because the effects of automation are qualitative rather than quantitative. The authors name the underlying assumption the substitution myth and write that capitalising on a strength of automation does not replace a human weakness but creates new human strengths and weaknesses, often in unanticipated ways. Allocating a function also creates new functions for the other partner that did not exist before. They add that neither the list nor much of the supervisory control literature explains the cognitive work involved in deciding how and when to intervene or how to switch from level to level.
What it supports: That splitting the tasks is the wrong unit of design, and that the coordination at the boundary has been the acknowledged gap in this literature since 2002.
What it does not support: How large the effect is, or anything measurable at all. It is an argument. Its supporting accident examples are cited rather than analysed.

function allocation · substitution myth · human-automation coordination · handoffs · how do humans and agents divide work the use or keep test

Peer-reviewed#

Lee, J. D. and See, K. A. (2004). Trust in automation: designing for appropriate reliance #

Human Factors, 46(1), 50-80

Method: Theoretical review synthesising the trust and automation literature into a model of trust, reliance and calibration.
Finding: Trust is an attitude, reliance is the behaviour it produces, and appropriate reliance requires trust calibrated to the system's actual capability. Miscalibration runs in both directions: disuse and misuse.
What it supports: The vocabulary this territory now runs on, and the reason a policy naming a human reviewer is not yet a control.
What it does not support: Any intervention that reliably produces calibration. It is a synthesis rather than a trial, and the interventions tried since have mixed or null results.

trust · oversight · automation bias · what is appropriate reliance

Peer-reviewed#

Dietvorst, B. J., Simmons, J. P. and Massey, C. (2015). Algorithm Aversion: People Erroneously Avoid Algorithms After Seeing Them Err #

Journal of Experimental Psychology: General, 144(1)

Method: Five experiments.
Finding: After seeing an algorithm err, people abandon it even when it demonstrably outperforms them.
What it supports: That trust in a model moves for reasons unrelated to its accuracy.
What it does not support: That this holds for conversational AI, which is far more recent and feels different to use.

algorithm aversion · trust · human ai decision making ai and human intuition ai and expert judgement

Compiled review#

Strauch, B. (2018). Ironies of Automation: Still Unresolved After All These Years #

IEEE Transactions on Human-Machine Systems, 48(5), 419-433. DOI 10.1109/THMS.2017.2732506. Manuscript received 4 January 2017, revised 1 May 2017, accepted 10 June 2017. Read at source 16 September 2026 in IEEE's accepted-article proof, which carries the full text and not the final pagination

Method: A narrative retrospective on Bainbridge (1983), setting her ironies against statutory accident investigations in aviation, rail, shipping and pipelines, plus a Google Scholar citation count. Cases are selected because they illustrate the ironies, so there is no sampling frame and no denominator. Reports no new data.
Finding: Google Scholar listed 1800 works citing Ironies of Automation as of early November 2016, against 564 for Wiener and Curry (1980) and 488 for Norman (1990), with ten further citing works appearing in a fortnight. Strauch reproduces Bainbridge's ironies in sequence with page citations and argues they are unresolvable rather than merely unresolved while systems can fail and human involvement is required. He names three later ironies: automation can disguise a shortcoming an operator already had; a minor anomaly can become catastrophic through the operator's attempt to resolve it with the same automation; and a person can become qualified to operate an automated system without the expertise to understand it.
What it supports: That the 1983 argument has been carried into the present by a reviewed source that engages with its evidence. For anyone without Elsevier access it is also the traceable route to Bainbridge's own words, recording at p. 775 and p. 776 the exact wording this research quotes.
What it does not support: Any rate. The accident narratives show the mechanism occurring and cannot show how often it occurs, and Strauch concedes in his own text that the fall in fatal US commercial jet accidents may be a short-term statistical aberration. His restatement that 'research demonstrating that visual monitoring quality deteriorates after about 30 min' carries NO CITATION, and Mackworth's name appears nowhere in the paper, so it must not be cited as a source for the half-hour figure. See the vigilance decrement page.

ironies of automation · monitoring · automation complacency · oversight · what are the ironies of automation what is the vigilance decrement

Argued perspective#

Elish, M. C. (2019). Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction #

Engaging Science, Technology, and Society, 5 (2019), 40-60, published 23 March 2019. DOI 10.17351/ests2019.260. Data and Society Research Institute. Open access under CC BY-NC-ND. Read at source 18 September 2026, both the journal's own PDF and its article record, which classifies the piece under its own DC.Type as comparative textual analysis. Graded on this date because the research has argued from this concept on several pages for months without the paper ever entering the evidence base, which qa_citations cannot catch: it checks that every cited URL has an entry and never that a load-bearing concept has one. Not the term's first appearance: the argument is stated earlier in Elish and Hwang (2015) for Data and Society, which is co-authored, so the term is Elish's and 2019 is the citable statement of it.

Method: Comparative textual analysis of several high-profile accidents involving complex automated socio-technical systems, together with the media coverage that followed. No new data is generated.
Finding: Introduces the moral crumple zone, in the author's words, to describe how responsibility for an action <q>may be misattributed to a human actor who had limited control over the behavior of an automated or autonomous system</q>. The analogy is load-bearing and asymmetric: a car's crumple zone protects the driver, whereas <q>the moral crumple zone protects the integrity of the technological system, at the expense of the nearest human operator</q>.
What it supports: That the pattern is describable and has recurred across named, documented cases, and it gives the research the vocabulary for what happens to a person placed in an oversight role they cannot discharge. The concept is now standard in AI accountability writing.
What it does not support: Any rate. It is a concept paper working from selected accidents and their press coverage, chosen because they exhibit the pattern, so it establishes that the arrangement occurs and never how often. It also measures nothing about the operators themselves.

moral crumple zone · accountability · oversight · automated decision · what is a moral crumple zone do employees know they are expected to override ai pricing work when production is cheap

Practitioner framework#

Kahneman, D., Lovallo, D. and Sibony, O. (2019). A structured approach to strategic decisions #

MIT Sloan Management Review, 60(3)

Method: Practitioner framework: decompose a decision into independent mediating assessments, score each separately, and delay the overall judgement until the end.
Finding: The authors argue that the structure that improves hiring interviews, independent assessment before global judgement, transfers to strategic decisions.
What it supports: A usable protocol, and the clearest statement of why an overall judgement made early contaminates everything after it.
What it does not support: Measured improvement on strategic decisions. The supporting evidence is the structured interview literature; the transfer is argued by analogy and has not been trialled.

decision quality · judgement · noise · models of judgement what is a noise audit

Peer-reviewed#

Logg, J. M., Minson, J. A. and Moore, D. A. (2019). Algorithm Appreciation: People Prefer Algorithmic to Human Judgment #

Organizational Behavior and Human Decision Processes, 151, 90-103, 2019. DOI 10.1016/j.obhdp.2018.12.005. Received 20 April 2018, accepted 4 December 2018. Harvard Kennedy School and Haas School of Business, Berkeley. Read in full at source on 17 September 2026 in the first author's own hosted copy, which carries the journal pagination, the DOI and both dates.

Method: Advice-taking experiments measuring Weight on Advice, the distance a participant moves from their first estimate towards advice, divided by the distance between the two. The advice is identical across conditions and only the source label changes. The abstract says six experiments; SEVEN numbered studies are reported (1A, 1B, 1C, 1D, 2, 3, 4) plus two further samples, an MBA definition sample of 77 and a benchmark study of 671. Ns: 1A 202, 1B 215, 1C 286 (MTurk), 1D 119 researchers, 2 154 university participants, 3 403, 4 301 MTurk plus 70 US national security professionals. Sample sizes set a priori by power analysis; 1B, 1C, 1D, 2, 3 and 4 pre-registered, 1A not, which the authors state; pre-registrations, materials and data on the Open Science Framework at https://osf.io/b4mk5/. A PRINTED INCONSISTENCY: the summary table gives study 1D as N=199 while the method text gives 119 from 120 completed surveys, and every 1D statistic carries df of 117 or 118, so 119 is the figure the analysis supports.
Finding: Identical advice was weighted more heavily under an algorithmic label. 1A: 0.45 against 0.30, F(1,200)=8.86, p=.003, d=0.42. 1C: 0.38 against 0.26, t(284)=3.50, p=.001, d=0.44. 1B on chart positions, beta=-0.34, t(214)=5.39, p<.001. Study 2: 0.50 against 0.35 seen separately, and 75 per cent chose the algorithm when both were offered. Study 3, using Dietvorst's own materials, 88 per cent chose the algorithm over another participant and 66 per cent over their own estimate, z=6.62, p<.001, so the effect shrinks when the rival is the self. Study 1D: academic researchers predicted the opposite result, mean 0.14 in the aversion direction against participants' observed appreciation, t(118)=14.03, p<.001, d=1.25, with graduate students no better calibrated than senior researchers. Study 4: 70 national security professionals discounted all advice more than a lay sample, interaction F(1,338)=5.05, p=.025, and were LESS accurate for it on Brier scores, interaction F(1,366)=4.16, p=.042. The authors' own words: algorithmic advice 'falls on deaf expert ears, with a cost to their accuracy'.
What it supports: Together with Dietvorst, that miscalibration runs in both directions and cannot be fixed by telling people to use judgement. Separately, that a published literature had been read as saying the opposite of what people do, and that the researchers who knew it best predicted wrongly by more than a standard deviation. That is the research's clearest instance of a field's consensus about behaviour failing against a direct test of the behaviour.
What it does not support: Which tendency dominates in any given workplace, and three further limits the authors state themselves. The expert result rests on 61 experts in the main analysis, 67 of the 70 were men, and the authors say the two samples 'likely differ in many aspects beyond just expertise in forecasting'. The human comparison is always a peer or an average of peers and never a human expert, which they say may be preferred strongly enough to swamp any appreciation of algorithms. And it does NOT show that people over-rely: in the benchmark study of 671, where the algorithm averaged 314 people and the correct weight was 100 per cent, participants fell short by a mean of 0.66, F(1,669)=275.08, p<.001, d=1.30, so BOTH labels were underweighted. The word 'algorithm' also meant something else in 2018: asked to define it, participants said mathematics, an equation or a calculation 42 per cent of the time and a step-by-step procedure 26 per cent. Nothing here tests a conversational model.

algorithm appreciation · trust · human ai decision making ai and expert judgement what is algorithm appreciation what is algorithm aversion

Practitioner framework#

Shrestha, Y. R., Ben-Menahem, S. M. and von Krogh, G. (2019). Organizational decision-making structures in the age of artificial intelligence #

California Management Review, 61(4), 66-83

Method: Framework proposed in an edited management venue, setting out three structures for combining human and machine decision-making and five criteria for choosing between them.
Finding: Proposes full delegation to the machine, hybrid sequential structures in either order, and aggregated human-machine decisions, chosen on the specificity of the decision space, the interpretability of process and outcome, the size of the alternative set, decision speed, and replicability.
What it supports: That the choice of structure can be made on stated criteria rather than on enthusiasm. It is the most rigorous published criteria set at the level of an organisational decision.
What it does not support: That the criteria have been tested, or that they help the individual deciding what to do with a specific piece of work. No data is reported.

delegation · decision rights · human and AI collaboration · the use or keep test models of judgement

Peer-reviewed#

Zhang, Y., Liao, Q. V. and Bellamy, R. K. E. (2020). Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making #

Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT* 20)

Method: Two online experiments on an income-prediction task from US census data, 40 trials each. Experiment 1: 72 Mechanical Turk participants, nine per cell in a two by two by two design. Experiment 2: nine further participants, analysed against two Experiment 1 cells for a total of 27. Human unaided accuracy 65 per cent, model accuracy 75 per cent.
Finding: Showing confidence scores significantly increased trust, F(1,64)=4.64, p=.035, and significantly improved trust calibration when model confidence was above 80 per cent, F(4,256)=15.8, p<.001. There was no significant difference in AI-assisted accuracy across the prediction and confidence conditions, a result the authors report as rejecting their own hypothesis. Local explanations produced no significant change against baseline on switching behaviour or accuracy, with a reverse trend on accuracy.
What it supports: That a confidence score moves how a person feels about a system more reliably than it moves what they catch, and that explanation alone did neither in this setting.
What it does not support: Much on its own. Nine participants per cell in Experiment 1 and 27 in Experiment 2, no effect sizes, confidence intervals or standard deviations reported, no exclusions or attention checks described, non-expert participants, and a contrived task carrying no responsibility. The authors also note the approach depends on the model's probabilities being well calibrated in the first place, which the calibration literature says they usually are not.

confidence · explanation · trust calibration · over-reliance · how should an ai agent communicate uncertainty

Peer-reviewed#

Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T. and Weld, D. (2021). Does the whole exceed its parts? The effect of AI explanations on complementary team performance #

CHI Conference on Human Factors in Computing Systems 2021

Method: Controlled experiments on human-AI team performance across tasks, comparing conditions with and without explanations of the model's output.
Finding: Explanations increased the rate at which people accepted the model's recommendation without improving the accuracy of the human-AI team. Acceptance rose for correct and incorrect outputs alike.
What it supports: That explanation is not a mechanism for calibrated reliance, and that the intuitive fix for oversight can make the measured outcome worse.
What it does not support: That explanations are worthless. They serve contestability, auditability and legal duties, which this study does not measure.

explainability · human in the loop · oversight · what is appropriate reliance does explaining an ai decision help

Peer-reviewed#

Bucinca, Z., Malaya, M. B. and Gajos, K. Z. (2021). To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making #

Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), April 2021. DOI 10.1145/3449287

Method: Experiment with 199 participants comparing three cognitive forcing interventions, designed from dual-process theory to compel more thoughtful engagement with AI-generated explanations, against two simple explainable-AI approaches and a no-AI baseline. Includes an audit for intervention-generated inequalities using the Need for Cognition scale.
Finding: Cognitive forcing significantly reduced overreliance compared with the simple explainable-AI approaches. Participants gave the least favourable subjective ratings to the designs that reduced overreliance the most. On average the interventions benefited participants higher in Need for Cognition more, so human cognitive motivation moderates the effectiveness of explainable AI. The authors argue people rarely engage analytically with each individual recommendation and instead develop general heuristics about when to follow the AI.
What it supports: That deliberately adding friction to an AI-assisted decision measurably reduces acceptance of wrong suggestions, and that the intervention people dislike most is the one that works best.
What it does not support: That slower decisions produce better organisational outcomes. This is a controlled task with 199 participants, not a field study, and it measures overreliance rather than downstream results. The uneven benefit by Need for Cognition means the effect will not be uniform across a workforce.

cognitive forcing functions · overreliance · automation bias · explainability · which decisions should become slower because of ai how do you design a stop button people will use how do we train people to use ai

Working paper#

Dell'Acqua, F. et al. (2023). Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality #

Harvard Business School and BCG working paper

Method: Field experiment, 758 BCG consultants, tasks inside and just outside GPT-4's competence.
Finding: Inside the frontier, AI-assisted consultants were dramatically better and faster. Outside it, they performed worse than consultants with no AI at all.
What it supports: That model competence is jagged rather than smooth, and that confident output suppresses scrutiny at exactly the wrong moment.
What it does not support: Where the frontier runs in your domain. That is local and must be learned.

jagged frontier · automation bias · verification · human ai decision making why learn to prompt is weak career advice ai and human judgement does ai actually make people more productive will ai replace programmers hiring signals when effort is free pricing work when production is cheap independent advisor or consultancy six objections to this work how do we train people to use ai what are the risks of using ai at work

Peer-reviewed#

Noy, S. and Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence #

Science, 381(6654), 187-192, 13 July 2023. DOI 10.1126/science.adh2586. Preregistered, AEA RCT Registry trial 10882

Method: Preregistered online randomised experiment. 453 college-educated professionals, including marketers, grant writers, consultants, data analysts, HR professionals and managers, given occupation-specific incentivised writing tasks of 20 to 30 minutes. Half were randomly exposed to ChatGPT. Run 27 January to 21 February 2023 with GPT-3.5.
Finding: Average time taken fell by 40 per cent and output quality rose by 18 per cent. Time on the post-treatment task dropped by 11 minutes, 0.75 standard deviations, against a control mean of 27 minutes, and evaluator grades rose by 0.45 standard deviations. Inequality between workers decreased: in the treatment group initial inequalities were more than half-erased, with the correlation between first-task and second-task grades falling to 0.14.
What it supports: That the tool compresses the performance distribution on tasks it does well, raising the floor much more than the ceiling. That is a pricing fact about expertise before it is a productivity fact.
What it does not support: That the effect generalises. The authors say they examined a limited range of occupations and tasks in which ChatGPT may be unusually useful, and speculate that real-economy effects will be somewhat lower. Short one-off tasks, online sample, an early model. Nothing about capability retention.

productivity · compression · inequality · writing · expertise · what becomes more valuable as ai gets cheaper does ai actually make people more productive

Working paper#

Peng, S., Kalliamvakou, E., Cihon, P. and Demirer, M. (2023). The Impact of AI on Developer Productivity: Evidence from GitHub Copilot #

arXiv:2302.06590, 13 February 2023. DOI 10.48550/arXiv.2302.06590. Authors at Microsoft Research, GitHub and MIT Sloan. Not peer reviewed

Method: Randomised controlled trial. 95 professional programmers recruited through Upwork from 166 offers, randomised to 45 treated and 50 control, given a standardised task of implementing an HTTP server in JavaScript. 35 in each group completed the task and survey.
Finding: Conditional on completion, the treated group averaged 71.17 minutes against 160.89 for control, a 55.8 per cent reduction in completion time, p = 0.0017, with a 95 per cent confidence interval on the improvement of 21 to 89 per cent. Participants in both groups estimated a 35 per cent productivity increase, which the authors describe as an underestimation of the 55.8 per cent revealed increase.
What it supports: A large measured speed gain on a well-specified task, and, separately, that self-reported productivity can UNDERSTATE the measured effect. Any claim that self-report systematically overstates gains has to answer this result.
What it does not support: Anything about quality. The authors state the study does not examine the effects of AI on code quality. The effect is estimated on 35 completers per arm rather than 95 participants, the task is standardised and greenfield rather than work inside a mature codebase, the authors are employed by the vendor and its parent. Not peer reviewed.

productivity · software development · self-report · perception gap · what becomes more valuable as ai gets cheaper does ai actually make people more productive will ai replace programmers

Peer-reviewed#

Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B. and Yang, Q. (2023). Why Johnny Can't Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts #

CHI 2023

Method: Design probe study with non-experts using a purpose-built prompt design tool.
Finding: Non-experts approached prompting opportunistically rather than systematically, over-generalised from single successes and failures, and struggled to form an accurate model of the system.
What it supports: That prompting is genuinely harder than it looks, which is the strongest case FOR teaching it.
What it does not support: That this persists. It used 2023 models, and providers are actively engineering the difficulty away.

prompt engineering · why learn to prompt is weak career advice

Practitioner method#

Anthropic (Schluntz, E. and Zhang, B.) (2024). Building effective agents #

Anthropic Engineering, published 19 December 2024, date read from the page itself. The page H1 reads 'Building effective agents' and its HTML title reads 'Building Effective AI Agents'; the H1 is cited. The live page now carries a note added after publication saying much of the tooling landscape it describes has changed, and the definitional passage is unaltered.

Method: An engineering practice note setting out implementation patterns and one architectural distinction. No study, no sample, no measurement.
Finding: Draws the line most widely used in practice: 'Workflows are systems where LLMs and tools are orchestrated through predefined code paths', while 'Agents, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks'. Both sit under the heading agentic systems. The post opens by conceding the problem in its own words: 'Agent can be defined in several ways.'
What it supports: That the operationally useful line runs through who chooses the next step, and that a firm building these systems draws it there.
What it does not support: That either pattern performs better, and nothing about any product. It is a taxonomy published by a company that sells the systems being taxonomised, with no data attached. Useful because it is operational and widely adopted, not because it is authoritative.

AI agent · agentic systems · workflow · tool use · what is an ai agent

Peer-reviewed#

Kim, S. S. Y., Liao, Q. V., Vorvoreanu, M., Ballard, S. and Wortman Vaughan, J. (2024). I'm Not Sure, But...: Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust #

Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT 24). Pre-registered at osf.io/mnrp9

Method: Pre-registered between-subjects experiment, four conditions, eight yes-or-no medical questions. 656 responses collected, 252 excluded on pre-registered criteria, final sample 404. Responses pre-generated and presented as a fictional system whose answers were correct on exactly half the questions. Conditions varied only the presence and perspective of uncertainty expression.
Finding: Access to the system raised agreement to 80.9 per cent against 58.4 per cent without, and lowered accuracy to 63.9 per cent against 74.2 per cent without. First-person uncertainty expression significantly reduced agreement to 74.8 per cent and significantly raised accuracy to 72.8 per cent. Impersonal expression moved both in the same direction without reaching significance. Intention to use fell significantly under first-person hedging, 2.91 against 3.25 in control and 3.36 for the impersonal version. Hedging reduced accuracy slightly when the system was correct and raised it more when the system was wrong.
What it supports: That hedging changes reliance in the direction that helps, that the perspective of the hedge matters, and that the version which helps most is the version users least want to keep using.
What it does not support: That uncertainty expression should be mandated. The authors state directly that regulators should avoid blanket requirements until more research is done, and flag that their system had low accuracy and expressed uncertainty often in a poorly calibrated manner. It did not eliminate over-reliance: participants without AI access still performed best. Reported as model-estimated means with significance stars, no test statistics, exact p values or effect sizes, and 38.4 per cent of collected responses were excluded.

uncertainty · over-reliance · trust calibration · interface design · how should an ai agent communicate uncertainty ai and human disagreement

Peer-reviewed#

Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R. and Gal, Y. (2024). AI models collapse when trained on recursively generated data #

Nature, 631, 755-759, published 24 July 2024. DOI 10.1038/s41586-024-07566-y. An Author Correction was published 21 March 2025 (DOI 10.1038/s41586-025-08905-3), fixing a single Greek-letter notation error in the theoretical section; no finding or figure changed. The term Model Collapse was introduced in the authors' own earlier preprint, The Curse of Recursion: Training on Generated Data Makes Models Forget, arXiv:2305.17493, submitted 27 May 2023, read at source.

Method: Controlled experiments training language models, variational autoencoders and Gaussian mixture models recursively on data generated by earlier versions of themselves, across successive generations, plus a theoretical analysis of why the effect must occur.
Finding: Indiscriminate training on model-generated content causes 'irreversible defects' in the resulting models: the tails of the original data distribution, meaning rare or unusual examples, disappear first, and generations converge on an increasingly narrow, generic approximation of the average. The authors demonstrate the effect across three different kinds of generative model, which they take as evidence that it is a general property of learning from recursively generated data rather than a quirk of language models specifically.
What it supports: That a model trained on its own kind's output degrades in a specific and measured direction, losing diversity before losing coherence, and that access to genuine, human-generated data becomes more rather than less valuable as AI-generated content accumulates online.
What it does not support: Anything about human cognition, and nothing about how much of today's open web is already AI-written or how far any real production model has moved towards collapse. It is a technical finding about training pipelines, not a measurement of the internet's current state, and is not to be confused with knowledge collapse, a separate and later argument about human societies rather than about models.

model collapse · training data · synthetic data · homogenisation · what is model collapse what is knowledge collapse

Peer-reviewed#

Vaccaro, M., Almaatouq, A. and Malone, T. (2024). When combinations of humans and AI are useful: a systematic review and meta-analysis #

Nature Human Behaviour, 8, 2293-2303

Method: Preregistered systematic review and meta-analysis: 106 experimental studies, 370 effect sizes, published January 2020 to June 2023.
Finding: Human-AI combinations performed significantly WORSE on average than the better of human or AI alone (Hedges' g = -0.23). Losses concentrated in decision-making; gains in content creation. Pairing gained where humans beat the AI and lost where the AI beat humans.
What it supports: That adding a human is not a control, and that undesigned pairing can subtract. The most under-absorbed result in the field.
What it does not support: That human-AI teams are useless. The benchmark is an oracle-selected best performer, which you rarely know in advance. Also predates current frontier models.

human in the loop · human-AI collaboration · oversight · human ai decision making how should leaders respond to ai ai and human judgement ai and human intuition the cost of delegation pricing work when production is cheap how much can one firm check six objections to this work

Peer-reviewed#

Xiong, M., Hu, Z., Lu, X., Li, Y., Fu, J., He, J. and Hooi, B. (2024). Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs #

ICLR 2024. arXiv:2306.13063

Method: Systematic evaluation of confidence elicitation across five models, Vicuna 13B, GPT-3, GPT-3.5-turbo, GPT-4 and LLaMA 2 70B, on eight datasets spanning arithmetic, commonsense, symbolic and professional reasoning. Metrics: expected calibration error, AUROC and AUPRC. No human subjects.
Finding: Average expected calibration error for plain verbalised confidence is 0.520 for GPT-3, 0.461 for Vicuna, 0.436 for LLaMA 2, 0.377 for GPT-3.5 and 0.180 for GPT-4. GPT-4's average AUROC is 62.7 per cent against a 50 per cent chance baseline. Stated confidences cluster in the 80 to 100 per cent range in multiples of five, which the authors suggest means models may be imitating human expressions when verbalising confidence.
What it supports: That a number a model gives when asked how sure it is is close to unusable as a calibrated quantity, and that its shape suggests it is generated as plausible text rather than measured.
What it does not support: Anything about how users read these numbers, since no humans were involved. Mitigation strategies reduce ECE substantially, to 0.028 in the best case, while still failing to predict incorrect answers on knowledge-heavy tasks. Models are of the GPT-4 and LLaMA 2 generation.

calibration · confidence · hallucination · AI agents · how should an ai agent communicate uncertainty

Peer-reviewed#

Zhou, K., Hwang, J. D., Ren, X. and Sap, M. (2024). Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertainty #

Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), Long Papers, 3623-3643. Also arXiv:2401.06730

Method: Model study: nine models prompted with 49 prompts over 284 MMLU questions, 125,244 queries. Human study: Prolific participants across a control setting and three interactive settings with calibrated, overconfident and underconfident systems, 25 recruited per setting.
Finding: Only about 5 per cent of generated answers include any epistemic marker. Among confidently expressed responses the error rate averages 47 per cent, and only 53 per cent of generations expressing certainty are correct. In the human study, hedged answers were relied on around 10 per cent of the time and confident ones around 90, but plain unmarked statements were also relied on nearly 90 per cent of the time, which the authors read as users interpreting the absence of a marker as certainty. Exposure to an overconfident system left mental models uncorrected, with participants averaging 76 per cent during miscalibrated rounds and 86 afterwards, while an underconfident system produced 66 and then 98. Reward modelling scores plain statements at 4.03, expressions of certainty at 0.82 and expressions of doubt at minus 1.86.
What it supports: That silence about uncertainty is read as confidence rather than as neutrality, and that the confident register is a product of a preference model that penalises hedging more than it rewards assurance.
What it does not support: Any of it with statistical rigour on the human side. No total human N is stated in the text, and no p values, confidence intervals or effect sizes are reported for any human result. Twenty-five participants per setting. US-only, which the authors themselves call a narrow and US-centric view. Body text and Table 1 disagree slightly on the marker rate, 5 against 6 per cent. Peer-reviewed venue corrected to ACL 2024 on 4 September 2026 after an earlier draft of this entry read the arXiv preprint as unpublished.

uncertainty · over-reliance · RLHF · confidence · how should an ai agent communicate uncertainty ai and human disagreement how do we train people to use ai what are the risks of using ai at work

Working paper#

Becker, J., Rush, N., Barnes, E. and Rein, D. (METR) (2025). Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity #

METR, 10 July 2025; arXiv:2507.09089, v2 25 July 2025. No journal reference as at 9 September 2026.

Method: Randomised controlled trial. 16 experienced open-source developers drawn from repositories averaging over 22,000 stars and a million lines of code, 246 real issues from their own projects, each randomly assigned to permit or prohibit AI tools. Tasks averaged about two hours. Screens recorded, implementation time self-reported, developers paid 150 dollars an hour. Tooling was mostly Cursor Pro with Claude 3.5 and 3.7 Sonnet. Twenty candidate explanations for the slowdown were tested and five judged contributory.
Finding: Developers were measured as 19 percent SLOWER when permitted to use AI tools. They had forecast a 24 percent speed-up beforehand, and after completing the tasks and experiencing the slowdown, still estimated AI had made them about 20 percent faster.
What it supports: That self-reported productivity gain is an unreliable measure of actual productivity gain, and that the error can run in the opposite direction to the truth by a wide margin.
What it does not support: That AI slows all developers or all software work, and NOT the current position. Sixteen participants, all experienced, all working on large mature codebases they knew well, using early-2025 tooling. METR themselves withdrew this as a current signal on 24 February 2026: see metr-2026-update. The durable finding is the perception gap, not the 19 per cent.

productivity · perception gap · self-report · software development · what is the metr study the best writing on ai usage theatre what is human ai collaboration ai and human judgement what becomes more valuable as ai gets cheaper the mid career squeeze how do you measure ai adoption properly how will ai change consulting the unclaimed hour what is the substitution myth does ai actually make people more productive will ai replace programmers what are the risks of using ai at work

Peer-reviewed#

Cemri, M., Pan, M. Z., Yang, S., Agrawal, L. A., Chopra, B., Tiwari, R., Keutzer, K., Parameswaran, A., Klein, D., Ramchandran, K., Zaharia, M., Gonzalez, J. E. and Stoica, I. (2025). Why Do Multi-Agent LLM Systems Fail?#

NeurIPS 2025 Datasets and Benchmarks Track. arXiv:2503.13657

Method: Empirical failure taxonomy built from expert annotation of 150 execution traces at inter-annotator kappa 0.88, then applied at scale with LLM-as-judge across more than 1,600 traces from seven multi-agent frameworks. Failure distribution computed on 210 traces.
Finding: Failures divide into system design issues 41.8 per cent, inter-agent misalignment 36.9 per cent and task verification 21.3 per cent. Inter-agent misalignment is defined as a breakdown in critical information flow during interaction and coordination, and comprises conversation resets 2.20 per cent, proceeding on wrong assumptions rather than seeking clarification 6.80 per cent, task derailment 7.40 per cent, withholding information another agent needed 0.85 per cent, ignoring another agent's input 1.90 per cent, and mismatch between reasoning and action 13.2 per cent. The authors state that context and communication protocols are often insufficient, because the errors occur even when agents in the same framework communicate in natural language. Improving role specification alone raised ChatDev success by 9.4 percentage points.
What it supports: That more than a third of observed multi-step agent failure sits at the joins rather than in any single agent's competence, and that standardising the message format does not fix it.
What it does not support: Anything about human-agent boundaries. Every handoff in the dataset is agent to agent and no humans appear anywhere. The authors caveat that the 210-trace distribution illustrates system-specific profiles rather than comparing performance across systems, so the percentages are not cross-benchmark comparable.

AI agents · multi-agent systems · handoffs · coordination failure · how do humans and agents divide work

Peer-reviewed#

Dell'Acqua, F., Ayoubi, C., Lifshitz, H., Sadun, R., Mollick, E., Mollick, L., Han, Y., Goldman, J., Nair, H., Taub, S. and Lakhani, K. (2025). The Cybernetic Teammate: A Field Experiment on Generative AI Reshaping Teamwork and Expertise #

NBER Working Paper 33641, April 2025. Published as 'The Cybernetic Teammate: A Field Experiment on Generative AI and Teamwork', Organization Science, June 2026. DOI 10.1287/orsc.2025.20702

Method: Pre-registered field experiment with 776 professionals at Procter and Gamble working on real product innovation challenges, randomised both on AI access and on working individually or in a two-person new product development team.
Finding: Individuals working with AI matched the performance of two-person teams working without it. AI use removed the functional split in proposals: without AI, research and development professionals proposed more technical solutions and commercial professionals more commercially oriented ones, while professionals using AI produced balanced solutions regardless of background. Participants using AI reported more positive emotional responses.
What it supports: That a model can substitute for measurable parts of what a second human teammate contributes, including some of the social and motivational function, and that it flattens the differences in output that come from professional training.
What it does not support: Client or business outcomes, which were not measured, or any effect over time. One firm, one task type, a single session. Procter and Gamble provided financial support to the institute involved and one author had consulted for the firm, both disclosed in the paper. That the flattened output is better rather than merely more balanced is not established.

human-AI collaboration · expertise · productivity · how will ai change consulting does ai make everyone think alike what is the shared prompt review what is automating versus informating

Working paper#

Kwa, T. and Cheng, V. (METR) (2025). How Does Time Horizon Vary Across Domains?#

METR blog, 14 July 2025. Not peer-reviewed. Read at source 3 October 2026

Method: Time-horizon analysis applied to nine benchmarks across several domains, including software, computer use, mathematics contests, scientific question answering, video understanding and autonomous driving, using human-time estimates that differ by benchmark.
Finding: METR's summary reports generally similar rates of improvement to the seven-month doubling of the original work. The authors state that economically valuable tasks are much more diverse than the domains examined, that the key skills of management and nursing are mostly unrepresented, and that all benchmarks are expected to be easier than real-world tasks.
What it supports: That the rapid-improvement pattern is not confined to software benchmarks, within the benchmarks tested.
What it does not support: Any rate of improvement for unmeasured work. The authors call individual benchmark results fairly noisy, note that many benchmarks lack human baselines so task lengths are estimated, and say some horizons are extrapolations beyond the task lengths available. Per-domain figures were NOT used on any page because the fetched summary could not be checked against the post's own table.

time horizon · benchmark · external validity · will ai get better at this soon

Peer-reviewed#

Kwa, T., West, B., Becker, J., Deng, A., Garcia, K., Hasin, M., Jawhar, S., Kinniment, M., Rush, N., Von Arx, S., Bloom, R., Broadley, T., Du, H., Goodrich, B., Jurkovic, N., Miles, L. H., Nix, S., Lin, T., Painter, C., Parikh, N., Rein, D., Sato, L. J. K., Wijk, H., Ziegler, D. M., Barnes, E. and Chan, L. (METR) (2025). Measuring AI Ability to Complete Long Software Tasks #

NeurIPS 2025; arXiv:2503.14499, v1 18 March 2025, v4 10 July 2026. Venue taken from arXiv's journal-reference field on v4; the proceedings text was not read.

Method: A benchmark metric fitted by logistic regression. 170 tasks in three suites: 97 software tasks from HCAST running 1 minute to 30 hours, 7 machine-learning research engineering tasks from RE-Bench all 8 hours, and 66 software atomic actions of 1 to 30 seconds. Task length is the geometric mean time of SUCCESSFUL human baselines. Over 800 baselines totalling 2,529 hours; 148 of 169 tasks have a human baseline and 21 carry a researcher estimate. Baseliners were paid professionals with about five years of experience, recorded, and bonused for speed. 12 frontier and 4 near-frontier models, 8 runs per model-task pair. The paper is internally inconsistent on its own counts, giving 170 tasks in the abstract and figure captions and 169 in the method, and 12 models in one place and 11 in another.
Finding: The 50%-task-completion time horizon, defined as the human-expert task length a model completes with about 50 per cent success, doubled every 207 days from 2019 to 2025 (95 per cent bootstrapped CI 166-240 days). GPT-2 sits at 2 seconds and o3 at 110 minutes. The 80 per cent horizon doubles on a similar clock at 204 days but is 4 to 6 times SHORTER in level. The authors labelled their own tasks against 16 messiness factors and report a mean of 3.2/16 with none above 8, estimating 9 to 15 for a task like writing a good research paper; each messiness point costs roughly 8.1 percentage points of success. Rate of improvement was the same in the messy and clean halves of the suite.
What it supports: That the length of software task a model will attempt at a coin-flip success rate has grown on an exponential measured across six years, and that the slope survives moving the success threshold from 50 to 80 per cent.
What it does not support: Reliability, and not general capability. The authors state the tasks are 'systematically different from real tasks' because all are automatically scored, none involves other agents, few constrain a scarce resource, few punish a single mistake and environments are static. They say they 'cannot confidently measure time horizons at very high success rates (e.g. 95%)'. Their confidence in the 2024-25 acceleration is 'low because there are only seven frontier models in this time span', and the 2024-only fit gives three months with the note that 'any extrapolation into the future would not be robust'. They are 'more confident in the slope of the time horizon trend than in the time horizon of any particular model', so a single model's horizon is the weaker half of the paper. Their contract baseliners took 5 to 18 times longer than repository maintainers, so horizons correspond 'to the labor of a low-context human'. Their own footnote: the forecasts concern 'a 1-month horizon on software tasks, not 1-month AGI'. NO ABSOLUTE 80 PER CENT HORIZON IS STATED ANYWHERE IN THE PAPER: only the 4-6x ratio and the 204-day doubling. Any minute figure for an 80 per cent horizon is somebody's arithmetic and must not be attributed to the authors. NOT the same study as metr-2025, the developer trial, and the two must never be cited as one result.

time horizon · benchmark · external validity · messiness · capability measurement · what is the metr time horizon the cost of delegation how much can one firm check

Practitioner framework#

Sebastian, I. M., Weill, P., Haskamp, T. and vom Brocke, J. (2025). A framework for determining when AI can make decisions #

MIT Center for Information Systems Research, reported by MIT Sloan

Method: Framework from a research centre, sorting decisions on two axes, ambiguity and risk, into four types with a different human role in each.
Finding: Proposes routine, consequential, exploratory and strategic decisions, with the human role moving from monitoring at one corner to owning the decision at the other.
What it supports: That a two-axis sort of decisions by ambiguity and risk is the current management-side answer to when a machine may decide.
What it does not support: That the four types are exhaustive or that the sort has been validated. It addresses the executive allocating work, not the person doing it, and reports no data.

decision rights · delegation · risk · the use or keep test

Peer-reviewed#

Steyvers, M., Tejeda, H., Kumar, A., Belem, C., Karny, S., Hu, X., Mayer, L. W. and Smyth, P. (2025). What large language models know and what people think they know #

Nature Machine Intelligence, 7, 221-231

Method: Two behavioural experiments, 301 participants recruited through Prolific, each assigned 40 questions from pools of 350 multiple-choice and 336 short-answer items. Model confidence from GPT-3.5, PaLM2 and GPT-4o compared against participant confidence after reading model explanations.
Finding: Model confidence discriminates correct from incorrect answers at AUC 0.751 for GPT-3.5, 0.746 for PaLM2 and 0.781 for GPT-4o. Participants reading default explanations reached 0.589, 0.602 and 0.592, which the authors describe as only slightly better than random guessing. They name the shortfalls the calibration gap and the discrimination gap, and attribute human miscalibration primarily to overconfidence, people believing LLMs are more accurate than they are. Longer explanations significantly raised participant confidence without improving discrimination, mean participant AUC 0.54 for long explanations.
What it supports: That the reader is a worse judge of an answer's correctness than the model is, on the same items, and that adding explanation length makes readers surer rather than righter.
What it does not support: That closing the gap improves task accuracy. The modified-explanation result is a simulation via post-hoc filtering rather than a live deployment. Participants had no domain expertise and their own accuracy was 33 per cent against the model's 39. Statistics are Bayes factors only, with no p values or effect sizes, and the ECE values sit inside a figure rather than in the text.

calibration · confidence · over-reliance · explanation · how should an ai agent communicate uncertainty ai and human disagreement

Institutional survey#

Becker, J. (METR) (2026). Measuring the Self-Reported Impact of Early-2026 AI on Technical Worker Productivity #

METR, 11 May 2026

Method: Survey of 349 technical workers fielded February to April 2026: 87 software engineers, 71 researchers, 129 academics and PhD students, 48 founders and managers. Convenience sample sourced from GitHub, academic directories, METR and staff networks, and X. Email response rate around 2 per cent. Roughly 70 per cent of participants paid, averaging 200 dollars. Respondents averaged 12 years programming, 19 months using AI for programming and 7 months using agentic coding tools; 50 per cent regularly use Claude Code. Ten respondents removed for inconsistent answers. The design separates value produced from speed, on the argument that speed overstates value when AI changes which tasks a person takes on.
Finding: Median self-reported change in the value of work was between 1.4 and 2 times; median self-reported speed change was 3 times. Asked the same question about different years, respondents put themselves at 1.3 times in March 2025, 2 times in March 2026, and forecast 2.5 times for March 2027. METR's own staff gave the lowest value-change answers of any subgroup studied, which the authors suggest reflects those staff having the perception-gap finding in mind. METR restate the size of that gap here: participants in the early-2025 trial overestimated AI's effect on their time by 40 percentage points on average.
What it supports: That the value-versus-speed distinction is large and measurable in self-report, with speed running roughly double value on the same respondents. It is also the clearest statement by the authors of the 19 per cent trial of how big the perception error was.
What it does not support: Any actual productivity effect. Every number here is self-reported, the sample is a convenience sample with a 2 per cent response rate and heavy selection towards enthusiastic adopters, and METR say so. The staff subgroup result is an observation on a small non-random group and not a designed test of whether knowing about the perception gap changes an answer.

productivity · self-report · perception gap · value versus speed · what is the metr study usage theatre how do you measure ai adoption properly the unclaimed hour the cost of delegation

Working paper#

Becker, J. (METR) (2026). Measuring the Self-Reported Impact of Early-2026 AI on Technical Worker Productivity #

METR, 11 May 2026. Not peer reviewed

Method: Survey of 349 technical workers on self-reported change in speed and in value of work attributable to AI tools, set against the published field-experiment literature.
Finding: Median self-reported change in the VALUE of work is 1.4 to 2 times, and median self-reported SPEED change is 3 times. METR state that to their knowledge only one study has gathered survey and field experiment results on the same population and metric, Becker et al. (2025), which finds developers overestimate productivity gains by over 40 percentage points. They add that public survey estimates have tended to exceed field-experiment estimates, while stating it is difficult to determine the extent to which surveys overestimate gains relative to experimental data, and that speed measures are likely biased upwards relative to value measures.
What it supports: That the evidence base for the most repeated claim in enterprise AI, that surveys overstate productivity, rests on a single population, and that the people who own that finding say so themselves. It also separates speed from value, which almost no business case does.
What it does not support: That self-report always overstates. The GitHub Copilot randomised trial found the opposite direction, with participants estimating 35 per cent against a measured 55.8 per cent. A survey of a self-selected technical population, not peer reviewed, and its own authors decline the general claim.

self-report · productivity · measurement · perception gap · what becomes more valuable as ai gets cheaper does ai actually make people more productive

Working paper#

Becker, J., Rush, N., Cunningham, T., Rein, D. and Mahamud, K. (METR) (2026). We are Changing our Developer Productivity Experiment Design #

METR, 24 February 2026

Method: Second randomised task-level study, begun August 2025: 57 developers (10 returning from the original study, 47 newly recruited), 143 repositories, more than 800 tasks, paid 50 dollars an hour against 150 in the original. Accompanied by participant surveys and interviews.
Finding: METR state the data gives an unreliable signal of the current productivity effect of AI tools, because 30 to 50 per cent of developers reported declining to submit tasks they did not want to do without AI, and an increased share declined to take part at all. Raw results now point the other way: an estimated speedup of -18 per cent for returning developers (CI -38 to +9) and -4 per cent for new recruits (CI -15 to +9), against the original +19 per cent slowdown (CI +2 to +39). They believe developers are likely more sped up in early 2026 than in early 2025, while stating their own data is only very weak evidence for the size of that change.
What it supports: That the 19 per cent slowdown belongs to early 2025 and should not be quoted as the current effect. It also demonstrates a measurement problem that will worsen: as adoption rises, the people most helped by AI are the ones most likely to select themselves out of any study that asks them to work without it.
What it does not support: That AI now speeds developers up by a specific amount. Every confidence interval here crosses zero, and METR say so. It does not retract the original study, whose perception-gap finding, that participants estimated a 20 per cent speed-up while measured slower, is untouched.

productivity · self-report · software development · selection effects · what is the metr study usage theatre the best writing on ai the ai reports worth reading ai and human judgement what becomes more valuable as ai gets cheaper the mid career squeeze how do you measure ai adoption properly how will ai change consulting the unclaimed hour what is the substitution myth does ai actually make people more productive will ai replace programmers

Working paper#

Li, Z., Ye, S., Guo, F. and Dang, Z. (2026). SilentProbe: Measuring Silent Failure in Production APIs Used as Agent Tools #

arXiv 2609.00035, 29 August 2026, 12 pages, CC BY 4.0. Read at source 17 September 2026

Method: Three studies. An audit of 721,320 parameters across 2,501 independently published OpenAPI documents. 219 schema-derived perturbations executed against live commercial endpoints from 27 vendors, all reached through one aggregation layer. Then twelve models across eight families run on ordinary tasks against those endpoints, and the full agent loop run on the resulting failures. Code, schemas, perturbation sets, agent transcripts and per-call run identifiers are released. Preprint, not peer reviewed, and the endpoints are reached through a single intermediary rather than called directly.
Finding: An agent calling a production API cannot tell a query that matched nothing from a query the server did not understand: both return HTTP 200 with a parsable body. Of the documents audited, 7.5 per cent declare an enumeration and 15.2 per cent declare any machine-checkable constraint at all, while 40.1 per cent state at least one constraint in prose that their schema does not encode. Constraint form rather than vendor identity predicted honesty: machine-checkable constraints produced an honest error in 111 of 111 cases and prose-only constraints failed silently in 44 of 61 (p = 2e-13). A vocabulary that the description gave only examples of was missed by every model on 88 of 88 attempts; written out in full it was used correctly 88 to 91 per cent of the time. Running the full loop on the resulting silent failure, models detected it in 12 per cent of cases, repaired it in 0 per cent, asserted a false negative to the user in 41 per cent and invented a figure in 12 per cent. Promoting the vocabulary into the schema took the failure from 88 of 88 to 0 of 89.
What it supports: That when a tool fails without saying so, current models mostly do not notice, never repair it, and in a majority of cases tell the user something untrue: either that there is nothing there, or a number they made up. It also shows the fault is in how the interface declares its constraints rather than in model quality, because one schema change removed it.
What it does not support: Anything about a human's ability to catch the same failure, which is not tested. The percentages are conditional on this perturbation set, these 27 vendors and one aggregation layer, and the twelve models are not named in the abstract. It is a preprint with no independent replication. The 41 and 12 per cent describe what a model said, not what a person then believed or acted on.

silent failure · AI agents · confabulation · verification · tool use · what is silent failure the invisible work of oversight

Working paper#

METR (2026). Time Horizon 1.1 #

METR blog, 29 January 2026. Not peer-reviewed. Read at source 3 October 2026

Method: Re-estimate of METR's time-horizon measure on an expanded suite of 228 tasks, up from 170 (73 added, 15 removed, 53 updated), with tasks of eight hours or more rising from 14 to 31, on new evaluation infrastructure.
Finding: Doubling time for the full 2019 to 2025 period is 196 days, the same as the first version. Since 2023 it is 131 days against 165, and since 2024 89 days against 109. 50 per cent time horizons: Claude Opus 4.5 320 minutes (interval 170 to 729), GPT-5 214 (117 to 480), o3 121 (74 to 201).
What it supports: That on this suite, estimated by its builders, the measured horizon has lengthened with a doubling time of months, and that the recent trend depends on how the suite is composed.
What it does not support: Capability on tasks outside the suite, or reliability. METR writes that the trend is somewhat sensitive to task composition, that the confidence intervals are still very wide, and that human baseline times exist for only 5 of the 31 long tasks. Software tasks only.

time horizon · benchmark · capability measurement · will ai get better at this soon the cost of delegation how much can one firm check

Vendor research#

Massenkoff, M., Lyubich, E., McCrory, P., Appel, R. and Heller, R. (2026). Anthropic Economic Index report: Learning curves #

Anthropic, 24 March 2026. Read at source 2 October 2026

Method: Descriptive and regression analysis of a sample of one million conversations from Claude.ai and Anthropic's first-party API, 5 to 12 February 2026, comparing users who signed up at least six months before the data pull (high tenure) with newer users. Success is Claude's own assessment of whether a conversation was successful.
Finding: High-tenure users are more likely to iterate on their work and much less likely to delegate through directive use patterns, and are 7 percentage points more likely to use Claude for work. The headline reads a 10% higher success rate; the regression table gives about 5 percentage points bivariately, about 3 with O*NET task and request-cluster fixed effects, and about 4 with full controls for model, language, use case and country.
What it supports: That in Anthropic's own logs, longer-serving users work more interactively and score slightly higher on the system's own success measure, after controlling for task and country.
What it does not support: That practice caused either. The authors state that high-tenure users are self-selected, that the differences could reflect stable characteristics, and that survivorship bias is present because people who stopped using the product are missing. Success is judged by the model being used, not by an outside check of the work. Tenure is time since sign-up, not expertise in any domain. The usage data and the success measure are the vendor's and cannot be checked from outside.

human-AI collaboration · learning by doing · expertise · how do expert ai users work differently

Vendor research#

Microsoft (WorkLab), with Edelman Data x Intelligence and LinkedIn (2026). 2026 Work Trend Index Annual Report: Agents, human agency, and the opportunity for every organization #

Microsoft, May 2026

Method: Edelman survey of 20,000 full-time knowledge workers who use AI at work in ten countries (Australia, Brazil, France, Germany, India, Italy, Japan, Netherlands, UK, US), fielded 18 February to 7 April 2026. Combined with analysis of over 100,000 anonymised Microsoft 365 Copilot chats (105,000 samples from one week in February 2026, trend data March 2025 to March 2026), a July 2025 Microsoft People Science survey of 1,800 employees, and LinkedIn labour market data.
Finding: Organisational factors (culture, manager support, talent practices) account for 67% of the variance in AI's reported impact, against 32% for individual mindset and behaviour. 49% of analysed Copilot conversations support cognitive work: analysing information, solving problems, evaluating and thinking creatively. The report describes four modes of working with AI (delegation, collaboration, asking, exploration) by human and agent intensity. Agent adoption is broad in software and technology, which accounts for nearly one in five firms using agents, while manufacturing has fewer adopting firms deploying at greater scale. The Transformation Paradox: employees are ready to change how they work but metrics, incentives and norms reinforce the old way.
What it supports: That among AI-using knowledge workers, organisational conditions explain more of reported impact than individual attitude, and that most Copilot use is cognitive rather than clerical.
What it does not support: The sample is restricted to people already using AI, so it cannot speak to non-adopters. Impact is self-reported and the telemetry is Microsoft's own product. Microsoft sells Copilot and agents.

knowledge workers · organisational factors · ai agents · copilot telemetry · human agency

Argued perspective#

Romasanta, A., Thomas, L. D. W. and Levina, N. (2026). Researchers asked LLMs for strategic advice. They got trendslop in return #

Harvard Business Review, 16 March 2026. Article read at source 26 September 2026; the underlying study was not accessible and the article page discloses no sample, no measures and no figures

Method: An article by three business school researchers reporting their own testing of leading language models on strategic trade-offs. The method is described only in general terms on the page read: models were asked for strategic advice across classic trade-offs and their recommendations compared.
Finding: Reports that leading models consistently recommend strategies that match current managerial trends and vocabulary rather than with the logic of the situation put to them, a pattern the authors name trendslop. Their recommendation is to use these systems to expand options rather than to make choices.
What it supports: That researchers at three business schools have named and published a convergence pattern in model-generated strategic advice, and that the recommendation from inside that finding is to treat model output as option generation rather than as selection.
What it does not support: Any rate or effect size. No sample, no measures and no figures were available on the page read, so nothing here supports a quantitative claim, and the underlying study has not been examined.

strategy · homogenisation · big picture thinking · does ai make strategy generic big picture thinking in the age of ai

Working paper#

Tomasev, N., Franklin, M. and Osindero, S. (2026). Intelligent AI Delegation #

Google DeepMind. arXiv:2602.11865, submitted 12 February 2026. Preprint, not peer-reviewed

Method: A framework paper proposing how AI agents should decompose problems and delegate across other agents and people, covering task assignment, monitoring, trust, permissions and accountability in long delegation chains. No experiment, no data. The capability material is section 5.6, Risk of De-skilling.
Finding: The authors name oversight readiness as a property a future workforce may lack, arguing that expertise is built through the repetitive execution of narrowly scoped tasks, that those are the tasks most likely to be delegated to agents first, and that fully automating them would deprive junior staff of the experience needed for strategic judgement. Their proposed remedies are unusual for a frontier lab: curriculum-aware task routing that allocates work inside a junior's zone of proximal development, and a delegation framework that should occasionally introduce minor inefficiencies by routing tasks to humans deliberately, to maintain their skills.
What it supports: That the missing rungs argument is being reached independently by people building delegation infrastructure rather than only by those writing about its consequences, and that at least one frontier lab has proposed deliberate inefficiency as a capability-preservation mechanism.
What it does not support: Anything measured. It is a preprint framework paper with no empirical component, so it establishes that the risk is taken seriously by practitioners and not that the risk has been observed. The proposed routing systems also assume an organisation can assess what a junior can currently do, which is the unsolved part.

oversight readiness · the missing rungs · synthetic seniority · delegation · what is oversight readiness what is a frontier firm who manages ai agents where your ai practice lives

Working paper#

Welsch, R. (2026). When the AI Leaves the Tailorshop: Measuring What an LLM Advisor Leaves Behind in Complex Problem Solving #

arXiv 2610.00163, submitted 14 September 2026. Abstract read at source 9 October 2026

Method: Two preregistered experiments (N = 200 and N = 198) in which participants managed a simulated clothing factory with or without an LLM adviser, followed by a phase without it.
Finding: 'Across studies, AI-supported participants reported greater confidence and understanding with less effort.' In the first study assistance raised company value with no detectable difference in prediction accuracy, and 'Within the AI-supported group, more frequent recommendation alterations predicted better unaided performance.' In the second, 'More frequent recommendation alterations predicted higher knowledge within the AI-supported group.'
What it supports: That in a controlled task people felt they understood more with AI than the measures showed, and that those who changed the adviser's recommendations more often did better or knew more afterwards.
What it does not support: Cause: the link with altering recommendations is an association inside the supported group. The results are mixed, with a small knowledge advantage for supported participants in the second study. A simulation by one author, read as an abstract, not peer-reviewed.

unaided performance · over-reliance · human-AI collaboration · illusion of competence · does ai make juniors better or only look better

Empathy, creativity and what stays human

Where machine performance already exceeds ours, and what that does not settle.

Clinical observation#

Clance, P. R. and Imes, S. A. (1978). The imposter phenomenon in high achieving women: Dynamics and therapeutic intervention #

Psychotherapy: Theory, Research & Practice, 15(3), 241-247. DOI 10.1037/h0086006. Full text read at source 19 September 2026. Note the spelling: the article's own title and running head use 'imposter', while the body text and abstract use 'impostor'.

Method: Five years of the authors' own clinical and teaching practice at Georgia State University: individual psychotherapy, theme-centred interactional groups and college classes with more than 150 high-achieving women. The sample as the authors give it is 95 undergraduates and 10 PhD faculty at a small midwestern college; 15 undergraduates, 20 graduate students and 10 faculty at a large southern university; six medical students; and 22 professional women. Primarily white, middle to upper class, aged 20 to 45. About a third were therapy clients. No control group, no instrument, no measurement.
Finding: The authors name an internal experience of intellectual phoniness in women whose accomplishments plainly contradict it, and report that achievement does not touch it: 'Numerous achievements, which one might expect to provide ample objective evidence of superior intellectual functioning, do not appear to affect the impostor belief.' They state directly that 'We have not found repeated successes alone sufficient to break the cycle.' Four maintaining behaviours are described: overwork, intellectual inauthenticity, the cultivation of a mentor's approval, and the avoidance of being seen as a successful woman. The therapy proposed is group disclosure and Gestalt work, not the accumulation of evidence.
What it supports: The origin and the exact content of the term, which is cited constantly and read rarely. It is the primary document for what impostor phenomenon meant when it was named, and the authoritative refusal of the folk belief that evidence of competence cures it.
What it does not support: Any rate, in any population. There is no sampling frame, the sample is the authors' own caseload at two institutions, and a third of it arrived already in therapy, so nothing here estimates prevalence. It says nothing about men beyond a footnote recording the authors' clinical impression that the phenomenon is rarer and milder in men, which they themselves flag as needing research and which later work has not supported. Everything downstream, including the 1985 Clance scale, is a different source.

impostor phenomenon · the new imposter syndrome · self-assessment · the illusion of competence · what is the new imposter syndrome

Argued perspective#

Szmyd, J. (2014). Anthropological Regression in the Modern World Versus Anna-Teresa Tymieniecka's Metaphysics of Ontopoiesis of Life #

In Tymieniecka, A.-T. (ed.), Phenomenology of Space and Time, Analecta Husserliana, vol. 116, Springer, 2014. DOI 10.1007/978-3-319-02015-0_9. Read as described on the publisher's page on 4 October 2026; the chapter itself was not read and the review status of the volume was not checked

Method: A philosophical chapter in an edited volume. It reports no data. GRADED FROM THE PUBLISHER'S DESCRIPTION ONLY.
Finding: Uses the phrase anthropological regression for the harmful psychological and spiritual effects the author attributes to modern technical civilisation and neoliberal economics: emotional impoverishment, instrumental human relations, consumerism in place of authentic being, and confusion between virtual and real experience.
What it supports: That the phrase was in published use, in a different sense from the 2026 encyclical's, twelve years earlier. It is evidence about the phrase and not about the world.
What it does not support: Anything empirical. It measures nothing, and it says nothing about AI or about employment, so it cannot support a claim that either causes regression of any kind.

anthropological regression · what is anthropological regression

Peer-reviewed#

Ayers, J. W. et al. (2023). Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum #

JAMA Internal Medicine, 183(6), 589-596

Method: Cross-sectional study, 195 real patient questions from a public forum, blind-rated by licensed healthcare professionals.
Finding: Chatbot responses were rated good or very good quality 78.5 percent of the time against 22.1 percent for physicians, and empathetic or very empathetic 45.1 percent against 4.6 percent.
What it supports: That on the observable, textual performance of empathy, the machine already wins comfortably. The claim that empathy is safe from AI is empirically wrong.
What it does not support: That a machine can care for anyone. Doctors answering strangers free of charge between patients are not doing the job they trained for.

empathy · what stays human · what stays human outsourced recognition

Peer-reviewed#

Hohenstein, J., Kizilcec, R. F., DiFranzo, D., Aghajari, Z., Mieczkowski, H., Levy, K., Naaman, M., Hancock, J. and Jung, M. F. (2023). Artificial intelligence in communication impacts language and social relationships #

Scientific Reports, 13, 5487. DOI 10.1038/s41598-023-30938-9

Method: Two randomised experiments on algorithmic response suggestions (smart replies) in live text chat. Study 1: 438 Mechanical Turk crowdworkers in 219 pairs discussing a policy question, smart-reply availability randomised separately for each partner, analysed with an instrumental-variable design. Study 2: 582 crowdworkers in 291 pairs, with the sentiment of the suggested replies manipulated. Preregistered (AsPredicted #40389).
Finding: Smart replies accounted for 14.3 per cent of messages and produced 10.2 per cent more messages per minute. ON LANGUAGE: greater use of smart replies by a partner led the other person to send messages with more positive sentiment (IV estimate b=0.178, t(205)=2.02, p=0.045), and the effect held when smart-reply messages were EXCLUDED from the sentiment score (b=0.208, t(205)=2.17, p=0.031), so it appears in sentences the person composed themselves. Merely having suggestions available without using them did not move sentiment (b=0.019, p=0.1801). Study 2 manipulated the emotional tone of the suggestions and conversation sentiment followed it. ON PERCEPTION: greater ACTUAL use by a partner improved the other person's rating of their cooperation (b=15.66, p=0.018) and felt affiliation towards them (b=21.79, p=0.007), with no effect on dominance. Greater SUSPECTED use had the opposite sign: the more a participant believed their partner used smart replies, the less cooperative (p<0.0001) and less affiliative (p<0.0001) they rated them, after controlling for actual use. Suspicion tracked actual use only weakly (Pearson's r=0.22).
What it supports: Two separate things. On language, that an algorithmic suggestion system changes what a person writes in their own words, under randomisation, in the direction of the system's own tone. That is the mechanism behind homogenisation observed at the level of the individual, and the null result for mere availability locates it in the act of adopting the phrasing rather than in exposure to it. On perception, that the interpersonal penalty for AI-assisted messaging attaches to being suspected rather than to using it, and that suspicion is a poor detector; actual use made people seem warmer, not colder.
What it does not support: Anything longitudinal, which the authors state directly. The suspicion result is correlational and they say it does not show causally how attitudes shift in response to actual use. Participants were crowdworkers discussing policy with strangers over a few minutes, not intimates and not colleagues. The language effect is measured as sentiment, which is one dimension of style and not the range of what a person might have said, so it evidences convergence in tone and not convergence in thought. The system studied is a smart-reply suggester with short canned options, not a generative model writing paragraphs. A publisher correction (10.1038/s41598-023-43601-0, 3 October 2023) added an omitted funding statement and changed no result.

AI-mediated communication · authenticity · disclosure · homogenisation · voice · should i use ai to write personal messages outsourced recognition does ai make everyone think alike what is the human signal what is the shared prompt review is it still my idea if ai helped me write it

Peer-reviewed#

Jakesch, M., Bhat, A., Buschek, D., Zalmanson, L. and Naaman, M. (2023). Co-Writing with Opinionated Language Models Affects Users' Views #

CHI 2023, ACM

Method: Online experiment with 1,506 participants writing a post on whether social media is good for society, assisted by a tool configured to argue for or against. Opinions rated by 500 independent judges, plus a post-task attitude survey.
Finding: The opinionated model changed both the opinions expressed in participants' writing and their own opinions in the subsequent attitude survey. The effect held among participants who had ample time to write independently. The authors call it latent persuasion.
What it supports: That writing assistance moves what people think, not only what they type.
What it does not support: Generalisation across topics, or persistence after the task. One topic, one configuration, self-reported attitudes.

· how do i keep my own voice when using ai is ai dangerous is it still my idea if ai helped me write it what is the new imposter syndrome

Peer-reviewed#

Anderson, B. R., Shah, J. H. and Kreminski, M. (2024). Homogenization Effects of Large Language Models on Human Creative Ideation #

Creativity and Cognition (C&C '24), ACM, DOI 10.1145/3635636.3656204; arXiv 2402.01536. Abstract and arXiv record read at source 1 October 2026

Method: A 36-participant comparative user study in which people generated ideas with ChatGPT or with an alternative creativity support tool, with semantic distinctness of ideas compared across users.
Finding: Different users produced less semantically distinct ideas with ChatGPT than with the alternative tool. ChatGPT users generated a greater number of more detailed ideas but felt less responsible for the ideas they generated.
What it supports: That in a small controlled comparison an LLM pushed different people's ideas closer together while raising the volume and detail of each person's output, and that ownership of the ideas was weaker.
What it does not support: Anything about a larger or more varied sample, professional creatives, or tasks other than ideation. With 36 participants the effect sizes are imprecise. It does not show the narrowing persists once people stop using the tool.

homogenisation · creativity · ownership · is ai bad for creativity

Peer-reviewed#

Doshi, A. R. and Hauser, O. P. (2024). Generative AI enhances individual creativity but reduces the collective diversity of novel content #

Science Advances, 10(28)

Method: Online experiment in which roughly 300 writers produced short stories, some with ideas supplied by a language model, and 600 evaluators rated them. The 300 and 600 are the authors' institutional figures; the paper's own methods section returned a 403 to the fetcher on 1 October 2026, and this entry previously said 293 writers, which could not be confirmed.
Finding: AI-assisted stories were rated more creative, better written and more enjoyable, with the largest gains for the least creative writers, and were markedly more similar to one another.
What it supports: That individual creative quality and collective creative range move in opposite directions. A social dilemma: every writer is right to use it, and the literature gets duller.
What it does not support: That this generalises beyond one short creative task with one form of assistance.

creativity · homogenisation · what stays human what is the human signal what is the shared prompt review is ai bad for creativity

Peer-reviewed#

Draxler, F., Werner, A., Lehmann, F., Hoppe, M., Schmidt, A., Buschek, D. and Welsch, R. (2024). The AI Ghostwriter Effect: When Users do not Perceive Ownership of AI-Generated Text but Self-Declare as Authors #

ACM Transactions on Computer-Human Interaction, 31(2), 1-40, published 5 February 2024. DOI 10.1145/3637875. Preprint arXiv:2303.03283, first version 6 March 2023, second 7 November 2023. Venue confirmed at Crossref 19 September 2026: the arXiv page still carries the pre-publication note 'currently under review', which understates it.

Method: Two experiments, 126 participants in total. Study 1, n=30, compared personalisation and four interaction methods for producing a personalised text, then asked who should be credited when the text was uploaded. Study 2, n=96, replicated the effect in a larger sample and added a comparison condition in which participants believed a human ghostwriter had written the text. Ownership measured on psychological-ownership scales; authorship measured by what participants actually entered in a credit field.
Finding: Participants did not regard themselves as the owners or authors of text a model had produced for them, and mostly did not name the model when publishing it either. In Study 1 about 20 per cent mentioned AI in the free-text credit field. In Study 2, with a multiple-choice list rather than free text, fewer than one in five declared the AI as an author. Ownership rose with the participant's own influence over the text. Ownership was attributed more readily to a supposed human ghostwriter than to the AI, producing a wider ownership-authorship gap for the human.
What it supports: That the gap between who a person privately feels wrote something and whom they publicly credit is measurable, and that it opens without anybody intending to deceive: the same participants who withheld the credit gave transparency and ethics as reasons AI should be credited. It is the closest thing in the literature to a measurement of unattributable authorship, and the only one this research holds.
What it does not support: Anything about not being able to reconstruct where an idea came from. The experiment measures attribution of a finished text the participant watched a machine produce, which is a different question from the one raised by an argument that fused over weeks. It is also short, artificial, a postcard and a blog upload rather than professional work, and 126 people in total.

authorship · ownership · the new imposter syndrome · attribution · disclosure · what is the new imposter syndrome is it still my idea if ai helped me write it

Peer-reviewed#

Fleisig, E., Smith, G., Bossi, M., Rustagi, I., Yin, X. and Klein, D. (2024). Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discrimination #

Proceedings of EMNLP 2024, 13541-13564. University of California, Berkeley. Preprint arXiv:2406.08818

Method: Ten varieties of English: Standard American and Standard British, plus African American, Indian, Irish, Jamaican, Kenyan, Nigerian, Scottish and Singaporean. Around fifty native-speaker messages per variety on everyday topics, put to GPT-3.5 Turbo and GPT-4 in a plain condition and an imitate-the-style condition. Annotated for ten linguistic features per variety at Krippendorff's alpha 0.97, then evaluated by native speakers recruited through Prolific, at least eleven per variety.
Finding: Responses to non-standard varieties carried more stereotyping (19 per cent worse), more demeaning content (25 per cent worse), less comprehension (9 per cent worse) and more condescension (15 per cent worse), all significant after correction. The more striking result is retention: a Standard American English input keeps 77.9 per cent of its distinctive features in the reply and Standard British 72.2 per cent, while five of the eight minoritised varieties keep only 2 to 3 per cent. Indian, Nigerian and Kenyan English sit between the two groups at 10 to 16 per cent, and the authors report that retention rate tracks the estimated maximum speaker population of a variety, which they take as a proxy for training data volume. Asked to imitate, GPT-4 improved comprehension and warmth while stereotyping rose 18 per cent.
What it supports: That homogenisation towards one dialect is measured rather than impressionistic, and that it is severe: the model does not merely prefer Standard American English, it strips almost every marker of the others. The clearest evidence the research holds that convergence on a single voice has a distributional cost, borne by particular speakers.
What it does not support: That models exaggerate or caricature dialects, which is the claim usually attached to this paper in secondary coverage. The measured default is the opposite, features are stripped rather than amplified, and the exaggeration line rests on one annotator's free-text comment about Singlish. It also says nothing about human speech: this is model output, not people.

homogenisation · standardisation · the human signal · voice · does ai make everyone think alike what stays human

Working paper#

Yakura, H., Lopez-Lopez, E., Brinkmann, L., de la Serna, I., Kirfel, L., Gupta, P., Soraperra, I., Eisenmann, T. F., Wulff, D. U. and Rahwan, I. (2024). Empirical evidence of Large Language Model's influence on human spoken communication #

arXiv:2409.01754. Max Planck Institute for Human Development. v1 submitted 3 September 2024, v2 30 June 2025, v3 8 July 2025, v4 16 July 2026. STILL NOT PEER-REVIEWED after four versions: the arXiv record carries no journal reference and no DOI other than the preprint server's own, checked 7 September 2026. Substantially replaced by the authors: v1's YouTube analysis is gone, and the six-author list on the version most secondary coverage cites is now ten

Method: TWO DIFFERENT STUDIES UNDER ONE IDENTIFIER, and the distinction is the whole entry. v1 (2024): 279,480 YouTube transcripts from academic-institution channels, filtered from nearly three million videos, spanning 36 months before and 18 months after the release of ChatGPT, with a hierarchical Bayesian regression on the monthly log frequency of videos containing a given word and a change point at the release date. v4 (2026): 737,083 hours of conversation from 824,634 podcast episodes screened for unscripted speech, analysed by synthetic control, in which each treated word's monthly relative document frequency is compared against a counterfactual built from donor words with near-zero GPT scores and matched pre-release trajectories; plus a preregistered experiment with 496 participants who played a referential image-guessing game with a chatbot covertly prompted to use particular synonyms, then described the target image aloud after a three-minute arithmetic distractor.
Finding: v1 reported that words the model favours rose over the 18 months after release: adept by 51 per cent, delve 48, meticulous 40, realm 35, with the word list taken from Liang and colleagues at Stanford, who compared 10,000 human-written abstracts with their ChatGPT-edited versions. v4 reports the same direction on a far better design and does not report those percentages: it finds an abrupt rise in GPT-preferred words in spontaneous speech, attributes it to the release by synthetic control, with a placebo test at p = 0.01 for the most-cited of those words, and finds experimentally that a brief chatbot interaction led participants to adopt its words as their own, persisting past a distractor task and confirmed in forced lexical choice.
What it supports: That the vocabulary of the machine appears in human material at scale and on a timeline consistent with the model's arrival, and, on the current version, that a short exposure is sufficient to move an individual's active vocabulary under randomisation. The strongest quantified evidence the research has for a pattern everybody claims to notice.
What it does not support: THE 51 PER CENT SHOULD NOT BE QUOTED AT ALL. It is adept alone, not a general figure; its outcome is the PREVALENCE OF VIDEOS containing a word, not how often speakers use it; its population is academic institutional channels rather than speakers at large; and when the authors hand-checked fifty videos containing the most-cited of those words, 32 per cent showed signs of the speaker reading from a script. Decisively, neither the word adept nor the figure 51 appears anywhere in v4: the authors have withdrawn the analysis it came from. The current version has its own limits, stated by the authors: the corpus is English-only and drawn from a self-selected, public-facing population of podcast hosts and guests; the main analysis covers the first 18 months and treats OpenAI's models as the dominant driver, while deployment has since fragmented across providers, making attribution harder; residual interference in the synthetic control cannot be ruled out; and the outcome is still the share of episodes containing a word rather than a speaker's frequency of use. Cite the direction, never the number.

homogenisation · AI-speak · the human signal · does ai make everyone think alike

Peer-reviewed#

Yin, Y., Jia, N. and Wakslak, C. J. (2024). AI can help people feel heard, but an AI label diminishes this impact #

PNAS, 121(14), e2319112121

Method: Experiments comparing AI-generated and human-written responses, with and without disclosure.
Finding: AI-generated replies made recipients feel MORE heard than replies from untrained humans, and labelling the reply as AI removed the advantage.
What it supports: That the value of recognition is not in the words but in the belief that a person chose to attend to you. The most clarifying study in this debate.
What it does not support: That the label effect is stable. Norms around disclosed AI assistance are moving, and nobody has measured this over time.

recognition · outsourced recognition · empathy · outsourced recognition what stays human

Institutional survey#

Common Sense Media (2025). Talk, Trust, and Trade-offs: How and Why Teens Use AI Companions #

Common Sense Media, San Francisco, 16 July 2025

Method: Nationally representative survey of 1,060 US teens aged 13 to 17, April and May 2025. Risk items were asked of the 758 respondents who had used an AI companion.
Finding: 72 per cent of teens had used an AI companion at least once and 52 per cent were regular users, a few times a month or more. Among users, 33 per cent had chosen an AI companion over a real person for something important or serious, 34 per cent had felt uncomfortable with something a companion said or did, and 24 per cent had shared personal or private information. The same survey found 67 per cent of teens rating AI conversations as less satisfying than conversations with real friends against 10 per cent more satisfying, 80 per cent of users spending more time with friends than with companions, and 50 per cent not trusting the advice.
What it supports: That AI companion use among US teenagers is majority behaviour rather than a fringe one, and that a substantial minority of users have taken serious conversations and personal information to them.
What it does not support: Anything about developmental effects, which are not measured here. Self-report at a single point in time, by an organisation that backs legislation to ban these products for minors, and the risk percentages are of users rather than of all teens: 33 per cent of users choosing a companion for a serious conversation is roughly 24 per cent of teens. The press release framing of a generation replacing human connection is not supported by the survey's own Q5 and Q7.

AI companions · young people

Working paper#

De Freitas, J., Oguz-Uguralp, Z. and Kaan-Uguralp, A. (2025). Emotional Manipulation by AI Companions #

arXiv 2508.19258, submitted 15 August 2025, last revised 7 October 2025 (v3). No journal publication found on 5 October 2026

Method: Behavioural audit of 1,200 real farewells across the most-downloaded companion apps, plus four preregistered experiments with 3,300 nationally representative US adults. Abstract read at arxiv.org on 5 October 2026; full text not read.
Finding: Per the abstract, one of six recurring tactics, including guilt appeals and fear-of-missing-out hooks, appeared in 37 per cent of the farewells. Manipulative farewells boosted post-goodbye engagement by up to 14 times in controlled chats, through reactance-based anger and curiosity rather than enjoyment, and also raised perceived manipulation, churn intent, negative word-of-mouth and perceived legal liability.
What it supports: That a common companion-app design pattern exists, that it raises engagement in controlled chats, and that users register it as manipulative.
What it does not support: Any effect on loneliness or well-being. It is written for marketers and measures engagement and attitudes. The 37 per cent applies to the apps and farewells sampled, and the preprint has not been through peer review at the time of entry.

AI companions · dark patterns · engagement · is it bad to talk to ai when you are lonely

Working paper#

Fang, C. M., Liu, A. R., Danry, V., Lee, E., Chan, S. W. T., Pataranutaporn, P., Maes, P., Phang, J., Lampe, M., Ahmad, L. and Agarwal, S. (2025). How AI and Human Behaviors Shape Psychosocial Effects of Extended Chatbot Use: A Longitudinal Randomized Controlled Study #

arXiv 2503.17473, submitted 21 March 2025, revised (v2). No journal publication found on 5 October 2026

Method: Four-week randomised controlled experiment, 981 completers of 2,539 enrolled, recruited through CloudResearch in the United States, mean age 39.9, English-fluent. Participants were assigned an interaction mode (text, neutral voice, engaging voice) and a conversation type (open-ended, non-personal, personal) with ChatGPT. Outcomes were loneliness, socialisation with real people, emotional dependence on AI and problematic use. More than 300,000 messages. Abstract and full text read at arxiv.org on 5 October 2026.
Finding: The authors report no significant effects of the assigned conditions, including no significant effect of modality or task on loneliness or socialisation. Voluntary use varied, and participants who spent more time with the chatbot each day showed higher loneliness and more emotional dependence on it; higher trust and social attraction towards the chatbot went with more dependence and problematic use.
What it supports: That in this design, changing the voice mode or conversation topic did not change psychosocial outcomes over four weeks, and that heavier voluntary use went with worse outcomes.
What it does not support: That chatbot use causes loneliness: nobody was randomised to more or less use, and the study has no no-chatbot control group, which the authors list as a limitation. Results are specific to ChatGPT and to English-speaking US adults who completed four weeks. A companion paper from some of the same authors (Phang et al., arXiv 2504.03888) reports this trial alongside an analysis of more than three million conversations; it is not graded separately here. Not peer-reviewed at the time of entry.

AI companions · loneliness · emotional dependence · is it bad to talk to ai when you are lonely

Peer-reviewed#

Joshi, N. and Vogel, D. (2025). Writing with AI Lowers Psychological Ownership, but Longer Prompts Can Help #

ACM Conversational User Interfaces 2025

Method: Two within-subjects experiments, 31 and 34 participants, writing short stories across conditions from a three-word prompt to writing unaided.
Finding: Psychological ownership rose steadily with prompt length, from a mean of 1.80 with a three-word prompt to 6.29 writing alone. The benefit plateaued once the prompt reached roughly the length of the target text, and no AI-assisted condition reached the ownership of writing unaided.
What it supports: That how much of yourself you put in changes whether the output feels like yours, and by a large margin.
What it does not support: Anything about professional or long-form writing. Short fiction, small samples.

· how do i keep my own voice when using ai what is the human signal is it still my idea if ai helped me write it

Peer-reviewed#

De Freitas, J., Oğuz-Uğuralp, Z., Uğuralp, A. K. and Puntoni, S. (2026). AI Companions Reduce Loneliness #

Journal of Consumer Research, 52(6), 1126-1148. Published online 25 June 2025. DOI 10.1093/jcr/ucaf040

Method: A series of online experiments (the journal abstract says five studies; the earlier arXiv and working-paper versions say six). Participants recruited through online panels including CloudResearch Connect. Designs included a seven-day study with a daily 15-minute chatbot session against a loneliness-reporting control, and a single-session comparison of an empathetic companion, a general assistant and a limited-function chatbot. Loneliness was measured before and after with a three-item UCLA scale. Abstract and journal citation read at the OUP page; design and limitations read in the Harvard Business School working-paper PDF dated 7 November 2025, which may differ in detail from the published version.
Finding: Per the abstract, AI companions alleviated loneliness on par only with interacting with another person and more than other activities such as watching YouTube videos. Participants underestimated the benefit in advance, and feeling heard by the chatbot explained the effect more than the chatbot's performance did. In the working-paper version the authors describe the effect as momentary: loneliness fell after each interaction and had not persisted to the start of the next.
What it supports: That a short conversation with a companion chatbot reduces self-reported loneliness immediately afterwards, more than a task-only chatbot or a passive activity, and that feeling heard is the likely mechanism.
What it does not support: Anything about loneliness after weeks or months of use, or about whether the relief substitutes for contact with people. The authors state the effects are momentary, effect sizes are small to moderate, pre-post measurement may raise the salience of loneliness, and samples are online panels. The study does not test the apps' retention design, which a separate preprint does.

AI companions · loneliness · is it bad to talk to ai when you are lonely

Peer-reviewed#

Folk, D. and Dunn, E. (2026). How Does Turning to AI for Companionship Predict Loneliness and Vice Versa?#

Psychological Science, 37(4), 2026. DOI 10.1177/09567976261427747

Method: Twelve-month longitudinal survey with four waves. 2,149 adults at the start (UK about 50 per cent, US 28, Canada 14, Australia 8), 979 completing all four waves; mean age about 40; recruited through Prolific. Social chatbot use and emotional isolation were self-reported, isolation with a single item. Cross-lagged analysis. Not preregistered; the authors state all findings should be treated as exploratory. Page and abstract read at the SAGE site on 5 October 2026; publication date is given as 23 March 2026 on the journal page and May 2026 in one news report, so only the year is given here.
Finding: Roughly 26 to 30 per cent reported using chatbots for social purposes at each wave. Emotional isolation at one wave predicted more social chatbot use four months later, and increased chatbot use predicted further increased emotional isolation at the next wave. For broader social connection the first pattern appeared and the second did not.
What it supports: That in this sample the relationship between loneliness and companionship use runs in both directions over a year.
What it does not support: Cause. The authors say strong causal claims are not supported because the assumptions are unmet. The isolation measure is a single item, the sample skews towards technologically savvy Prolific users and its 30 per cent social chatbot use exceeds general population rates, and the exploratory design raises multiple-comparison concerns.

AI companions · loneliness · emotional isolation · is it bad to talk to ai when you are lonely

Compiled review#

Sourati, Z., Ziabari, A. S. and Dehghani, M. (2026). The Homogenizing Effect of Large Language Models on Human Expression and Thought #

Trends in Cognitive Sciences

Method: Synthesis across linguistics, psychology, cognitive science and computer science. Not an original experiment.
Finding: Argues that models reflect and reinforce dominant styles while marginalising alternatives, and that reliance on a small number of systems amplifies convergence across users.
What it supports: That the concern is taken seriously across several fields rather than being a commentator's intuition.
What it does not support: Any specific stylistic feature converging. It presents no new measurement of its own.

· how do i keep my own voice when using ai

Peer-reviewed#

Wenger, E. and Kenett, Y. N. (2026). Large language models are homogeneously creative #

PNAS Nexus 5(3), article pgag042, published 24 March 2026. Abstract and results table read at source 1 October 2026

Method: 102 human participants and 22 large language models completed three standardised creativity tasks (the Alternative Uses Task, the Divergent Association Task and Forward Flow). The authors compared the variability of responses across each population, with controls for confounding variables.
Finding: Responses from different LLMs mirrored one another far more than human responses mirrored other humans. Population-level variability scores were 0.459 for LLMs against 0.699 for humans on the Alternative Uses Task, 0.518 against 0.642 on Forward Flow and 0.632 against 0.791 on the Divergent Association Task.
What it supports: That the narrowing seen when one model is used as a creative partner is not particular to that model: across 22 models the outputs converge more than human outputs do.
What it does not support: What happens to a person who works with a model. The study compares models with people, not people with and without models, and the authors say only that using LLMs as creative partners may drive users towards similar outputs. The tasks are short divergent-thinking tests, not creative work.

homogenisation · creativity · is ai bad for creativity

Peer-reviewed#

Zhang, Y., Zhao, D., Hancock, J. T., Kraut, R. and Yang, D. (2026). Interaction with AI companions and psychological well-being #

Nature Human Behaviour, published 4 August 2026. DOI 10.1038/s41562-026-02516-2

Method: Observational study of 1,131 US adults who use Character.AI, with survey responses and 4,664 donated chat sessions (464,687 messages) from 237 participants, triangulating self-reported usage, relationship descriptions and real chat histories. Well-being measured with the Comprehensive Inventory of Thriving. Abstract and author list read at nature.com on 5 October 2026; the article is not open access and the full text was not read.
Finding: Smaller social networks were associated with naming companionship as the primary chatbot use, which was associated with lower well-being. For self-reported companionship use the association was stronger when interactions were intensive and when they were highly disclosive. The authors conclude the association is not uniform and depends on offline social environment and how the chatbot is used.
What it supports: That companionship use of a companion app is associated with lower well-being specifically among users with small offline networks, and more so with intensive, highly disclosive use.
What it does not support: Direction of cause: lower well-being could lead people to companionship use. The sample is US adults using one product, the chat logs come from the 237 who chose to donate them, and the full text was not read, so limitations beyond the abstract are unchecked. Press reports vary on the publication date by a day; the journal page says 4 August 2026.

AI companions · loneliness · well-being · companionship use · is it bad to talk to ai when you are lonely

Work, jobs and the labour market

Exposure, adoption and measured effect, which are three different things.

Compiled review#

Barney, J. (1991). Firm Resources and Sustained Competitive Advantage #

Journal of Management, 17(1), 99-120, March 1991. DOI 10.1177/014920639101700108

Method: Theoretical article in strategic management. No new data. Builds on the stated assumptions that strategic resources are heterogeneously distributed across firms and that those differences are stable over time.
Finding: Sets out four empirical indicators of the potential of a firm resource to generate sustained competitive advantage: value, rareness, imitability and substitutability. Applies the model to several firm resources and draws out implications for other business disciplines. The founding statement of the resource-based view.
What it supports: That a widely used and long-established test exists for whether a resource can confer advantage, against which a purchasable AI licence can be assessed.
What it does not support: Anything empirical, and nothing about AI, which postdates it by three decades. It is a theory with a substantial critical literature, and applying it to a technology three years into commercial diffusion is an argument rather than a finding. The abstract read at source uses rareness, imitability and substitutability; the VRIN acronym is later shorthand and does not appear in the paper's abstract.

competitive advantage · resource-based view · strategy · if everyone has ai where is the advantage

Peer-reviewed#

Garicano, L. (2000). Hierarchies and the Organization of Knowledge in Production #

Journal of Political Economy, 108(5), 874-904. University of Chicago Press

Method: Theoretical model of knowledge acquisition and problem-solving in production, with communication costs and knowledge acquisition costs traded off against each other. No empirical estimation.
Finding: A knowledge-based hierarchy is a natural way to organise the acquisition of knowledge when matching problems with those who know how to solve them is costly. Production workers acquire knowledge of the most common or easiest problems and refer exceptions upward to specialist problem solvers, with problems passed on until somebody solves them or the conditional probability of a solution is too low to justify continuing. Adding layers of problem solvers raises the utilisation rate of knowledge and economises on knowledge acquisition, at the cost of increasing the communication required.
What it supports: That the number of layers in an organisation is a function of two costs rather than of custom, which gives a testable prediction for what happens when either cost falls. It is the model every subsequent claim about AI flattening organisations implicitly relies on.
What it does not support: Anything measured. It is a model, published a quarter of a century before generative AI, and it contains no data about firms, layers or technology adoption.

knowledge hierarchy · organisational design · span of control · problem solving · what happens to work that moves information

Peer-reviewed#

Gathmann, C. and Schoenberg, U. (2010). How General Is Human Capital? A Task-Based Approach #

Journal of Labor Economics, 28(1)

Method: German administrative employment panel, using task overlap between occupations to measure task-specific human capital.
Finding: Task-specific human capital accounts for up to 52 per cent of overall wage growth. Workers move to occupations with similar task profiles, and the distance of those moves shrinks with experience.
What it supports: That skill is portable along task lines rather than being either fully general or locked to one job.
What it does not support: That broad general education transfers well. If anything it argues the opposite, which complicates the study-anything advice rather than supporting it.

· what should i tell my children to study

Working paper#

Frey, C. B. and Osborne, M. A. (2013). The Future of Employment: How Susceptible Are Jobs to Computerisation?#

Oxford Martin School working paper, 17 September 2013. Later published in Technological Forecasting and Social Change, 114 (2017)

Method: Probability of computerisation estimated for 702 detailed US occupations using a Gaussian process classifier, trained on 70 occupations hand-labelled by machine learning researchers at an Oxford workshop.
Finding: In the authors' own words: 'about 47 percent of total US employment is at risk'. The paper's title asks how SUSCEPTIBLE jobs are, and the estimate is of technical susceptibility to computerisation, not a forecast of job losses.
What it supports: That a large share of US employment sits in occupations whose tasks were, in 2013, judged technically susceptible to computerisation.
What it does not support: That 47 per cent of jobs will be, or have been, lost. It is not a prediction, carries no date attached to any loss, and models whole occupations rather than tasks within them. The routine citation as '47 per cent of jobs will disappear' reverses what the paper claims. The working paper is 2013; the journal version is 2017, and the two dates are frequently confused.

labour market · the most quoted ai statistics checked

Peer-reviewed#

Bloom, N., Garicano, L., Sadun, R. and Van Reenen, J. (2014). The Distinct Effects of Information Technology and Communication Technology on Firm Organization #

Management Science, 60(12), 2859-2885. DOI 10.1287/mnsc.2014.2013. Earlier version NBER Working Paper 14975, May 2009

Method: Survey data on worker and plant manager autonomy and span of control, combined with measures of technology adoption, and instrumented using distance from ERP's place of origin and heterogeneous telecommunication costs arising from regulation. The working-paper version describes approximately 1,000 manufacturing firms with 100 to 5,000 employees across the US, France, Germany, Italy, Poland, Portugal, Sweden and the UK, drawn from the CEP double-blind management survey, with technology data from the Harte-Hanks ICT panel.
Finding: Information technology is a decentralising force and communication technology is a centralising force. Better information technologies, ERP for plant managers and computer-assisted design or manufacturing for production workers, are associated with more autonomy and a wider span of control. Technologies that improve communication, such as data intranets, decrease autonomy for workers and plant managers. Instrumenting strengthens the result.
What it supports: That 'technology' has no single organisational effect, and that predicting what a tool does to authority requires knowing whether it lowers the cost of knowing or the cost of telling.
What it does not support: Anything about generative AI, which is both kinds of technology in one interface. Manufacturing plants only, data predating 2009. The sample description above is verified in the working paper; the published article abstract says only 'American and European manufacturing firms', and the typeset article could not be opened at the publisher.

decentralisation · organisational design · span of control · autonomy · what happens to work that moves information

Peer-reviewed#

Bloom, N., Garicano, L., Sadun, R. and Van Reenen, J. (2014). The Distinct Effects of Information Technology and Communication Technology on Firm Organization #

Management Science, 60(12), 2859-2885, 2014. DOI 10.1287/mnsc.2014.2013. Circulated as NBER Working Paper 14975, May 2009, DOI 10.3386/w14975. PROVENANCE: the abstract was read at source on the NBER working-paper record on 21 September 2026, which also confirms the published venue. The published article is paywalled and was not opened; the LSE copy returns an error page. No coefficient or sample size is quoted here.

Method: An original dataset of firms in the United States and seven European countries, measuring production-worker autonomy, plant-manager autonomy and spans of control against separate measures of information technology and communication technology. Identification supported by 'exogenous variation in cross-country telecommunication costs arising from differential regulatory regimes'. Sample size not read at source.
Finding: In the authors' words, 'better information technologies (Enterprise Resource Planning for plant managers and CAD/CAM for production workers) are associated with more autonomy and a wider span of control. By contrast, communication technologies (like data networks) decrease autonomy for both workers and plant managers.' Their account is that technologies reducing information costs let lower-level agents acquire more knowledge, and so widen what those agents may settle for themselves, while technologies reducing communication costs 'substitute agent's knowledge for directions from their managers, and lead to centralization'.
What it supports: That decision rights inside firms move when technology changes the cost of information or of communication, and that the direction of the move depends on which of the two costs falls. It is the measured counterpart to the claim that an unallocated decision right drifts rather than staying put.
What it does not support: Anything about AI, which postdates the technologies measured. The results are associations between technology use and organisational form, with one instrument carrying the causal argument, on manufacturing plants. Nothing here shows that an explicit allocation of decision rights produces better decisions.

decision rights · delegation · autonomy · organisational design · centralisation · how should ai decision rights be allocated

Institutional modelling#

United States Bureau of Labor Statistics (2018). Occupational Projections Evaluation, 2006 to 2016 #

BLS Employment Projections programme

Method: The agency's own retrospective scoring of its 2006 ten-year projections for 840 detailed occupations against actual 2016 outcomes.
Finding: BLS correctly projected whether an occupation would grow or decline 78 per cent of the time, but correctly projected which occupations would grow faster than the economy as a whole only 57 per cent of the time. Projected average occupational growth was 10.4 per cent against an actual 3.6 per cent, the gap driven by a recession the projections could not foresee.
What it supports: That the only organisation which scores its own occupational forecasts gets the useful question, relative growth, barely better than a coin toss, and that its largest errors come from shocks.
What it does not support: That forecasting is worthless. Direction of change at aggregate level was reasonably good. It also predates generative AI, and no equivalent scored record exists for a technological discontinuity.

· what should i tell my children to study

Peer-reviewed#

Acemoglu, D. and Restrepo, P. (2019). Automation and New Tasks: How Technology Displaces and Reinstates Labor #

Journal of Economic Perspectives, 33(2), 3-30. DOI 10.1257/jep.33.2.3

Method: Task-based theoretical framework in which production is allocated between capital and labour, with an empirical decomposition of US industry-level data over recent decades.
Finding: Automation shifts the task content of production against labour through a displacement effect, and therefore ALWAYS reduces the labour share in value added, and may reduce labour demand even while raising productivity. The counterweight is the creation of new tasks in which labour has a comparative advantage, which always raises the labour share. Their decomposition attributes slower employment growth over three decades to an accelerating displacement effect, a weaker reinstatement effect and slower productivity growth.
What it supports: That the distributional question is separate from the productivity question, and that a technology can raise output while reducing labour's share of it. This is the framework almost every serious argument in this area now runs through.
What it does not support: That AI specifically will behave this way. The empirical work predates generative AI and concerns industrial automation and robotics.

labour market · who captures the productivity gains from ai

Peer-reviewed#

Brynjolfsson, E., Rock, D. and Syverson, C. (2021). The Productivity J-Curve: How Intangibles Complement General Purpose Technologies #

American Economic Journal: Macroeconomics, 13(1), 333-72, January 2021. DOI 10.1257/mac.20180386. Earlier version NBER Working Paper 25148

Method: Theoretical model of general purpose technology adoption with unmeasured intangible complementary investment, applied to US national accounts data on computer hardware and software.
Finding: General purpose technologies enable and require significant complementary investments that are often intangible and poorly measured in national accounts. This produces underestimation of productivity growth in a new technology's early years and overestimation later, when the benefits of the intangible investments are harvested, a pattern the authors name the Productivity J-curve. Adjusting for intangibles related to computer hardware and software yields a total factor productivity level 15.9 per cent higher than official measures by the end of 2017. The authors state the AI-related intangible capital effects on measured productivity are currently small but growing.
What it supports: That an early productivity read on a general purpose technology is biased downwards for a structural and quantified reason, and that the direction of the later bias is upwards. It supplies the argument for staging an AI investment review rather than taking a single reading.
What it does not support: That AI will follow the same curve, or on what timescale. The 15.9 per cent figure is for computer hardware and software to 2017, not for AI, and the authors describe current AI intangible effects as small. It is also a measurement argument rather than a forecast of returns to any individual firm.

productivity · intangible capital · general purpose technology · measurement · how long before you know if an ai investment worked if everyone has ai where is the advantage

Peer-reviewed#

Yang, L., Holtz, D., Jaffe, S., Suri, S., Sinha, S., Weston, J., Joyce, C., Shah, N., Sherman, K., Hecht, B. and Teevan, J. (2022). The effects of remote work on collaboration among information workers #

Nature Human Behaviour, 6, 43-54. DOI 10.1038/s41562-021-01196-4, published online 9 September 2021

Method: Observed telemetry on emails, calendars, instant messages, video and audio calls and workweek hours of 61,182 US Microsoft employees over the first six months of 2020, using workers already remote before the pandemic as a comparison to separate firm-wide remote work from other pandemic effects.
Finding: Firm-wide remote work caused the collaboration network to become more static and siloed, with fewer bridges between disparate parts of the organisation, a decrease in synchronous and an increase in asynchronous communication. The authors state these effects may make it harder for employees to acquire and share new information across the network.
What it supports: That changing how information moves reorganises who knows what, without anybody redesigning a role, and that the change is measurable in the network rather than only in self-report.
What it does not support: Anything about AI, and nothing about outcomes. It measures communication structure rather than performance, in one very large technology company, during a pandemic. Full text is paywalled; this entry rests on the published abstract.

collaboration · information flow · organisational network · tacit knowledge · what happens to work that moves information where your ai practice lives

Peer-reviewed#

Acemoglu, D. (2024). The Simple Macroeconomics of AI #

NBER Working Paper 32487; published in Economic Policy, 40(121), 2025

Method: Task-based macroeconomic model applying Hulten's theorem to existing estimates of AI task exposure and task-level cost savings.
Finding: Estimates total factor productivity gains of no more than 0.66 percent over ten years, revised to under 0.53 percent once the difficulty of hard-to-learn tasks is accounted for. Argues AI is likely to widen the gap between capital and labour income rather than reduce labour income inequality.
What it supports: That plausible macroeconomic gains are an order of magnitude smaller than the headline value estimates in circulation.
What it does not support: That AI is unimportant. It models productivity through task-level cost savings, and would not capture effects running through new products, new tasks or capability change.

productivity · macroeconomics · total factor productivity · inequality · how should leaders respond to ai ai workforce strategy will ai replace my job ai and human judgement does ai actually make people more productive the cost of delegation

Working paper#

Autor, D. (2024). Applying AI to Rebuild Middle Class Jobs #

NBER Working Paper 32140, DOI 10.3386/w32140, issued February 2024. Still a working paper: the NBER record carries no published version as at 15 September 2026. Its own acknowledgements say it was drafted for NOEMA Magazine and appeared there as 'How AI Could Help Rebuild The Middle Class'.

Method: An essay rather than a study, written for a general magazine and circulated through the working-paper series. Argument and synthesis rather than empirical test, drawing on the author's prior work on task structure and labour demand.
Finding: Argues that AI's distinctive opportunity is to extend the reach of expertise, letting a wider set of workers with complementary knowledge perform higher-stakes decision tasks currently reserved to elite experts.
What it supports: That there is a serious, well-argued case for AI as an expertise-widening technology rather than an expertise-replacing one.
What it does not support: That this will happen. The author is explicit that the thesis is an argument about what is possible rather than a forecast, and no evidence yet shows it occurring at scale.

expertise · middle-skill work · complementarity · task structure · what is the judgement premium will ai replace my job staying valuable in the age of ai what happens to work that moves information what becomes more valuable as ai gets cheaper pricing work when production is cheap

Working paper#

Bick, A., Blandin, A. and Deming, D. J. (2024). The Rapid Adoption of Generative AI #

NBER Working Paper 32966

Method: Nationally representative US surveys of generative AI use at work and at home.
Finding: By late 2024, nearly 40 percent of US adults aged 18-64 used generative AI and 23 percent of employed respondents had used it for work in the previous week, but only 1 to 5 percent of all work hours were assisted.
What it supports: Enormous reach, thin penetration into actual hours. Adoption is not transformation.
What it does not support: Quality of use. Self-reported use counts any use at all.

AI adoption · ai workforce strategy why learn to prompt is weak career advice

Working paper#

Bonney, K., Breaux, C., Buffington, C., Dinlersoz, E., Foster, L., Goldschlag, N., Haltiwanger, J., Kroff, Z. and Savage, K. (2024). Tracking Firm Use of AI in Real Time: A Snapshot from the Business Trends and Outlook Survey #

US Census Bureau, Center for Economic Studies Working Paper CES 24-16, March 2024; also NBER Working Paper 32319. Not peer reviewed

Method: Analysis of the AI questions in the Business Trends and Outlook Survey, a high-frequency nationally representative survey of US firms, over the collection period covered by the paper.
Finding: Bi-weekly estimates of the AI use rate rose from 3.7 to 5.4 per cent, with an expected rate of about 6.6 per cent by early autumn 2024. 94.6 per cent of AI-using businesses reported no net change in employment in the previous six months attributable to AI use. On discontinuation, which the survey does not ask about directly, 67.9 per cent of current AI users expect to use it in future, 14.5 per cent do not, and 17.6 per cent do not know, so about one in seven current users may de-adopt; the authors attribute this to experimentation that does not yield anticipated benefits or organisational synergies. The authors state explicitly that the analysis does not seek to identify a causal link between AI use and firm performance, only whether use is associated with better performance in general, and that causal analysis awaits integrated data and repeat collections after about three to four years.
What it supports: That the body with the best firm-level data in the world declines to make the causal claim that consultancy and vendor reporting makes routinely, and says in its own text what would be required to make it.
What it does not support: Any effect of AI on firm performance, by the authors' own statement. Adoption rates here are not comparable with later Census figures, because the question was broadened in November 2025 from use in producing goods or services to use in any business function. Census working papers carry the standard disclaimer that they have not undergone the review accorded Census Bureau publications.

AI adoption · measurement · causal inference · firm performance · what becomes more valuable as ai gets cheaper which ai investments should we stop

Vendor research#

Cisco Systems (2024). Cisco 2024 Data Privacy Benchmark Study (release: More than 1 in 4 Organizations Banned Use of GenAI Over Privacy and Data Security Risks)#

Cisco investor news release, 25 January 2024. Seventh edition of the benchmark. Read at source 5 October 2026

Method: Survey of 2,600 privacy and security professionals across 12 geographies. Self-report. Sampling frame and questionnaire not published with the release.
Finding: 27 per cent said their organisation had banned generative AI at least temporarily. 63 per cent had established limits on what data can be entered and 61 per cent on which tools employees can use. 48 per cent admitted entering non-public company information into generative AI tools and 45 per cent employee information.
What it supports: That among privacy and security professionals in early 2024, bans and data limits on generative AI were common, and that entering company and employee information into the tools was widely admitted even by the people responsible for protecting it.
What it does not support: Prevalence across all organisations or workers: the respondents are privacy and security professionals, not a representative sample of employers. It does not show what happened to the information entered or whether any harm followed. Cisco sells security and privacy products, and the 2024 figures may not describe organisations in 2026.

generative AI ban · shadow AI · data protection · AI policy · what are the risks of using ai at work my company banned ai what should i do

Peer-reviewed#

Eloundou, T., Manning, S., Mishkin, P. and Rock, D. (2024). GPTs are GPTs: Labor market impact potential of LLMs #

Science, 384(6702), 1306-1308

Method: Human and model ratings of task exposure across occupational task descriptions.
Finding: Around 80 percent of US workers could have at least 10 percent of tasks affected; about 19 percent could see at least half affected.
What it supports: Where pressure is likely to fall across the occupational structure.
What it does not support: That any job will be lost. This is exposure, not displacement, and the authors say so explicitly. It is the most misquoted number in the field.

task exposure · labour market · will ai replace my job

Vendor research#

Microsoft and LinkedIn (2024). 2024 Work Trend Index Annual Report: AI at work is here. Now comes the hard part #

Microsoft WorkLab, 8 May 2024. Read at source 18 September 2026

Method: Survey of 31,000 full-time employed or self-employed knowledge workers across 31 countries, fielded 15 February to 28 March 2024, combined with LinkedIn labour market data and Microsoft 365 telemetry. Published by a company that sells the tools being surveyed.
Finding: 78 per cent of AI users said they were bringing their own AI tools to work, 80 per cent at small and medium-sized companies. 52 per cent of people who use AI at work said they were reluctant to admit using it for their most important tasks, and 53 per cent worried that using it on important tasks made them look replaceable. 60 per cent of leaders worried that their organisation's leadership lacked a plan and vision to implement AI.
What it supports: That reluctance to disclose AI use and fear of appearing replaceable were reported by about half of AI-using knowledge workers across 31 countries in early 2024, two years before the same pattern was measured in the UK.
What it does not support: Any outcome, and the population is knowledge workers who already use AI. Self-report, with a sponsor whose products are the subject, and the questionnaire is not published in full.

BYOAI · shadow AI · AI adoption · knowledge workers · disclosure · what should we tell employees about ai and headcount should i tell my boss i used ai how do we train people to use ai my company banned ai what should i do

Operator account#

Arvind Krishna, reported by The Wall Street Journal and PYMNTS (2025). IBM CEO: layoffs due to AI led to 'more investment' in other roles #

PYMNTS, 6 May 2025, reporting a Wall Street Journal interview of the same day

Method: A chief executive's account of his own company in a newspaper interview. IBM is reported to have replaced a few hundred human resources staff with AI agents doing spreadsheet analysis, research and email drafting; the timeframe is not stated.
Finding: Krishna: 'While we have done a huge amount of work inside IBM on leveraging AI and automation on certain enterprise workflows, our total employment has actually gone up, because what it does is it gives you more investment to put into other areas.' The areas named are software engineering, sales and marketing, which he characterised as work needing critical thinking and dealing with people.
What it supports: That one large employer describes the same decision the other way round from Klarna: automate the routine, then choose where the freed money goes. The choice of where it goes is the leadership act, and IBM's chief executive presents it as one.
What it does not support: That AI raised IBM's headcount; total employment moves for many reasons and no figures separating them were given. 'A few hundred' HR roles is the company's own round number. It is a chief executive's description of his own choices.

ai leadership · ibm · reinvestment · headcount · work design · automation · ai leadership

Peer-reviewed#

Autor, D. and Thompson, N. (2025). Expertise #

NBER Working Paper 33941, DOI 10.3386/w33941, issued June 2025; published in the Journal of the European Economic Association, 23(4), August 2025, 1203-1271, DOI 10.1093/jeea/jvaf023. Originally the Joseph R. Schumpeter lecture to the European Economic Association, Rotterdam, 29 August 2024. Both records re-read at source 15 September 2026 and the journal details reproduce exactly.

Method: Four decades of task data across 303 US occupations, 1980-2018, with a novel content-agnostic measure of task expertise.
Finding: Automation that removed the LESS expert tasks raised wages and reduced employment. Automation that removed the EXPERT tasks lowered wages and increased employment.
What it supports: That which tasks are automated matters more than how many, and gives a testable way to ask whether a given role is appreciating or commoditising.
What it does not support: Anything measured about generative AI. The data ends in 2018, so this is a lens, not a forecast.

expertise · task automation · human capability · what is the judgement premium staying valuable in the age of ai will ai replace my job which tasks do workers not want automated what becomes more valuable as ai gets cheaper

Peer-reviewed#

Babina, T., Fedyk, A., He, A. and Hodson, J. (2025). Firm Investments in Artificial Intelligence Technologies and Changes in Workforce Composition #

Chapter 3 in Technology, Productivity, and Economic Growth, NBER Studies in Income and Wealth 83, University of Chicago Press, July 2025. Earlier version NBER Working Paper 31325

Method: Worker resume and job-posting datasets combined to measure firm-level AI investment against workforce composition variables including educational attainment, specialisation and hierarchy.
Finding: AI investments are associated with a flattening of firms' hierarchical structure, with significant increases in the share of workers at the junior level and decreases in the shares in middle-management and senior roles.
What it supports: That the composition of the flattening, and not only its existence, is measurable, and that the layer shrinking is the one that has historically developed juniors into seniors.
What it does not support: Any magnitude. The abstract read for this entry states direction and significance and no percentage, so none should be attributed to it. Association rather than causation, and firms that invest in AI differ from those that do not in many other ways.

middle management · workforce composition · flattening · AI investment · what happens to work that moves information

Institutional survey#

CIPD; Jake Young and Derek Tong (2025). CIPD Good Work Index 2025 #

CIPD, June 2025

Method: CIPD/YouGov UK Working Lives survey of 5,017 UK employees, fielded in January and February 2025 with quotas and weights to ONS figures on gender, working hours, organisation size, sector, industry and age. Annual series since 2018.
Finding: 16% of UK employees say job tasks have been automated by AI, ranging from 25% of 18 to 24 year olds to 6% of those aged 55 and over. Of those, 85% say it improved their performance and 5% say it worsened it. Tasks automated were most often repetitive (69%), complex (45%) or dangerous (17%). Employees with AI-automated tasks report higher job satisfaction (78% versus 53%) and are more likely to say work has a positive effect on mental health (72% versus 39%).
What it supports: That in early 2025 AI had touched the tasks of a minority of UK workers, concentrated among the young, and that those workers report better job quality than others.
What it does not support: The satisfaction gap is not causal; younger, higher-skilled workers in adopting sectors may already have better jobs. All measures are self-reported and the AI questions are a small part of a broader job quality survey.

uk workforce · job quality · task automation · job satisfaction · age differences

Working paper#

Ewens, M. and Giroud, X. (2025). Corporate Hierarchy #

NBER Working Paper 34162, issued August 2025, revised October 2025. DOI 10.3386/w34162. Not peer reviewed

Method: A measure of corporate hierarchy for over 3,100 US public firms, built from online resumes of 7 million US-based workers, 2016 to 2023, using a network estimation technique to identify hierarchical layers. AI adoption is proxied by AI job postings following Babina, Fedyk, He and Hodson.
Finding: Firms average ten hierarchical layers and a pyramidal structure, with the average and median number of layers declining across the sample period. More hierarchical firms show a more educated workforce, higher internal promotion rates, longer tenure, higher operating performance and higher administrative costs. Companies flattened their hierarchies following adoption of AI technologies, while pharmaceutical companies added layers after Covid-19. The authors state the AI tests are under-powered and that point estimates are significant at the 10 per cent level regardless of the adoption metric.
What it supports: That the Garicano prediction has now been tested against firm-level data and points in the predicted direction, which is more than the flattening discourse previously had.
What it does not support: That AI causes flattening. The authors call their own tests under-powered at the 10 per cent level, adoption is measured by job postings rather than by use, hierarchy is inferred from self-reported resumes, and the sample is US public firms. Not peer reviewed. Figures of 2,500 firms and 16 million employees circulate from an earlier draft and are wrong for this version.

organisational design · flattening · middle management · AI adoption · what happens to work that moves information

Institutional survey#

Georgetown University Center on Education and the Workforce (2025). The Major Payoff: Evaluating Earnings and Employment Outcomes Across Bachelor's Degrees #

Georgetown CEW

Method: American Community Survey earnings and employment data across 152 majors for prime-age workers and 142 for early career.
Finding: Median prime-age earnings run from 58,000 dollars in education and public service to 98,000 in STEM. Within STEM alone the range is 64,000 to 146,000, and several humanities majors beat the STEM 25th percentile.
What it supports: That the spread within a field is often wider than the gap between fields, which undercuts advice given at the level of STEM against humanities.
What it does not support: Anything about the future. It is a cross-sectional snapshot of people already employed.

· what should i tell my children to study

Working paper#

Humlum, A. and Vestergaard, E. (2025). Still Waters, Rapid Currents: Early Labor Market Transformation under Generative AI #

NBER Working Paper 33777, revised March 2026

Method: Adoption surveys linked to administrative labour records, roughly 25,000 workers across 7,000 Danish workplaces in 11 exposed occupations.
Finding: Precise null effects on earnings and hours two years after ChatGPT, ruling out effects larger than 2 percent, alongside substantial task reorganisation and new tasks in AI oversight and integration.
What it supports: That the structure of work moves well before earnings do, and that pay is the slowest available indicator.
What it does not support: That the same holds elsewhere. Denmark is high-trust, high-wage and heavily unionised, and two years is early.

labour market · task reorganisation · AI adoption · ai workforce strategy will ai replace my job ai and human judgement is ai already causing job losses

Institutional survey#

Institute of Student Employers (2025). Student Recruitment Survey 2025 #

Institute of Student Employers, 2025, with 2026 outlook published 7 January 2026

Method: Trade association survey of 155 employers covering over 31,000 student hires from more than 1.8 million applications in the 2024-2025 cycle.
Finding: An average of 89 applications per vacancy. The ISE reports a projected 7 per cent drop in student vacancies for 2026, while 30 per cent of employers increased student hiring.
What it supports: That the contraction is uneven. Nearly a third of employers increased student hiring in a falling market, which cuts against any account of uniform collapse.
What it does not support: Whole-market figures. It is a membership survey, and its members are organisations that run structured student recruitment in the first place.

graduate recruitment · early careers · applications · do apprenticeships still work

Operator account#

Luis von Ahn, reported by Fortune (2025). Duolingo CEO walks back AI-first comments: 'I do not see AI as replacing what our employees do'#

Fortune via Yahoo Finance, May 2025, reporting an April 2025 all-hands email and a LinkedIn post of 23 May 2025

Method: Two statements by a chief executive about his own company, a month apart, reported by the press. No data.
Finding: The April email said Duolingo would 'gradually stop using contractors to do work AI can handle', that AI use would count in performance reviews, and that headcount would be added only 'if a team cannot automate more of their work'. After a public backlash, von Ahn wrote on 23 May: 'I do not see AI as replacing what our employees do (we are in fact continuing to hire)' and 'I see it as a tool to accelerate what we do, at the same or better level of quality'.
What it supports: That an AI-first declaration without a written boundary underneath it was reversed in a month under public pressure, with no change in the technology in between. It is the clearest public example of the slogan outrunning the strategy.
What it does not support: That anything operational changed. The retreat is a statement; the contractor line was not withdrawn; contractors are not employees. Both statements are one man's framing of his own company.

ai leadership · ai-first · duolingo · contractors · reversal · is ai first a strategy or a slogan what happened to companies that cut staff for ai

Operator account#

Microsoft (2025). Learn about retention for Copilot and AI apps #

Microsoft Learn, Purview documentation, page dated 26 September 2025. Read at source 29 September 2026

Method: Vendor technical documentation describing where Microsoft 365 Copilot prompts and responses are stored and who can search them.
Finding: Data from generative AI messages is stored in a hidden folder in the mailbox of the user who runs the AI app. The folder is not designed to be directly accessible to users or administrators but holds data that compliance administrators can search with eDiscovery tools. Messages remain searchable until permanently deleted, and messages visible in an AI app are not an accurate reflection of whether they are retained or permanently deleted for compliance purposes.
What it supports: That Microsoft documents prompts and responses as stored in a searchable mailbox folder for compliance administrators, and that what a user sees is not a guide to what is kept.
What it does not support: What any organisation has configured or searched, or anything about other vendors' tools. The documentation is the vendor's own and is revised over time.

data retention · eDiscovery · workplace monitoring · can my employer see what i put into ai who owns the capability you built at work where your ai practice lives what it would take to leave

Institutional survey#

OECD (2025). How widespread is algorithmic management in workplaces?#

OECD, December 2025. The report's web summary was read at source on 21 September 2026; the PDF would not open through the fetcher, and the country figures were read in The Next Web's report of 20 September 2026 and CNBC's of the same day

Method: An employer survey of 6,047 firms in France, Germany, Italy, Japan, Spain and the United States, fielded by Ipsos between June and August 2024 among mid-level managers at workplaces with 20 or more employees, per The Next Web. Algorithmic management is defined by the OECD as software used to automate or support managerial tasks in how work is instructed, monitored and evaluated.
Finding: Per The Next Web's account, the share of managers reporting at least one algorithmic management tool was 90 per cent in the United States, 81 in France, 78 in Germany and Spain, 76 in Italy and 40 in Japan; 67 per cent of US firms used a tool to sanction poor performance against 4 per cent across the four European countries and 1 per cent in Japan; 91 per cent of managers said workers were made aware, a figure the OECD questioned, and most said workers could not opt out. The OECD's own summary records managers' concerns about unclear accountability for algorithmic decisions, inability to follow the tools' logic and inadequate protection of workers' health.
What it supports: That by mid-2024 software to instruct, monitor or evaluate workers was in use at the large majority of surveyed workplaces in five of six countries, that its use to discipline workers was concentrated in the United States, and that the managers operating it reported not being able to follow its logic or say who was accountable for its decisions.
What it does not support: Anything about generative AI or agents, which the survey predates in most workplaces. Manager self-report, and the country figures here are as reported by a news outlet rather than read in the PDF. It does not measure outcomes for workers or how often a human reviews a sanction.

algorithmic management · monitoring · surveillance · accountability · discipline · OECD · what board oversight of ai looks like

Peer-reviewed#

Reif, J. A., Larrick, R. P. and Soll, J. B. (2025). Evidence of a social evaluation penalty for using AI #

Proceedings of the National Academy of Sciences, 122(19), e2426766122, May 2025. Read at source 18 September 2026

Method: Four preregistered online experiments with over 4,400 participants in total, the abstract's own figure, run between March 2024 and February 2025. Participants rated described workers or candidates who used AI, who received equivalent help by other means, or who received none; one study placed participants as hiring managers evaluating candidates who reported different frequencies of AI use.
Finding: People described as using AI for a task were rated as lazier, less competent, less diligent, less independent and less self-assured than people described as receiving comparable non-AI help, with no difference on ambition or dominance. The penalty was largest among evaluators who used AI little themselves and smallest among those who used it weekly or daily; hiring managers who did not use AI preferred candidates who reported no use, while managers who used it regularly preferred candidates who used it daily. The penalty disappeared when the tool's usefulness for the specific task was made explicit. Participants also reported reluctance to disclose their own AI use to managers and colleagues.
What it supports: That a measurable social penalty for visible AI use existed in these samples, that it depends on the evaluator's own familiarity with the tools and on whether the tool's fit to the task is stated, and that the people who anticipate the penalty say they would conceal use.
What it does not support: Behaviour in real organisations: the targets are described, not observed, the participants are online panels, mostly in the United States, and the authors say the penalty is likely to shift as AI use becomes ordinary. It measures perception of users, not the quality of their work.

AI stigma · social evaluation · disclosure · impression management · AI adoption · what should we tell employees about ai and headcount should i tell my boss i used ai

Peer-reviewed#

Schilke, O. and Reimann, M. (2025). The transparency dilemma: How AI disclosure erodes trust #

Organizational Behavior and Human Decision Processes, 188, article 104405, May 2025. Read at source 30 September 2026

Method: Thirteen experiments with a within-paper meta-analysis, testing how observers' trust in an actor changes when the actor's use of AI is disclosed, across communication, analytic and creative tasks.
Finding: Actors who disclose their AI use are trusted less than those who do not. In the final experiment a tax advisor exposed by a news leak was trusted less than one who disclosed voluntarily, though disclosure still reduced trust against no disclosure. The penalty was smaller among people with positive attitudes to technology and among those who rated the AI as accurate, and did not vanish for them.
What it supports: A consistent trust penalty for disclosed AI use in online experiments, and that being found out is worse than saying so.
What it does not support: What happens to a real firm's customers, contracts or retention. Participants rated described actors, and the paper does not test disclosure wording that states what the human checked.

AI disclosure · trust · transparency · should we tell customers we used ai

Operator account#

Sebastian Siemiatkowski, reported by Bloomberg and CX Dive (2025). Klarna changes its AI tune and again recruits humans for customer service #

CX Dive, 9 May 2025, reporting a Bloomberg interview of 8 May 2025

Method: A chief executive's account of his own company, given in an interview, reported by trade press. No data beyond the company's own statements. In February 2024 Klarna had said its AI assistant handled two-thirds of customer service chats, 2.3 million in its first month, doing the work of 700 agents.
Finding: Siemiatkowski: 'As cost unfortunately seems to have been a too predominant evaluation factor when organizing this, what you end up having is lower quality. Really investing in the quality of the human support is the way of the future for us.' And: 'I just think it's so critical that you are clear to your customer that there will be always a human if you want.' The company said it would recruit human agents again, keeping the assistant for routine enquiries.
What it supports: That the most-cited corporate case for replacing people with AI was reversed by the man who made it, on the grounds that the decision had been made on cost alone. It is the clearest public example of a leadership decision, not a technology failure, being the thing that went wrong.
What it does not support: How far Klarna reversed, or what the quality gap was; no figures were published for either. It is one company and one chief executive's framing of his own change of mind, and the earlier 700-agent claim was also his.

ai leadership · klarna · customer service · reversal · cost versus quality · human in the loop · ai leadership

Working paper#

Shao, Y., Zope, H., Jiang, Y., Pei, J., Nguyen, D., Brynjolfsson, E. and Yang, D. (2025). Future of work with AI agents: auditing automation and augmentation potential across the US workforce #

arXiv preprint

Method: An audit of 844 tasks across 104 occupations, combining what 1,500 domain workers say they want automated with expert assessment of what current systems can do, and scoring each task on a five-level Human Agency Scale from AI operating alone to AI unable to function without continuous human involvement.
Finding: Worker preference and technical capability diverge widely. The audit identifies a set of tasks where systems are capable and workers do not want automation, and another where workers want it and capability is absent.
What it supports: That preference about automation can be measured at the level of the individual task, and that a scale of desired human involvement can be applied consistently across occupations.
What it does not support: What should be automated. The scale records what workers want and what experts judge possible, neither of which is a normative answer. It is also a preprint.

automation · augmentation · delegation · levels of automation · the use or keep test levels of automation

Vendor research#

Slack Workforce Lab (Salesforce) (2025). The New AI Advantage: Daily AI-Users Feel More Productive, Effective, and Satisfied at Work #

Slack, June 2025

Method: Survey of 5,156 desk workers in Australia, France, Germany, Japan, the UK and the US, fielded by Qualtrics from 9 April to 1 May 2025, not targeting Slack or Salesforce employees or customers. Comparisons are to the November 2024 wave of the Workforce Index.
Finding: 60% of desk workers use AI at work and 42% use it at least weekly; daily use is 233% higher than in November 2024. Daily users are 64% more likely to report very good productivity, 58% more likely to report very good focus and 81% more likely to report very good job satisfaction than non-users. 96% of AI users have used it for tasks outside their expertise. 40% have used AI agent chatbots and 23% have assigned an agent to complete work.
What it supports: That among desk workers in six rich economies, self-reported AI use grew fast in 2024 to 2025 and frequent users rate their own work experience more highly.
What it does not support: Cross-sectional self-report cannot separate whether AI improves productivity or whether already productive, satisfied workers adopt AI. Salesforce sells AI agents, and the six-country desk-worker frame excludes frontline and non-desk work.

desk workers · daily ai use · job satisfaction · ai agents · self-reported productivity

Operator account#

Tobi Lütke, reported by TechCrunch (2025). Shopify CEO tells teams to consider using AI before growing headcount #

TechCrunch, 7 April 2025, reporting an internal memo Lütke published on X the same day

Method: An internal memo from a chief executive to his company, made public by the author after it leaked. No data; a statement of policy.
Finding: 'Before asking for more headcount and resources, teams must demonstrate why they cannot get what they want done using AI.' The memo also asks teams: 'What would this area look like if autonomous AI agents were already part of the team?'
What it supports: That one large employer made AI the default option before a new hire, in writing, publicly and with a date. It is the clearest published example of an AI-first rule, and the memo frames it as a question about the work rather than a statement about people.
What it does not support: What the rule did. Shopify has not published its effect on hiring, output or quality. A published policy is evidence of intent, not of outcome.

ai leadership · ai-first · headcount · shopify · operating model · is ai first a strategy or a slogan what happened to companies that cut staff for ai

Institutional survey#

Allen, J. S. (2026). Monitoring AI Adoption in the U.S. Economy #

FEDS Notes, Board of Governors of the Federal Reserve System, 3 April 2026. DOI 10.17016/2380-7172.4032

Method: Comparison of three independent US adoption measures: the Census Business Trends and Outlook Survey (firm-level, around 20,000 responses per wave), the Real-Time Population Survey (individual-level, 5,000 to 6,000 responses) and the Atlanta Fed Survey of Business Uncertainty (senior leaders, 1,032 responses). Four-period moving averages used for all BTOS calculations.
Finding: About 18 per cent of firms had adopted AI as of year-end 2025 on the BTOS. Work-related generative AI adoption in the RPS stood at about 41 per cent of the workforce as of November 2025, with daily use at 12 per cent. The SBU gives an employment-weighted firm adoption rate of about 78 per cent and an LLM adoption rate of about 54 per cent. Allen attributes the variation mainly to differences in sampling distributions and units of analysis, with question framing, the materiality of reported usage, information asymmetries and social desirability bias also contributing, and states senior leaders may face pressure to report AI usage as an efficiency initiative. The Census Bureau broadened its question in November 2025; the do-not-know rate ran at 10 to 11 per cent.
What it supports: That headline AI adoption figures differing by sixty points can all be correct, because they measure different units, and that the choice of measure decides the answer before any analysis begins.
What it does not support: Any productivity or employment effect. The note explicitly does not estimate AI's contribution to output, GDP or productivity, and names those as open questions beyond its scope. It also makes no claim that adoption has plateaued; the only slowdown language is deceleration in the second quarter of 2025.

AI adoption · measurement · survey method · if everyone has ai where is the advantage how long before you know if an ai investment worked

Vendor research#

Anthropic; Maxim Massenkoff, Eva Lyubich, Szymon Sacher, Zoe Hitzig, Shaoyi Zhang, Ryan Heller, Peter McCrory (2026). Anthropic Economic Index report: Cadences #

Anthropic, June 2026

Method: Privacy-preserving analysis of Claude.ai, Claude Desktop, Claude Code and first-party API usage from 10 April to 10 June 2026, classified to O*NET tasks and output types. Paired with an Anthropic Economic Index survey of about 81,000 Claude users (December 2025) and roughly 9,700 respondents linked to their own usage data from mid-May to early June 2026.
Finding: Personal (non-work) conversations rise from around 35% of the total on weekdays to just under 50% at weekends. 93% of conversations produce an identifiable output; the most common are explanations (17%), documents and reports (15%) and guidance (11%). Over a third of surveyed users expect AI to be able to do most or nearly all of their work tasks next year, while 10% rate losing their own job as likely or very likely. Users with more automated usage patterns are more optimistic about AI's effect on pay and job security.
What it supports: That on one frontier model, work use follows the working week and most conversations end in a concrete output; and that heavy users hold both high capability expectations and low personal job risk expectations.
What it does not support: Claude users are self-selected and skew technical; usage patterns on one product are not labour market outcomes. The survey is of the vendor's own customers about the vendor's product, and Anthropic has a commercial interest in the framing.

ai usage data · task classification · augmentation · worker expectations · claude

Argued perspective#

Barr, M. S., Board of Governors of the Federal Reserve System (2026). Economic Conditions and Monetary Policy #

Speech at the Detroit Economic Club, 29 September 2026. Read at source 2 October 2026

Method: A central bank governor's speech with one section on AI and the labour market, reporting no new data.
Finding: 'While there are some indications that AI may already be a factor limiting new job opportunities for entry-level workers in sectors heavily exposed to AI, across the economy there is little evidence of significant displacement so far.' 'If labor market changes happen quickly, it will be hard for workers to adjust and dislocations might be large, whereas a more gradual adoption might permit more orderly adjustments.' Barr says 'now is the time for society to begin to consider how to address these potential disruptions, while AI adoption is in its relatively early stages'.
What it supports: How a Federal Reserve governor read the US evidence at the end of September 2026.
What it does not support: Any measurement: it is a judgement on others' data, about the United States, in a speech mainly on monetary policy.

labour market · displacement · central banks · AI and employment · is ai already causing job losses

Working paper#

Bonney, K., Breaux, C., Dinlersoz, E., Foster, L., Haltiwanger, J. and Pande, A. (2026). The Microstructure of AI Diffusion: Evidence from Firms, Business Functions, and Worker Tasks #

US Census Bureau, Center for Economic Studies Working Paper CES-26-25, April 2026

Method: Nationally representative data from the 2026 AI supplement to the US Census Bureau's Business Trends and Outlook Survey, analysed at three layers: overall firm use, deployment across business functions, and worker-task use. Reference period November 2025 to January 2026.
Finding: 18 per cent of firms used AI in a business function, rising to 32 per cent employment-weighted, with adoption expected to reach 22 per cent within six months. Use rates reach 50 to 60 per cent, and 60 to 70 per cent employment-weighted, for very large firms in Information, Professional Services and Finance. Among adopters, 57 per cent integrate AI in three or fewer business functions, most commonly Sales and Marketing (52 per cent), Strategy and Business Development (45 per cent) and IT (41 per cent). Workers use AI in work-related tasks in 23 per cent of firms, 41 per cent employment-weighted, and 65 per cent of firms limit use to three or fewer tasks. Most users, 66 per cent, rely on AI solely to augment tasks, and AI-related employment decreases occur in only 2 per cent of firms. Regression shows a positive correlation between firm commercial performance and the breadth of AI integration, holding across functional deployment, task-level use and operational investment; functional breadth and operational investment are positively associated with employment decreases, while worker-task integration shows no significant link to headcount reduction once the other two are accounted for.
What it supports: That AI diffusion is highly uneven by firm size and sector, and shallow even among adopters, which contradicts the premise that competitors hold equivalent capability.
What it does not support: Causation in either direction between adoption and performance: this is a cross-section of firms, and better-run firms may simply adopt more. Also not a stable time series. The Census Bureau broadened the underlying question in November 2025 from use in producing goods or services to use in any business function, which moved the level. A working paper, not peer reviewed, and US-only.

AI adoption · diffusion · firm size · augmentation · if everyone has ai where is the advantage how long before you know if an ai investment worked which tasks do workers not want automated what happens to work that moves information

Working paper#

Boyd-Swan, C. and Reynolds, C. L. (2026). Early Evidence on Employer Responses to College Student Use of Generative AI #

Annenberg Institute EdWorkingPaper 26-1604, October 2026. Abstract and paper read at source 9 October 2026

Method: Survey experiment on about 1,750 US professionals with hiring responsibilities, recruited through Prolific between 21 May and 8 June 2026 and randomised to a control or to one of two framings of student AI use (assisting students, or substituting for their effort and ability) before reviewing two constructed CVs.
Finding: 'Results show that the framing increased respondents' concerns about evaluating job applicants, with a larger effect in the second treatment group.' The substitution framing 'clearly decreased' interview interest and willingness to negotiate starting salaries. Of the information that might allay concern, 'only in-person tests were selected by both treatment groups.'
What it supports: That being reminded students use AI makes people who hire less sure what an application shows, and that an in-person test is the evidence they reach for.
What it does not support: Real hiring: 'Our outcomes are hypothetical evaluations rather than real hiring decisions', the CVs were written by the researchers, and the panel is online and US only. Not peer-reviewed.

hiring · graduates · credentials · proof of work · does ai make juniors better or only look better

Working paper#

Brynjolfsson, E., Chandar, B. and Chen, R. (2026). Canaries in the Coal Mine? Six Facts about the Recent Employment Effects of Artificial Intelligence #

Stanford Digital Economy Lab, updated August 2026

Method: ADP payroll microdata covering millions of US workers, comparing employment by age and by occupational AI exposure since the release of ChatGPT.
Finding: No widespread economy-wide displacement. But employment among 22 to 25 year olds in highly AI-exposed occupations sits about 19 percent below where it would be had it tracked similarly aged workers in less-exposed occupations. The underlying levels matter and are easily lost: employment of that age group in the two most exposed quintiles fell about 11 percent between November 2022 and June 2026, while the same age group in the three least exposed quintiles grew about 10 percent. The 19 is the distance between those two, not a fall of 19. The divergence runs through reduced hiring rather than increased separations, and declines concentrate in occupations where AI substitutes for human tasks; where it complements, employment is flat or rising, especially for experienced workers.
What it supports: That the entry-level effect is real, measurable in payroll data rather than inferred, and specific to substitution rather than to AI exposure as such.
What it does not support: Economy-wide job destruction, which the authors explicitly rule out on current evidence. Nor does it establish causation: youth hiring is sensitive to interest rates, cohort size and hiring freezes, and the design is observational. Two further cautions, both from the paper itself. First, the phrase 'below trend' is wrong and is how this figure is usually repeated: the comparison is with same-aged workers in less-exposed occupations, not with a historical trend line and not with older workers. Second, the figures in circulation are not a worsening time series. 13 and 16 percent come from an earlier regression estimator, 15 and 19 from the descriptive kept-pace measure the authors now prefer, so quoting them in sequence as though the effect grew misrepresents a change of method as a change in the world. Note also that the ADP figures widely cited alongside this paper are the same payroll data reported differently, not independent corroboration.

entry-level employment · youth employment · substitution versus complementarity · hiring · will ai replace entry level jobs missing rungs the best writing on ai what is the ai employment gap is ai already causing job losses

Institutional survey#

Deloitte UK, fieldwork by Ipsos UK (2026). British workers spend one billion pounds of their own money on gen AI for work (UK GenAI Workforce Survey)#

Deloitte UK press release, 16 September 2026. Read at source 18 September 2026

Method: Online survey of 25,000 UK workers aged 18 to 70, employed and self-employed, fielded by Ipsos UK between 7 May and 10 June 2026, quota sampled and weighted for gender, age, working status, sector, region, social grade, education and ethnicity. Self-report throughout. The full report and questionnaire were not published with the release.
Finding: 63 per cent of UK working adults say they use generative AI for work and 24 per cent use it every day. Of the users, 31 per cent use it without their employer's knowledge, 23 per cent perceive a stigma around using it at work, and 64 per cent of weekly users worry that their manager will think AI can do their job. About half say they have had no formal training in using it safely and effectively, and 65 per cent report a lack of convincing leadership guidance. 17 per cent pay for at least one tool themselves, which Deloitte scales to £958 million a year. Users report saving about 70 minutes a week. Hayley McKelvey, Deloitte UK's chief AI officer, is quoted: 'Workers who feel stigma around adopting GenAI tools are more likely to conceal use.'
What it supports: That in a large weighted UK sample, concealed use, perceived stigma and fear of being judged replaceable are reported together and at scale, and that most users say they have had neither training nor convincing guidance from leadership.
What it does not support: Any outcome: the time saved, the quality of the work, and the stigma are all self-reported. The 31 per cent measures use the employer does not know about, which is not the same as use a policy forbids. Deloitte advises organisations on the adoption it is measuring, and the questionnaire is not public, so the wording behind each figure cannot be checked.

shadow AI · generative AI · AI adoption · workforce · stigma · AI training · what should we tell employees about ai and headcount what to do when people work around the ai policy should i tell my boss i used ai can my employer see what i put into ai how do we train people to use ai what are the risks of using ai at work my company banned ai what should i do

Argued perspective#

Department for Business and Trade (UK) (2026). Make Work Pay: consultation on workplace monitoring technologies #

UK government consultation, issued 8 July 2026, closed 30 September 2026. Read at source 2 October 2026

Method: A government consultation document of 68 pages setting out a definition, eight principles and three options (a statutory code of practice, a legislative requirement to consult and negotiate, and non-statutory guidance) for England, Wales and Scotland.
Finding: Defines workplace monitoring technologies as 'digital tools used by employers to collect, track, analyse or make decisions based on information about workers and their activities'. Cites 'a 2025 survey of UK managers finding that 1 in 3 organisations actively monitor an employee's digital activity, up from 1 in 5 employers in ICO research in 2023'. Under the first option a tribunal would have discretion to adjust compensation by up to 25 per cent where an employer unreasonably failed to follow the code. Under the second, employers' plans to adopt the technology 'would be subject to consultation and negotiation, with a view to agreement of trade unions or elected staff representatives where there is no trade union', and the process 'would not require agreement to be reached in all cases'. The document says the options do 'not indicate a settled government preference'.
What it supports: What the UK government was considering in 2026 and the principles it regards as good practice, including worker engagement and human oversight with a route to challenge.
What it does not support: Any decision: the consultation closed on 30 September 2026 and no response has been published. The one-in-three figure comes from a survey the document does not name. It covers monitoring technology and not AI in general.

workplace monitoring · worker voice · human oversight · consultation · algorithmic management · can an employer use ai to discipline or dismiss someone how should employees be involved in decisions about ai

Institutional survey#

Dixon, J.C. (2026). I surveyed workers to see if AI had caused job losses and was surprised by the findings #

The Conversation, 27 August 2026. YouGov survey commissioned by the author, College of the Holy Cross. Described by the author as a study in progress and not peer-reviewed

Method: 1,250 employed US workers surveyed online by YouGov between 30 July and 4 August 2026, 25 questions, weighted to the employed US population on age, gender, race and education. Opt-in panel. The author discloses an AI consulting business.
Finding: About 3 per cent said they had lost a job to AI since 2023, against roughly 6 per cent who said they held a job that did not exist before AI and about 9 per cent reporting an AI-related promotion. Around 95 per cent said no, with 3 to 4 per cent unsure.
What it supports: That self-attributed AI job loss is rare among people currently in work, and that reported AI-created gain in the same sample runs at twice the reported loss. Useful as a counterweight to displacement figures drawn from payroll data, which cannot ask anyone why.
What it does not support: Displacement in the workforce, and the reason is structural rather than a matter of sampling error. Every respondent was employed when surveyed, in Dixon's own words 'none were jobless, whether due to AI or another reason'. Anyone displaced by AI and still out of work is excluded from the numerator and the denominator alike, so the 3 per cent counts only those who lost a job and have since found another. It is a floor among survivors, not an estimate of displacement. It is also not a measure of fear: the question asked what happened, not what respondents expect.

entry-level employment · displacement · self-report · will ai replace entry level jobs what is the ai employment gap

Working paper#

Fairlie, R. W. and Wu, J. (2026). The Early Impacts of AI on Employment among Recent College Graduates #

NBER Working Paper 35796, September 2026. Abstract read at source 2 October 2026

Method: US Current Population Survey microdata for June, July and August 2026; difference-in-differences and event-study models comparing recent graduates with older graduates and with young workers without a degree; an expanded unemployment measure adding those who report wanting a job; interactions with occupational AI exposure and remote work availability.
Finding: For recent graduates, 'unemployment rates did not spike in summer 2026 relative to summer months in previous years and did not rise in a significant way relative to older college graduates or young workers without a college degree'. The expanded measure 'adds nearly two percentage points' to graduate unemployment and still shows no significant increase. The authors find 'some evidence of a positive relationship with remote work availability'.
What it supports: That three months of US survey data show no jump in unemployment among recent graduates in the first summer the authors think an effect could be detected.
What it does not support: Anything about hiring that did not happen, about employment as distinct from unemployment, about later months or about the UK. The authors take 'an agnostic approach to defining treatment timing'. Not peer-reviewed; abstract only.

entry-level jobs · graduates · unemployment · AI and employment · is ai already causing job losses

Institutional survey#

Federal Reserve Bank of New York (2026). The Labor Market for Recent College Graduates #

New York Fed, data through 2026 Q2

Method: Current Population Survey and ACS tracking of unemployment and underemployment by major since 1990.
Finding: As at the second quarter of 2026, unemployment among recent graduates runs around 5.6 per cent and underemployment around 42 per cent, the highest since 2020.
What it supports: That the graduate labour market has tightened measurably, and that underemployment is the larger number.
What it does not support: Attribution to AI. The series is descriptive and the Fed states it is not a forecast.

· what should i tell my children to study

Institutional survey#

Gallup; Andy Kemp (2026). Organizational AI Adoption Jumps Six Points #

Gallup, July 2026

Method: Gallup Panel survey of 22,573 employed US adults aged 18 and over, working full or part time, fielded 6 to 20 May 2026 with a margin of error of plus or minus 0.9 percentage points at 95% confidence. Quarterly series since Q2 2023; the previous wave (4 to 19 February 2026) had 23,717 respondents.
Finding: 52% of US workers use AI in their role at least a few times a year, 30% a few times a week or more and 15% daily; in Q2 2023 the any-use figure was 21%. 47% say their organisation has integrated AI tools, up from 41% the previous quarter. The most common uses are writing and editing (51%), search or research (49%) and general assistance or problem-solving (39%). Among employees using AI for one or two purposes, 45% report a positive effect on productivity; among those using it for seven or more purposes, 90% do. In Q1 2026, 8% of employees in AI-adopting organisations strongly agreed that AI had transformed how work gets done, and 18% of all employees thought their job was likely to be eliminated within five years.
What it supports: That US workplace AI use has more than doubled in three years on a probability-based panel, and that reported benefit scales with breadth of use.
What it does not support: Self-reported productivity is not measured output, and breadth of use may follow from being in roles where AI helps most. US only. It does not measure quality of work or skill change.

us workforce · ai adoption trend · frequency of use · breadth of use · probability panel

Working paper#

Garicano, L. (2026). The Vanishing Advantage of Specialization: AI, Knowledge Utilization, and the Boundary of the Firm #

Brookings Papers on Economic Activity conference draft, summary posted 23 September 2026, presented 25 September 2026. Brookings summary read at source 28 September 2026; the paper itself not read

Method: A knowledge-economics argument that AI lowers the fixed cost of acquiring specialist knowledge, tested against US employment in six occupations covering 5.9 million workers, comparing 2022 to 2025 with the pre-AI trend, with supporting evidence from ChatGPT usage data and corporate legal departments.
Finding: In four of the six occupations, lawyers, accountants, software developers and management analysts, employment in specialised outside firms fell faster between 2022 and 2025 than the pre-AI trend. 'Artificial intelligence reduces the fixed cost and hence benefits those who use this knowledge less often.' Garicano names a 'broken ladder': the less complex jobs in outside firms that trained graduates go first.
What it supports: That the work of outside specialist firms is moving in-house in the occupations most exposed to language models, on US employment data, and that the entry-level rungs in those firms are the first to go.
What it does not support: Causation: the comparison is against a trend rather than a control, and interest rates, offshoring and other causes are not ruled out in the summary read; nothing outside the United States; a conference draft, not peer-reviewed.

AI and the professions · entry-level jobs · boundary of the firm · specialisation · broken ladder · will human judgement protect lawyers from ai

Institutional survey#

High Fliers Research (2026). The Graduate Market in 2026 #

High Fliers Research, January 2026

Method: Annual survey of graduate recruitment at the UK's 100 leading graduate employers. An employer survey of a selected group, not an official statistic.
Finding: Graduate recruitment fell 5.1 per cent in 2025, after a 14.6 per cent drop in 2024 and a 6.4 per cent decrease in 2023, with a further 0.5 per cent decrease forecast for 2026. Graduate recruitment at these employers has fallen 24.5 per cent since 2022, the lowest level since 2012. Employers received on average 23 per cent more applications in the first half of the 2025-2026 season, with applications roughly doubling since 2023.
What it supports: That the graduate entry route at the UK's largest recruiters has contracted by roughly a quarter in three years while competition for what remains has roughly doubled.
What it does not support: Attribution to AI, which the survey does not establish, and it covers 100 leading employers rather than the whole labour market.

graduate recruitment · early careers · United Kingdom · do apprenticeships still work will ai replace entry level jobs

Institutional survey#

IBM Institute for Business Value, with Oxford Economics (2026). 2026 CHRO Study: Designing the Thinking Organization #

IBM Institute for Business Value, global C-suite study series, published 21 September 2026 with a press release datelined Armonk, New York. The report's own page and the press release were read at source on 23 September 2026; the figures were checked against Campus Technology and Virtualization Review (21 September) and Fair Play Talks (23 September). A commercial publication by a vendor of AI software, not peer reviewed; response rates and sampling frames are not published

Method: Two surveys fielded April to June 2026: 1,500 chief human resources officers and senior workforce strategy executives across 21 geographies and 23 industries, in organisations with annual revenue or budget from 250 million to 123.7 billion dollars and from 200 to more than 880,000 employees; and 8,800 full-time employees across 28 countries. Respondents were segmented by the study's own measure of strategic maturity, and the outcome figures are respondents' estimates of their own organisations.
Finding: 71 per cent of executives rank the ability to supervise, validate or override AI output among the most essential skills, against 38 per cent of employees; 57 per cent of executives and 49 per cent of employees put critical thinking and problem framing first; 29 per cent of employees rank judgement among the most important skills for their near future. 26 per cent of organisations clearly define work across human-led, AI-assisted and AI-executed activities, while 52 per cent of employees say their tasks changed in the past year because of AI. 36 per cent of executives say unclear accountability complicates AI deployment; 43 per cent of employees say the blame falls on them when AI fails; 80 per cent of CHROs believe AI creates invisible work and 42 per cent of employees say AI has increased unrecognised work. 60 per cent of employees worry AI is eroding their skills, three quarters of those say it already has, and 46 per cent of executives name skill erosion as a top concern, which the study distinguishes from the skills gap that the 80 per cent of organisations with a reskilling roadmap address. Where judgement is built into the workflow 62 per cent of CHROs report rising employee confidence in AI-enabled decisions; where it is not, 57 per cent report it falling. 46 per cent of organisations do not involve the CHRO when AI strategy is defined. Organisations with clearly defined workflows report 18 per cent lower risk and 20 per cent higher quality.
What it supports: That in a large, multi-country sample the people who set the expectation that AI output will be supervised and overridden and the people expected to do it disagree by 33 points on whether that is part of the job, that most organisations had not written down which work is human-led, and that a large minority of employees believe failures are blamed on them. It is the first large-sample count of the moral crumple zone as employees experience it, and the largest self-report of skill erosion to date.
What it does not support: Any capability. A gap in perceived priority is not a gap in ability to supervise, and a worry about erosion is not a measured loss; nothing in the study tests what anyone can do unaided. All figures are self-reported to a vendor whose products are the subject, the outcome figures are respondents' estimates of their own organisations rather than measured results, no override rate is reported, and the study did not verify any case of blame. 'Thinking Organization' is IBM's own label for its high-maturity segment and carries no independent standing.

human oversight · override · skill erosion · deskilling · moral crumple zone · invisible work · accountability · CHRO · work redesign · do employees know they are expected to override ai what is a moral crumple zone what is deskilling what are ai red lines

Compiled review#

Imas, A. and Schaal, J. (2026). Has AI impacted the labor market yet? Evaluating the state of the evidence #

Ghosts of Electricity (Substack), 29 September 2026. Read at source 2 October 2026

Method: A review by two economists of the published studies of AI and employment to September 2026, selecting its own literature.
Finding: Concludes that 'the impact of AI on the overall labor market has been consistently muted'. Reports the payroll dashboard figure of employment among 22 to 25 year olds in high-exposure roles 19 per cent below less-exposed peers by June 2026; Lambert and Schindler's result that when remote work is entered jointly 'the coefficient for AI exposure drops to zero or even flips positive'; Humlum and Vestergaard's 'precise null effects on earnings, recorded hours, and wages' for 25,000 Danish workers; and null results for Finnish and Norwegian youth.
What it supports: That the published studies disagree about young workers in exposed jobs and agree that no aggregate effect has been measured.
What it does not support: Which side is right. An unreviewed essay that selects its own studies, and it gives two different figures for the July 2025 vintage of the payroll study without reconciling them.

labour market · entry-level jobs · evidence review · AI and employment · is ai already causing job losses

Regulator guidance#

Information Commissioner's Office (UK) (2026). Data protection and monitoring workers #

ICO guidance, employment practices and data protection. Read at source 29 September 2026; page metadata dated 16 June 2026

Method: Statutory regulator's published guidance on how UK data protection law applies to monitoring workers, using must, should and could to separate legal requirements from good practice.
Finding: Employers must make workers aware of how and what personal information is collected during monitoring, must tell them in an accessible and easy to understand way, and apart from very exceptional circumstances where covert monitoring is justified must inform workers about any monitoring. They must identify a lawful basis, and must carry out a data protection impact assessment before processing likely to cause high risk to workers' interests. The guidance lists tracking of internet activity and keystrokes among monitoring technologies.
What it supports: The UK regulator's stated view of what an employer must do before and while monitoring workers.
What it does not support: Anything about whether a given employer complies, or the position outside the UK. It is guidance on the law and not itself the law, and it does not address AI chat logs by name.

workplace monitoring · transparency · data protection · can my employer see what i put into ai can an employer use ai to discipline or dismiss someone

Working paper#

Jadhav, R. and Danve, J. (2026). The AI Skills Shift: Mapping Skill Obsolescence, Emergence, and Transition Pathways in the LLM Era #

arXiv 2604.06906, 8 April 2026

Method: Benchmarking of four frontier models (LLaMA 3.3 70B, Mistral Large, Qwen 2.5 72B, Gemini 2.5 Flash) across 263 text-based tasks covering all 35 skills in the US Department of Labor O*NET taxonomy, 1,052 model calls. Cross-referenced against the Anthropic Economic Index. Measures models, not people. Preprint.
Finding: Introduces a Skill Automation Feasibility Index. Mathematics scores 73.2 and programming 71.8 for automation feasibility; active listening scores 42.2 and reading comprehension 45.5. All four models converge to similar skill profiles within a 3.6-point spread. Reports that 78.7 per cent of observed AI interactions are augmentation rather than automation, and a "capability-demand inversion" in which the skills most demanded in AI-exposed jobs are those the models perform least well at.
What it supports: That measured model capability and stated employer demand point in different directions, which is a useful counterweight to displacement forecasts built on exposure scores alone.
What it does not support: What happens to human capability. The authors state their index "measures LLM performance on text-based representations of skills, not full occupational execution". No humans were studied.

skills taxonomy · automation feasibility · O*NET · augmentation · cognitive debt and capability debt

Institutional survey#

Jones, J. M., Gallup (2026). AI Benefits at Work Unevenly Distributed #

Gallup, 6 October 2026: first results from the second year of the American Job Quality Study, with Jobs for the Future and The Families & Workers Fund. Read at source 9 October 2026

Method: Web and mail surveys, 26 January to 24 March 2026, of a random sample of 17,874 US working adults aged 18 to 75, drawn from the Gallup Panel and an address-based sample. The article's findings rest on 15,482 employees, of whom 7,845 have used AI in their job.
Finding: 28 per cent of employees use AI at least weekly: 37 per cent of managers against 25 per cent of individual contributors. Among employees who have used AI, 63 per cent say they do their job tasks faster; 46 per cent say they produce higher quality work and 43 per cent say they do not; 31 per cent say their employer has asked them to take on more responsibilities. 52 per cent of all employees report less influence on the adoption of new technology than they would like.
What it supports: That in a large probability sample speed is the commonly reported benefit of AI at work, that views on quality divide almost evenly, and that regular use is higher among managers.
What it does not support: That AI caused any of it: the answers are self-reports, and nothing here measures the quality of the work as others would judge it.

AI at work · productivity · work quality · managers · survey · what is the judgment class

Working paper#

Kochan, T., Kelly, E. and Minster, A. (2026). Negotiating to Work in Partnership on AI: The 2025 Agreement Between Kaiser Permanente and the Alliance of Health Care Unions #

MIT Sloan School of Management report, 2026, as summarised by Betsy Vereckey, MIT Sloan Ideas Made to Matter, 28 September 2026. The summary was read at source on 2 October 2026; the report itself was not read

Method: A case study of one negotiation by researchers at MIT Sloan and San Francisco State University.
Finding: The 2025 agreement between Kaiser Permanente and an alliance of unions representing 62,000 healthcare workers incorporates 'worker input into every stage of decision-making around how AI will be developed and used'. An AI task force 'comprises 10 senior leadership members (five from labor and five from KP) who have direct decision-making authority over technology investments'; more than 3,500 unit-based teams look for uses; peer advisers are trained to help colleagues. About 300 union and 80 company leaders met before bargaining in February 2025.
What it supports: That a large employer and its unions have written worker involvement into AI decisions, with joint authority over investment, and how they got there.
What it does not support: That it works: the summary reports no outcomes. One US healthcare employer with a long-standing labour partnership. The report was read only through MIT Sloan's own summary.

worker voice · collective bargaining · AI governance · decision rights · how should employees be involved in decisions about ai

Working paper#

Lambert, P. J. and Schindler, Y. (2026). The Broken Ladder: AI, Remote Work, and Early-Career Hiring #

Discussion paper 228/26 (hosted at rfberlin.com, dated May 2026 on its masthead, series date September 2026); also on SSRN, abstract 6787638. Not peer-reviewed. Read at source 3 October 2026 through a summary of the PDF; the SSRN page returned a rate-limit error

Method: Two microdata sources across the US, UK, Canada and Australia, 2017 to 2025: 243 million employer-employee linked records of new hires from Revelio Labs resume data and 407 million online job vacancy postings from Lightcast. Occupation-region and firm-level analysis comparing remote-work (WFH) exposure and generative-AI exposure as predictors of the junior share of hiring.
Finding: By 2025 a two-standard-deviation increase in WFH exposure predicted a fall of around 5 percentage points in the junior share of new hires and around 3 points in the share of job ads requiring limited experience. Estimated jointly, the WFH effect remains while the GenAI coefficient attenuates heavily and often becomes statistically insignificant.
What it supports: That in these data the early-career hiring fall is associated with remote-work exposure at least as strongly as with AI exposure, and that the AI association weakens when both are entered.
What it does not support: That AI has no effect. The authors say they do not interpret the evidence as ruling out strong impacts of GenAI on labour markets, that it bears only on relative junior and senior hiring up to 2025, and that a longer post-period is needed. Remote-work and AI exposure are strongly correlated across occupations, which limits what separating them can show. Unrefereed. Author affiliations as stated on the paper: Warwick and LSE; Ellison Institute of Technology, Oxford.

early careers · remote work · entry-level hiring · missing rungs · is it harder to get a first job now

Institutional survey#

Makridis, C. (Gallup) (2026). Using AI More Does Not Reassure Workers, Managers Do #

Gallup Workplace, 9 September 2026. Read at source, including the Survey Methods section, 28 September 2026

Method: Gallup Panel, probability-based recruitment; self-administered web surveys in four waves from 2023 to the first quarter of 2026 among employed US adults aged 18 and over; nearly 30,000 worker observations, about 30 per cent of respondents observed in two or more waves; weighted to Current Population Survey targets. Workers rated how likely their current job was to be eliminated within five years because of new technology, automation, robots or artificial intelligence, and how often they used AI. Comparisons within broad occupation, with demographic controls, and within the same worker over time, netting out occupation and industry shifts.
Finding: Workers using AI daily or several times a week were more than twice as likely to say their job was very likely to be eliminated within five years as those using it a few times a month or year (6.3 against 3.09 per cent in Q1 2026). Roughly 19 per cent of all workers said elimination was somewhat or very likely. Among frequent users, feeling respected, and feeling that the organisation cared about their wellbeing, went with a smaller association between use and fear (6.8 and 11.1 points respectively); among infrequent users management made little difference. Workers who feared elimination scored lower on engagement and job satisfaction and were more likely to be job-hunting.
What it supports: That in a large US panel, heavier AI use goes with more fear of displacement rather than less, including when the same worker is followed over time, and that the gap is smaller where workers feel respected and cared for.
What it does not support: That using AI causes the fear, or that management causes the reduction: the design is observational, and Gallup says the strongest within-worker comparisons rest on the smaller multi-wave group and are less precise. Expectations, not jobs actually lost. The item names technology, automation and robots as well as AI. US only.

job displacement · AI adoption · management · worker wellbeing · does using ai make you less worried about your job

Vendor research#

Massenkoff, M. and McCrory, P. (2026). Labor market impacts of AI: A new measure and early evidence #

Anthropic, 5 March 2026. Corrected 8 March 2026. Read at source 5 September 2026

Method: A new occupational exposure measure, observed exposure, built from three inputs: O*NET task lists for roughly 800 US occupations, Eloundou and colleagues' theoretical LLM task-exposure scores, and Anthropic's own Claude usage data from the Anthropic Economic Index, weighting automated over augmentative use. Employment outcomes come from the Current Population Survey, in a difference-in-differences comparison of the top exposure quartile against the 30 per cent of workers with zero measured exposure.
Finding: No systematic increase in unemployment for highly exposed workers since late 2022; the pooled difference-in-differences estimate is small and indistinguishable from zero. On hiring, the monthly job-finding rate for 22 to 25 year olds entering the most exposed occupations fell by about 14 per cent in the post-ChatGPT period against 2022. The stable comparator runs at about 2 per cent per month in less exposed occupations, and entry into the most exposed jobs falls by roughly half a percentage point. The authors' own qualification, in their words, is that this is 'just barely statistically significant'. No such decrease appears for workers over 25.
What it supports: That a slowdown in youth hiring into exposed occupations shows up in a second dataset and a second method, alongside Brynjolfsson and colleagues on ADP payroll data. Also that the harder claim, a rise in unemployment among exposed workers, does not appear: the authors estimate they could detect a differential increase of about one percentage point and see nothing.
What it does not support: The 14 per cent is routinely restated as a fall against low-exposure peers. It is not. The paper's own words are 'compared to that in 2022 in the exposed occupations', so it is a change over time within one group, which is a weaker claim than the comparison usually reported. Nor does it establish cause: the authors list three benign readings, that unhired young workers may be staying in existing jobs, taking different ones, or returning to study, and note that survey-measured job transitions are prone to mismeasurement. TWO FURTHER CAUTIONS. The treatment variable is built from Anthropic's own product telemetry and cannot be reconstructed from outside the company. And on 8 March 2026 Anthropic corrected Figure 7, the job-finding figure this number is taken from, which had reversed the labels between the top-quartile and zero-exposure groups. Anyone citing a version of this chart captured between 5 and 8 March 2026 is citing the reversed one.

early careers · jobs · exposure · hiring · will ai replace entry level jobs should juniors use ai the most quoted ai statistics checked what is the ai employment gap

Operator account#

Microsoft (2026). Audit logs for Copilot and AI applications #

Microsoft Learn, Purview documentation, page dated 26 August 2026. Read at source 29 September 2026

Method: Vendor technical documentation listing the properties written to the audit log for Copilot and AI applications.
Finding: An audit record typically contains a prompt and response pair, flags whether a message is a prompt, flags a detected jailbreak attempt, and references the files, sites and other resources Copilot or other AI applications accessed to answer the prompt.
What it supports: What kinds of event the audit log is documented to record about an AI interaction.
What it does not support: That the audit record stores the wording of a prompt: the page does not say so, and that question is answered, for the mailbox copy, by the retention documentation. Nothing about how any organisation uses the log.

audit log · workplace monitoring · can my employer see what i put into ai

Operator account#

OpenAI (2026). Data access for your managed ChatGPT account #

OpenAI Help Center article 20001067. Read at source 29 September 2026

Method: Vendor help documentation describing what an administrator of a managed ChatGPT account may access. Not a study: a statement by the operator of the system about its own product's capability.
Finding: Administrators may access content a user submits (prompts, uploaded files and outputs), conversation history, usage metadata and security settings, and can export, audit, retain, delete and opt in to share data tied to the account. Users are given a Managed ChatGPT Account Notice at set-up. A managed account and a personal account signed in together remain separate.
What it supports: What the vendor says an employer administrator is able to reach in a managed account, and that the vendor says users are notified of this at set-up.
What it does not support: How often any employer reads staff conversations, what any particular employer has switched on, or what happens on a personal account, a company device or a network. Capability as stated by the vendor, with no independent verification, and the page can change.

managed account · workplace monitoring · shadow AI · can my employer see what i put into ai who owns the capability you built at work what it would take to leave

Operator account#

OpenAI (2026). Enterprise privacy at OpenAI #

OpenAI, enterprise privacy page. Read at source 29 September 2026

Method: Vendor privacy documentation for its business products. A statement of policy and capability by the operator, not a measurement.
Finding: Within an organisation end users can view their own conversations, and workspace administrators can access an audit log of conversations and GPTs through the Enterprise Compliance API. Administrators control how long data is retained. Deleted conversations are removed from OpenAI's systems within 30 days unless legally required to be retained. By default business data is not used to train models.
What it supports: The vendor's stated position on administrator access, retention and training use for its business products.
What it does not support: Any practice at any employer, or the position for personal accounts. It describes OpenAI's own handling, and says nothing about what an employer does with data once it has been exported.

managed account · workplace monitoring · data retention · can my employer see what i put into ai who owns the capability you built at work what it would take to leave

Working paper#

Rohde, W. (AiSuNe Foundation) (2026). Short-Term Gain, Long-Term Fragility: AI Labor Substitution and the Erosion of Sustainable Capability #

SSRN abstract 6577818, written 20 April 2026, revised 27 April 2026; also arXiv 2605.27399

Method: Sole-authored conceptual synthesis, 19 pages, no new empirical data. The author's own words: "This paper is a conceptual synthesis rather than a new empirical study" and "the evidentiary strategy is selective and scoped". Preprint, not peer reviewed, no journal reference. Dates verified at the SSRN record on 28 August 2026; the arXiv abstract page could not be fetched, so the arXiv submission date remains unconfirmed and is not asserted.
Finding: Develops a mechanism of capability masking followed by capability erosion: AI output creates a persuasive appearance that organisational capability has been replaced while dependence on skilled human labour remains, supporting hiring restraint and deferred structural reform while costs accumulate. Frames the result as a stack of deferred obligations: technical debt in artifacts and systems, capability debt in the human layer that maintains them, and institutional debt in the wider structures that reproduce skill and resilience.
What it supports: That the capability-erosion argument has been reached independently, from software engineering and political economy rather than from organisational research, and that the masking-before-erosion sequence is not unique to this research's account.
What it does not support: Anything measured. It is a preprint that argues from other people's empirical work rather than presenting its own, and its societal-scale claims about fragility and concentration of power are inference rather than finding. It is also the reason no claim of first use is made for the term capability debt on this site. Checked at source on 28 August 2026: Rohde does not claim to have coined capability debt either. The paper contains no claiming language for it, and what he does claim is a mechanism, "to identify and formalize a mechanism of capability masking and capability erosion".

capability debt · deskilling · labour substitution · capability debt cognitive debt and capability debt

Institutional survey#

Saad, L. (Gallup) (2026). More U.S. Workers Fear Losing Their Jobs to Technology #

Gallup News, 15 September 2026. Read at source 28 September 2026

Method: Gallup's annual Work and Education telephone poll, 3 to 24 August 2026: 1,200 US adults, of whom 507 were employed. Margin of error plus or minus 4 points nationally and 6 points for the employed sample.
Finding: 27 per cent of US workers worried that technology could make their job obsolete, a new high, up from 20 per cent in 2025 and 13 per cent when the question was first asked in 2017. 34 per cent of workers aged 18 to 44 against 19 per cent of those 45 and over.
What it supports: That worry about technological obsolescence among US workers rose sharply between 2025 and 2026 on a consistent question.
What it does not support: Anything about AI use: the release carries no breakdown by how often workers use AI. A different instrument, mode and question from the Gallup Panel study, so the two figures cannot be combined. Small employed sample, plus or minus 6 points.

job displacement · worker attitudes · does using ai make you less worried about your job

Working paper#

Shao, Y., Zope, H., Jiang, Y., Pei, J., Nguyen, D., Brynjolfsson, E. and Yang, D. (2026). Future of Work with AI Agents: Auditing Automation and Augmentation Potential across the U.S. Workforce #

arXiv:2506.06576, v3 revised 1 February 2026

Method: Audio-enhanced mini-interviews with 1,500 US domain workers across 104 occupations, covering 844 tasks drawn from O*NET, paired with capability assessments from AI experts. Introduces the Human Agency Scale, H1 to H5, and sorts tasks into four zones by desire against capability.
Finding: Worker preferences diverge sharply from technical capability. Tasks fall into an Automation Green Light Zone, an Automation Red Light Zone where capability exists and workers do not want it used, an R&D Opportunity Zone and a Low Priority Zone. Human Agency Scale profiles vary widely by occupation, and the authors report early signals of core competencies shifting from information-focused skills towards interpersonal ones.
What it supports: That the automate-or-not framing is too coarse, and that there is a measurable, occupation-specific preferred level of human involvement which does not track what the technology can do.
What it does not support: Nothing about what happens to capability when a task is automated. It measures what workers WANT and what experts think is POSSIBLE, which are both stated positions rather than outcomes. A preprint, not peer reviewed, and US-only.

human agency scale · augmentation · automation · what stays human questions which tasks do workers not want automated what becomes more valuable as ai gets cheaper

Institutional modelling#

Waltmann, B. (2026). New Estimates of the Impact of Undergraduate Degrees on Lifetime Earnings #

Institute for Fiscal Studies, commissioned by the Department for Education

Method: Administrative linkage of school, university and tax records for the whole 2002 English GCSE cohort, tracked to age 37 with earnings simulated to 67.
Finding: Average net lifetime return to a degree is around 100,000 pounds, with very large variation by subject. Medicine and economics exceed 400,000 pounds on average. Creative arts, philosophy and languages show low or negative average returns. Around 20 per cent of women and 30 per cent of men are projected to see a negative net return.
What it supports: That subject choice carries a far larger financial spread than the decision to attend at all.
What it does not support: What any subject will return to someone choosing today. The authors explicitly decline to model structural change including AI.

· what should i tell my children to study

Institutional synthesis#

World Economic Forum in collaboration with PwC (2026). Artificial intelligence and the future of entry-level work: a framework for safeguarding and reinventing early career pathways #

World Economic Forum white paper, 22 June 2026. Publication page read at source 26 September 2026; the full PDF was not machine readable from this environment

Method: A framework developed from consultation with global stakeholders and workforce leaders across industries, organised on four dimensions: job access, job design, talent pipelines and education system alignment.
Finding: States that more than one in three young workers are employed in occupations with medium to high exposure to AI-driven task change, and proposes that organisations establish clear models for AI-enabled job redesign that use entry-level roles to keep building critical skills rather than layering AI onto existing ways of working.
What it supports: That a standing international body has framed the entry-level problem as one of job design rather than only of hiring volume, and has published a vocabulary for it that organisations are likely to encounter.
What it does not support: Any rate. The exposure figure is an exposure estimate rather than a measurement of jobs lost or roles changed, and the four dimensions are a consultative framework reporting no new primary data. The full report was read only through its publication page.

entry-level work · missing rungs · job design · early careers · missing rungs how do juniors become senior

Qualitative field study#

Ye, X. M. and Ranganathan, A. (UC Berkeley Haas) (2026). AI Doesn't Reduce Work, It Intensifies It #

Harvard Business Review, 9 February 2026, based on in-progress research described in the UC Berkeley Haas newsroom announcement of 18 February 2026. The HBR article is paywalled beyond its introduction; graded from the Haas announcement, read in full at source and consistent with the accessible HBR text

Method: Eight-month ethnographic study inside a single 200-person US technology company: real-time workplace observation, meeting attendance, informal conversations and more than 40 semi-structured interviews across functional groups. Described by the authors as in-progress research; no percentages, no control condition and no claim to generalise beyond the one firm studied.
Finding: Rather than reducing work, generative AI intensified it. Employees worked at a faster pace, took on a broader scope of tasks, and extended work into more hours of the day, often without being asked to. Three mechanisms recurred: job scope expanded, tasks bled into non-work time such as lunch and evenings, and people ran AI processes in parallel with other tasks rather than being freed by them.
What it supports: That inside at least one organisation, the time AI saves on a task is not experienced by the people doing the work as time returned; it is absorbed into more tasks, a longer day, or both, through three specific, observed mechanisms.
What it does not support: A rate, a percentage, or anything about organisations generally. One firm, one method, one still-unpublished study: strong on mechanism, silent on how common the pattern is.

work intensification · time savings · task creep · generative AI adoption · what should happen to the time ai saves

Undisclosed frame#

unitQ (2026). New study: Professionals say AI is already reshaping research hiring, and early-career roles are the most exposed #

PR Newswire, San Francisco, 3 September 2026. A press release from a vendor that sells the research platform the study was run on. Read in full at source 20 September 2026.

Method: Not stated. The release reports a study of '160 research, design and insights professionals' and gives no field dates, no recruitment source, no screening criteria, no geography, no margin of error and no base size for any subgroup figure. Its two body links go to unitQ's home page and to its product page; there is no report, topline or methodology page to follow. The release says of itself that 'the method was published alongside the finding'. The instrument and the product are the same thing: 'unitQ conducted the study using unitQ Research, an AI research platform for teams', available with a free 30-day trial.
Finding: Verbatim: 42 per cent 'said AI had reduced or slowed hiring or contributed to roles going unfilled'; 28 per cent 'said early-career researchers will feel the impact before anyone else'; 'more than half pointed to execution-heavy work as the first thing to be handed off'. Two further figures are qualified 'Among research and insights professionals', a subgroup of the 160 whose size is never given: 40 per cent 'warned of deskilling or of shallow, homogenized output' and 51 per cent 'said the people who thrive alongside AI are the ones fluent enough to direct it and to recognize when it's wrong'. An unnamed respondent is quoted calling the result an 'expertise drought'.
What it supports: That a vendor published these numbers on this date, and that its chief executive states the ladder-removal argument in public. Nothing else. It is carried in this base as a worked example of the third way a sampling frame fails, not as a measurement.
What it does not support: Any rate in any population. With no frame disclosed, the 42 and 28 per cent have no stated base at all, and the 40 and 51 per cent are percentages of an unstated subgroup that excludes the designers included in the 160. Writing '40 per cent of 160 professionals' would misstate the source. The release's own claim that a method was published is contradicted by the document containing it, which is the reason this entry exists.

sampling frame · deskilling · entry-level · vendor research · early careers · what is the illusion of competence

The seven human capabilities

The evidence behind curiosity, empathy, adaptability and the rest, including where it is thinner than the claims usually made for it.

Practitioner method#

Eberle, R. F. (1971). Scamper: Games for Imagination Development #

D.O.K. Publishers. The 1971 first edition could not be opened: the Internet Archive scan of this title is the 1996 Prufrock Press edition, and the resolvable object below is the 2023 Routledge combined edition, DOI 10.4324/9781003423560. Both checked at source 10 September 2026

Method: A classroom activity book. Eberle assembles a seven-part mnemonic, Substitute, Combine, Adapt, Modify or Magnify, Put to other uses, Eliminate, Rearrange or Reverse, from the idea-spurring checklist in Alex Osborn's Applied Imagination of 1953. No study design.
Finding: SCAMPER is a rewriting of an older checklist into a mnemonic for children, and its content belongs to Osborn before Eberle.
What it supports: That SCAMPER is an established teaching device with a traceable lineage, and that neither the letters nor the method are anybody's recent invention.
What it does not support: That running the seven prompts makes a person more creative or more curious. The 1971 edition was not read for this entry, and the date is taken from the bibliographic record rather than from the book.

curiosity · creativity · questioning · superskill curiosity

Practitioner framework#

Wack, P. (1985). Scenarios: uncharted waters ahead #

Harvard Business Review, September 1985

Method: An account by the head of Royal Dutch Shell's scenario planning group of how the method was developed and used, written after the events it describes. A participant account, not an evaluation.
Finding: Describes scenario planning as a way of changing managers' mental models rather than of predicting outcomes, and reports that Shell's scenarios had rehearsed an oil supply disruption before the 1973 shock arrived.
What it supports: Where scenario planning as a corporate practice came from, what it was intended to do, and that its stated purpose was rehearsal of mental models rather than forecast accuracy.
What it does not support: That Shell outperformed its competitors because of it. The claim comes from accounts written by the people who ran the scenarios and has never been benchmarked against any other firm, which is the usual form of this story and the reason it should not be cited as evidence of effect.

scenario planning · strategic foresight · big picture thinking · what is scenario planning models of big picture thinking superskill big picture thinking

Practitioner method#

Ohno, T. (1988). Toyota Production System: Beyond Large-Scale Production #

Productivity Press. First English printing 1988; the Japanese original, Toyota seisan hoshiki, is 1978. Publisher record and DOI confirmed at Taylor and Francis 10 September 2026, where the current reissue is dated 2019

Method: A plant director's account of a production system he built, setting out the five-times-why procedure with a worked example of a machine that stopped. No experiment, no sample, no control.
Finding: Ohno sets out asking why five times in succession as the standard procedure for reaching the cause of a fault rather than its symptom, and credits the underlying approach to Sakichi Toyoda. The procedure is a discipline for not stopping at the first answer.
What it supports: That the five-times-why has a dated, named industrial origin and belongs to Toyota rather than to whoever most recently put it on a slide.
What it does not support: That the procedure produces better causes than any other method, that five is the right number, or anything at all about curiosity as a psychological disposition. It is a shop-floor convention with sixty years of adoption and no controlled test behind it.

curiosity · root cause · questioning · superskill curiosity

Practitioner framework#

Heifetz, R. A. (1994). Leadership Without Easy Answers #

Harvard University Press, 1994; the practice is developed further in Heifetz, Grashow and Linsky, The Practice of Adaptive Leadership, 2009

Method: A theory of adaptive leadership argued from case analysis and clinical teaching practice. No controlled study.
Finding: Proposes the balcony and the dance floor: a leader has to step off the floor to see the pattern in what is happening and then return to act in it, and the discipline is the movement between the two rather than a preference for either.
What it supports: That the move between immersion and perspective is a named and long-established leadership practice, predating any AI argument.
What it does not support: That leaders who make the move perform better. The account is argued from cases rather than measured.

big picture thinking · leadership · perspective · models of big picture thinking

Peer-reviewed#

Swan, G. E. and Carmelli, D. (1996). Curiosity and mortality in aging adults: A 5-year follow-up of the Western Collaborative Group Study #

Psychology and Aging, 11(3)

Method: Prospective cohort. 1,118 men, mean age 70.6 at baseline, with an ancillary sample of 1,035 women.
Finding: Higher curiosity at baseline was associated with survival at five-year follow-up, and state curiosity remained significant after adjustment for other risk factors.
What it supports: That the association survives controlling for medical risk factors in one large cohort.
What it does not support: That curiosity extends life. Residual confounding, in particular underlying health driving both curiosity and survival, cannot be excluded.

curiosity · superskill curiosity

Peer-reviewed#

Edmondson, A. (1999). Psychological Safety and Learning Behavior in Work Teams #

Administrative Science Quarterly, 44(2)

Method: Multi-method field study of 51 work teams in a single manufacturing company.
Finding: Team psychological safety predicted learning behaviour, which in turn mediated the relationship with team performance.
What it supports: That the route from safety to performance runs through learning behaviour rather than directly.
What it does not support: Causation, or generalisation beyond one manufacturing firm. It is a correlational field study.

change readiness · psychological safety · superskill change readiness

Practitioner framework#

Meadows, D. H. (1999). Leverage points: places to intervene in a system #

The Sustainability Institute, 1999, developing an essay first published in Whole Earth Review, 1997

Method: An argued ranking of twelve kinds of intervention in a system, from parameters at the weakest to paradigms and the ability to transcend paradigms at the strongest. Developed from systems modelling practice; no trial and no dataset.
Finding: Argues that interventions people reach for first, adjusting numbers and parameters, are the least powerful, and that the most powerful are changes to goals, to the paradigm from which the system arises, and to the capacity to hold paradigms lightly.
What it supports: That there is a published, widely used ordering of where in a system an intervention does most work, and that it is the canonical answer to the question of which level to act at.
What it does not support: The ordering itself, which is argued rather than measured. Meadows states in the essay that the list is not to be taken as precise and that leverage points are frequently counter-intuitive and often pushed in the wrong direction.

systems thinking · leverage points · big picture thinking · what are leverage points models of big picture thinking

Peer-reviewed#

Pulakos, E. D., Arad, S., Donovan, M. A. and Plamondon, K. E. (2000). Adaptability in the Workplace: Development of a Taxonomy of Adaptive Performance #

Journal of Applied Psychology, 85(4)

Method: Content analysis of over 1,000 critical incidents drawn from 21 jobs, followed by scale validation.
Finding: Eight dimensions of adaptive performance, including handling emergencies, managing stress, solving problems creatively and dealing with uncertain situations.
What it supports: That adaptability is decomposable into observable behaviours rather than being a single trait.
What it does not support: That employers reward these dimensions, or that they transfer across every occupation. The taxonomy was derived, not tested against outcomes.

change readiness · adaptive performance · superskill change readiness

Compiled review#

Simons, T. (2002). The High Cost of Lost Trust #

Harvard Business Review, September 2002

Method: Survey of more than 6,500 employees at 76 Holiday Inn hotels in the United States and Canada, matched to hotel financial records.
Finding: A one-eighth point improvement in managers' behavioural integrity rating was associated with a 2.5 per cent increase in profitability, roughly 250,000 dollars a year for an average hotel. No other measured aspect of manager behaviour had as large an effect on profits.
What it supports: That whether managers are seen to mean what they say tracks measurable financial outcomes.
What it does not support: Causation. It is a correlational study within one hotel chain, published in a practitioner magazine rather than a peer-reviewed journal.

principled innovation · superskill principled innovation

Peer-reviewed#

Orlitzky, M., Schmidt, F. L. and Rynes, S. L. (2003). Corporate Social and Financial Performance: A Meta-Analysis #

Organization Studies, 24(3)

Method: Meta-analysis of 52 studies, 33,878 observations.
Finding: A positive association between corporate social performance and financial performance, which the authors describe as bidirectional.
What it supports: That principled conduct and financial results are not in general tension.
What it does not support: Causation in either direction, and the relationship is not uniform across how social performance is operationalised.

principled innovation · superskill principled innovation

Practitioner framework#

Snowden, D. J. and Boone, M. E. (2007). A leader's framework for decision making #

Harvard Business Review, November 2007

Method: A sensemaking framework argued in an edited venue, sorting situations into simple, complicated, complex and chaotic domains, with a different mode of action appropriate to each. No dataset.
Finding: Argues that the error leaders make is applying the practice of one domain in another, most often treating a complex situation as a complicated one and demanding an analysis that the situation cannot support.
What it supports: That there is an established framework whose whole purpose is telling a decision maker which kind of situation they are in before they choose how to act, which is the nearest existing relative of any altitude-selection device.
What it does not support: That sorting a situation correctly improves outcomes. Cynefin is a sensemaking aid, has been revised repeatedly by its author, and reports no measurement.

sensemaking · complexity · big picture thinking · models of big picture thinking

Peer-reviewed#

Maddux, W. W. and Galinsky, A. D. (2009). Cultural Borders and Mental Barriers: The Relationship Between Living Abroad and Creativity #

Journal of Personality and Social Psychology, 96(5)

Method: Five studies with MBA and undergraduate participants, combining correlational designs with causal priming experiments.
Finding: Time spent living abroad predicted success on creative-insight tasks and creative negotiation outcomes. Time spent travelling abroad did not.
What it supports: That adaptation to living in another culture, rather than exposure to it, is what relates to creativity.
What it does not support: A specific effect size for the general population. The samples are business and undergraduate students.

global adaptability · superskill global adaptability

Peer-reviewed#

Harrison, S. H., Sluss, D. M. and Ashforth, B. E. (2011). Curiosity adapted the cat: The role of trait curiosity in newcomer adaptation #

Journal of Applied Psychology, 96(1)

Method: Longitudinal field study of 123 newcomers across 12 call-centre organisations.
Finding: Specific curiosity predicted information-seeking from colleagues, which in turn was associated with more creative handling of customer problems.
What it supports: That curiosity operates through a behavioural mechanism, asking, rather than as a disposition on its own.
What it does not support: That curiosity can be trained into people, or that the effect holds outside newcomer adaptation.

curiosity · superskill curiosity

Practitioner framework#

Kanter, R. M. (2011). Managing yourself: zoom in, zoom out #

Harvard Business Review, March 2011

Method: An argued framework in an edited management venue, drawing on the author's observation of leaders. No new data.
Finding: Proposes that leaders operate along a continuum between a close-in view, where relationships and particulars dominate, and a distant view, where patterns and principles dominate, and that the failure is getting stuck at either end rather than preferring one.
What it supports: That the zoom metaphor for levels of attention is long established in the management literature, and that the failure it names is fixity rather than choice of level.
What it does not support: That moving between the two improves decisions. No measurement is reported and no instrument is offered.

big picture thinking · altitude · attention · models of big picture thinking

Peer-reviewed#

Konrath, S. H., O'Brien, E. H. and Hsing, C. (2011). Changes in Dispositional Empathy in American College Students Over Time: A Meta-Analysis #

Personality and Social Psychology Review, 15(2)

Method: Cross-temporal meta-analysis of 72 samples of American college students, total N 13,737, from 1979 to 2009.
Finding: Empathic Concern fell by 48 per cent and Perspective Taking by 34 per cent across the period, with most of the decline after 2000.
What it supports: That self-reported dispositional empathy declined measurably in this population over three decades.
What it does not support: A cause. The authors speculate about individualism and media but test no mechanism. It is also American college students only.

empathy · superskill empathy

Peer-reviewed#

Sadri, G., Weber, T. J. and Gentry, W. A. (2011). Empathic emotion and leadership performance: An empirical analysis across 38 countries #

The Leadership Quarterly, 22(5)

Method: 360-degree ratings of 6,731 mid to upper-level managers across 38 countries. Subordinates rated empathy, superiors rated performance.
Finding: Managers rated as more empathic by subordinates received higher performance ratings from their own superiors, with the effect moderated by national power distance.
What it supports: That the association holds at scale and across many national contexts.
What it does not support: That empathy training improves performance. The design is cross-sectional and correlational.

empathy · superskill empathy

Peer-reviewed#

von Stumm, S., Hell, B. and Chamorro-Premuzic, T. (2011). The Hungry Mind: Intellectual Curiosity Is the Third Pillar of Academic Performance #

Perspectives on Psychological Science, 6(6)

Method: Path-model synthesis of prior meta-analytic correlation matrices. Component samples range from 608 to 28,471.
Finding: Intellectual curiosity predicts academic performance independently of intelligence and effort, which the authors describe as a third pillar.
What it supports: That curiosity carries predictive weight that conscientiousness and ability do not account for.
What it does not support: Causation. It is a correlational synthesis. The widely quoted figure of roughly 50,000 students comes from the accompanying press release rather than from the paper itself.

curiosity · superskill curiosity

Peer-reviewed#

Huang, J. L., Ryan, A. M., Zabel, K. L. and Palmer, A. (2014). Personality and Adaptive Performance at Work: A Meta-Analytic Investigation #

Journal of Applied Psychology, 99(1)

Method: Meta-analysis of 71 independent samples, total N 7,535.
Finding: Emotional stability and ambition predict adaptive performance, and the pattern differs from predictors of routine task performance.
What it supports: That individual differences carry predictive weight for adaptive performance specifically.
What it does not support: That adaptive performance is a distinct construct. This paper assumes the construct from earlier taxonomy work rather than establishing it.

change readiness · adaptive performance · superskill change readiness

Peer-reviewed#

Mehta, R., Zhu, R. and Meyers-Levy, J. (2014). When Does a Higher Construal Level Increase or Decrease Indulgence? Resolving the Myopia versus Hyperopia Puzzle #

Journal of Consumer Research, 41(2)

Method: Multi-study laboratory experiments.
Finding: Where the self is focal, a higher construal level increases indulgence rather than reducing it, reversing the effect the earlier literature predicted.
What it supports: That the benefit of stepping back is conditional, and the condition is identifiable.
What it does not support: That distant-future thinking is generally counterproductive. The effect is moderated, not reversed outright.

big picture thinking · superskill big picture thinking

Institutional survey#

United States Environmental Protection Agency (2015). Notice of Violation, Volkswagen Group #

US EPA enforcement record

Method: Regulatory enforcement notice.
Finding: Affected 2.0-litre vehicles emitted nitrogen oxides at up to 40 times the standard in normal driving while appearing compliant in laboratory testing.
What it supports: That the defeat device produced a measured gap between test and road conditions of that magnitude.
What it does not support: That the same multiple applies across the range. The separate 3.0-litre notice cited up to nine times.

principled innovation · superskill principled innovation

Institutional modelling#

Barton, D., Manyika, J., Koller, T., Palter, R., Godsall, J. and Zoffer, J. (2017). Measuring the Economic Impact of Short-Termism #

McKinsey Global Institute

Method: Corporate Horizon Index applied to 615 large and mid-cap United States public companies, 2001 to 2014.
Finding: Revenue of long-term firms grew cumulatively 47 per cent more than other firms, earnings 36 per cent more, and their share prices recovered faster after the financial crisis.
What it supports: That a measurable long-term orientation tracks with stronger cumulative growth in this sample.
What it does not support: Causation, or application outside large United States listed companies. The index is a constructed measure, not an observed policy.

principled innovation · superskill principled innovation

Practitioner framework#

Woodward, I. C., with Charan, R. (2017). The three altitudes of leadership #

INSEAD Knowledge, 27 October 2017. Read at source 26 September 2026

Method: A framework developed from decades of observing chief executives and leaders, set out in an institutional knowledge publication. No study design, no sample and no dataset are reported.
Finding: Proposes that leaders think at three altitudes, 50,000 feet for the strategic panorama, 50 feet for planning and execution, and 5 feet for self-awareness, and that effective leaders move between all three. Names a pathology, altitude sickness, in which a leader is trapped at one altitude, and states that around 70 per cent of senior executives display it, with the largest group stuck at 50 feet.
What it supports: That the altitude metaphor for levels of leadership thinking is established, named and institutionally published, and that the failure it describes is an inability to move between levels.
What it does not support: The 70 per cent figure. No study, sample or measurement is cited for it anywhere in the article, and it rests on the authors' observation. Nothing here establishes that moving between altitudes improves any outcome.

big picture thinking · altitude · leadership · the altitude lens models of big picture thinking what is altitude lock

Peer-reviewed#

Bratsberg, B. and Rogeberg, O. (2018). Flynn effect and its reversal are both environmentally caused #

Proceedings of the National Academy of Sciences, 115(26), 6674-6678. Published online 11 June 2018, in issue 26 June 2018. Edited by Richard E. Nisbett and approved 14 May 2018. PMID 29891660. Full text read at source on 23 September 2026.

Method: Analysis of Norwegian administrative registers linked to military conscription cognitive testing at age 18 to 19, birth cohorts 1962 to 1991, restricted to native-born men with two native-born parents present in Norway on their eighteenth birthday. Overall sample 817,611, of whom 736,808 (90.1 per cent) had a valid ability score. Family fixed-effects models on scored brothers (n=355,438 families with at least two scored brothers), plus a Bayesian model correcting for selection into scoring, since conscription test coverage fell from 93 per cent for the 1980 cohort to 83 per cent for 1991 and the missing were disproportionately the lower-scoring.
Finding: Mean IQ rose from 99.5 for the 1962 birth cohort to 102.3 for 1975, then fell to 99.4 by 1989 and 99.7 by 1991. The rise, the turning point and the fall are all recoverable from WITHIN-family variation, comparing brothers. Selection-corrected estimates give 0.20 IQ points per year of increase within families for 1962 to 1975 against 0.18 across families, and 0.33 points per year of decline within families for 1975 to 1991 against 0.34 across families.
What it supports: That both the rise and the reversal of the Flynn effect in this population are environmental and operate within families, which rules out dysgenic fertility and immigration composition, the two explanations most often offered for falling scores. It also fixes the date: the turning point is the 1975 birth cohort, tested around 1993, and the last falling cohort was tested around 2009.
What it does not support: Which environmental factor is responsible. The authors state they cannot identify the causal structure, and list changing media exposure, educational change, and nutrition or health among the hypotheses their design leaves standing. It is one country, men only, one conscription test, and cohorts born no later than 1991, so it says nothing whatever about any technology introduced since. IT IS NOT EVIDENCE THAT AI LOWERS OR RAISES INTELLIGENCE, in either direction; the whole series predates generative AI. Its use in this research is to date the decline people attribute to AI, not to explain it.

Flynn effect · cognitive ability · IQ · deskilling · is ai making us stupid

Compiled review#

Gino, F. (2018). The Business Case for Curiosity #

Harvard Business Review, September to October 2018

Method: Survey of more than 3,000 employees across a range of firms, reported in a practitioner magazine.
Finding: Around 92 per cent said curious people bring new ideas to their teams, while about 24 per cent reported feeling curious in their jobs regularly.
What it supports: That the stated value of curiosity and the felt experience of it diverge sharply inside organisations.
What it does not support: Any causal link between curiosity and a business outcome. Self-report, not peer-reviewed, and the sample is not nationally representative.

curiosity · superskill curiosity

Peer-reviewed#

Howick, J., Moscrop, A., Mebius, A. et al. (2018). Effects of empathic and positive communication in healthcare consultations: a systematic review and meta-analysis #

Journal of the Royal Society of Medicine, 111(7)

Method: Systematic review and meta-analysis of 28 randomised trials, 6,017 patients in total. Seven of the 28 tested empathic communication specifically; the remainder tested positive-expectation messaging.
Finding: The seven empathy-specific trials showed a small improvement in pain, anxiety and satisfaction, SMD -0.18, 95 per cent CI -0.32 to -0.03.
What it supports: That empathic communication has a measurable effect on patient-reported outcomes.
What it does not support: A large clinical benefit. The authors describe the effect as small, and only a quarter of the pooled trials tested empathy rather than positive framing.

empathy · superskill empathy

Institutional survey#

Lorenzo, R., Voigt, N., Tsusaka, M., Krentz, M. and Abouzahr, K. (2018). How Diverse Leadership Teams Boost Innovation #

Boston Consulting Group

Method: Survey of more than 1,700 companies across eight countries.
Finding: Companies with above-average management diversity reported innovation revenue 19 percentage points higher than below-average companies, 45 per cent of total revenue against 26 per cent.
What it supports: That reported diversity and reported innovation revenue move together at scale.
What it does not support: Causation. Nor is this audited financial data. Note that 19 percentage points is not the same as 19 per cent higher, a distinction frequently lost in citation.

principled innovation · superskill principled innovation

Peer-reviewed#

Stillman, P. E., Fujita, K., Sheldon, O. and Trope, Y. (2018). From 'Me' to 'We': The Role of Construal Level in Promoting Maximized Joint Outcomes #

Organizational Behavior and Human Decision Processes, 147

Method: Four laboratory and online experiments, pooled N approximately 691.
Finding: Prompting a higher level of construal led participants to choose options that maximised joint outcomes, including where doing so reduced their own payoff.
What it supports: That the level at which a problem is framed changes whether people optimise for themselves or for the whole.
What it does not support: Field behaviour. These are economic games with student and online samples.

big picture thinking · superskill big picture thinking

Peer-reviewed#

Schlaegel, C., Richter, N. F. and Taras, V. (2021). Cultural intelligence and work-related outcomes: A meta-analytic examination of joint effects and incremental predictive validity #

Journal of World Business, 56(4)

Method: Meta-analysis of 70 studies providing 80 independent samples, total N 18,359.
Finding: Cultural intelligence is moderately associated with work-related outcomes, with a reliability-corrected average effect of about .39.
What it supports: That the association is consistent across a large body of studies.
What it does not support: Causation, and it does not establish any single dimension as the strongest predictor. The paper is about joint effects across all four dimensions.

global adaptability · cultural intelligence · superskill global adaptability

Peer-reviewed#

De Freitas, J., Uguralp, A. K., Oguz-Uguralp, Z., Paul, L. A., Tenenbaum, J. and Ullman, T. D. (2023). Self-orienting in human and machine learning #

Nature Human Behaviour, 7

Method: Behavioural experiments with 124 human players across custom games, benchmarked against deep reinforcement learning agents.
Finding: Humans were near optimal at working out their own position and capabilities after conditions were altered. The reinforcement learning baselines were far from optimal at the same task.
What it supports: That rapid self-orientation after an unexpected change is currently a human advantage over the algorithms tested.
What it does not support: General workplace adaptability. These are simple custom games, the sample is modest, and the comparison is against specific algorithms rather than all AI approaches.

change readiness · superskill change readiness

Compiled review#

Lightcast (2025). Beyond the Buzz: Developing the AI Skills Employers Actually Need #

Lightcast, July 2025

Method: Analysis of over 1.3 billion job postings collected by Lightcast from more than 160,000 online sources, comparing postings that request AI skills with those that do not, with a focus on 2024 and growth since 2022. Five career areas are examined: marketing and PR, HR, finance, science and research, and education and training.
Finding: In 2024, 51% of job postings requesting AI skills were outside IT and computer science occupations. Postings mentioning AI skills advertise salaries 28% higher than those that do not, roughly $18,000 a year. Generative AI mentions in non-tech roles grew 800% since 2022; AI skill demand grew 66% in HR postings and 40% in finance, and 8% of marketing and PR postings request AI skills with 50% annual growth. The highest wage premiums appear in non-technical occupations such as architects and lawyers.
What it supports: That employer demand for AI skills has spread beyond technical occupations and is paired with higher advertised pay, especially where AI is combined with domain expertise.
What it does not support: Advertised salary premiums are not controlled for seniority, location or employer, so the 28% is not a causal return to AI skill. Postings are a demand signal, not hiring or performance. Lightcast sells labour market data and skills taxonomies.

job postings · ai skills demand · wage premium · non-tech occupations · domain expertise

Institutional synthesis#

World Economic Forum (2025). New Economy Skills: Unlocking the Human Advantage #

World Economic Forum white paper, 3 December 2025. Publication page read at source 26 September 2026; the report PDF is served from a host that refuses automated fetching and was not read

Method: Synthesis drawing on data from education industry and workforce technology providers, a review of existing research, and expert consultations. Proposes a framework for assessing, developing and credentialling human skills.
Finding: Argues that human-centric skills, the capabilities that let individuals, organisations and societies adapt to change and lead transformation, are central to business and economic performance, and that the open question is no longer whether they matter but how they are developed, assessed and credentialled.
What it supports: That the assessment and credentialling of human capability is now an active question for a standing international body, and that the gap it names is the same one this research names from the organisational side.
What it does not support: That any assessment method works. No instrument is validated in the publication page, no measurement is reported, and the detail of the proposed framework was not read because the report host refuses automated access.

human skills · assessment · credentialling · capability · how do you assess capability rather than output what is a capability audit superskills and the wef future of jobs report

Working paper#

Autor, D., Rodchenko, T., Martin, J., Iscenko, Z., Strand, S., Pearl, D. and Ferere, M. (2026). Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting #

NBER Working Paper 35720, issued September 2026, with the authors' account on the Google Research blog of 7 October 2026. Abstract page and blog read at source 9 October 2026; the full paper was not read

Method: Pre-registered randomised controlled trial (AEA RCT Registry AEARCTR-0015823). 133 practising patent lawyers at eleven US intellectual property firms over three months; two-thirds at each firm were given a then-unreleased Google AI patent drafting assistant and the rest some AI training and no tool. All work was scored by blinded expert patent attorneys. At three months every lawyer redlined a patent application without AI. Juniors are lawyers with under seven years of experience.
Finding: With AI, quality on benchmark drafting tasks rose by 0.34 SD at 10 days (p = 0.03) and 0.38 SD at 90 days (p = 0.01), 'with larger gains among junior lawyers'. On the unassisted task 'Treated lawyers outperformed controls by 0.32 SD (p = 0.04), but this advantage was concentrated entirely among senior lawyers (0.45 SD, p = 0.02).' 'Junior lawyers showed no average gain; their scores instead bifurcated, with sharply fewer mediocre scores offset by more poor and more good ones.' 'The largest gains from AI thus accrued to the lawyers who retained the least.'
What it supports: That in one profession, scored by blinded graders, the gain in work delivered with AI and the gain in unaided judgement came apart, and that the unaided gain went to the lawyers with more experience.
What it does not support: That AI harms juniors: no group did worse on average. Anything beyond 133 lawyers at top-tier firms over three months; the sizes of the junior and senior groups are not on the pages read. Google paid the direct costs, five of the seven authors are Google employees and a sixth is a paid contractor, the tool was Google's own and the firms do business with Google. Not peer-reviewed.

deskilling · synthetic seniority · junior staff · unaided performance · professional judgement · does ai make juniors better or only look better capability debt

Institutional survey#

Pearson, with Amazon Web Services (2026). AI Readiness: Building the Bridge from Higher Education to Work #

Pearson, April 2026

Method: Survey of 2,711 undergraduate students, university educators and administrators, and employers from small, mid-sized and large firms in Brazil, Malaysia, Saudi Arabia, the United States, the United Kingdom and Vietnam. Fieldwork dates are not stated; survey data is combined with secondary sources from the World Economic Forum, Stanford HAI and UNESCO.
Finding: 67% of respondents describe the pace of AI-driven change as extremely or very fast, and 24% believe universities are keeping pace. 78% of higher education leaders believe their graduates meet employer expectations. Employers rank communication and collaboration (50%) and adaptability (45%) as their highest priorities for graduates, and 42% of employers cite lack of hands-on experience with workplace AI tools as a major barrier. 63% of all respondents view a degree as more essential than five years ago.
What it supports: That across six countries employers and universities report different views of graduate readiness, and that employers name human skills ahead of AI tool skills.
What it does not support: Six countries with an unstated sampling frame do not generalise globally, and all measures are perceptions. Pearson and AWS sell education and AI services to the institutions surveyed. It is not a measure of actual graduate performance.

graduate readiness · higher education · employer expectations · adaptability · ai skills gap

Compiled review#

Storey, M.-A. (2026). From Technical Debt to Cognitive and Intent Debt: Rethinking Software Health in the Age of AI #

ACM Queue, preprint at arXiv 2603.22106, March 2026

Method: Conceptual synthesis by a Canada Research Chair at the University of Victoria, drawing on Naur (1985) on programming as theory building, Cunningham (1993) on technical debt, and the author's own empirical work with Starr on developer confusion. Illustrated with a single teaching anecdote rather than a study. Proposes a framework; presents no new data.
Finding: Proposes a triple debt model: technical debt lives in code, cognitive debt lives in people as the erosion of shared understanding across a team, and intent debt lives in artefacts as the absence of captured rationale, goals and constraints. Argues generative AI may reduce technical debt while accelerating the other two, because code can now be produced faster than a team can build the understanding needed to change it safely.
What it supports: That the debt metaphor has been extended into a structured framework by a serious researcher, and that cognitive debt now carries a team-level meaning distinct from the individual-level one in Kosmyna et al. The distinction is the author's own and she states it explicitly.
What it does not support: Anything empirical about prevalence or magnitude. It is a framework paper with an anecdote, and it says so. Its reference list also mis-cites Kosmyna et al. as a 2024 CHI workshop paper, which does not appear on the MIT Media Lab's own publications list for that author; the citation is not relied on here.

cognitive debt · intent debt · technical debt · shared understanding · cognitive debt and capability debt will ai replace programmers

Working paper#

Yi Duan, Ying Liu, Zirui Tang, Haodong Chen, Jun Zhou, Yumou Liu and 27 others (2026). The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement #

arXiv 2609.11873, submitted 10 September 2026

Method: A 33-author preprint proposing a five-level roadmap of self-improvement autonomy (L1 improvement execution, L2 improvement strategy, L3 experience acquisition, L4 environment adaptation, L5 recursive inheritance), a review of industry systems against it, and a Headroom-Closed Index that normalises benchmark scores between an entry-year frontier (0) and perfect performance (100). Not peer-reviewed at the time of entry.
Finding: By the paper's own index, 2026 systems have closed most of the headroom on advanced mathematics (HCI 86.4) and graduate science (85.8) but far less on software engineering (52.6) and tool-using agents (39.9); interactive capabilities lag bounded tasks. The paper places current industrial systems at levels L1 to L4 and describes genuine recursive meta-improvement (L5) as largely unrealised. In the systems it reviews, 'experts retain control over consequential corrections and deployment'.
What it supports: That a large group of researchers has written down what recursive self-improvement would mean, level by level, and by their own measure finds the field short of the last level. The index is a useful vocabulary for saying how far a capability has closed rather than whether it has arrived.
What it does not support: That an AI has built its successor; the title describes an aim, and the paper's own placement of the field is L1 to L4 with humans keeping the consequential decisions. The Headroom-Closed Index is the authors' construction and its entry-year frontiers are choices. The paper is a roadmap by people who want the destination, which is a reason to read it and a reason to read it carefully.

recursive self-improvement · autonomy levels · headroom-closed index · self-improving ai · agents · benchmarks · neither ai hype nor doom what is meaningful human oversight

The institutional record

What the most-quoted reports say, how they were made, and what they can carry.

Institutional survey#

United States Federal Aviation Regulations (1974). 14 CFR 121.441, Proficiency checks #

Electronic Code of Federal Regulations

Method: Binding regulation.
Finding: A pilot in command must pass a proficiency check every 12 calendar months, and within every 6 calendar months either a proficiency check or an approved simulator course.
What it supports: That one profession has made recurrent tested practice a legal condition of continuing to work.
What it does not support: That the intervals are calibrated to measured decay curves. The regulation sets a minimum, not an evidence-based optimum.

· what professions can learn from aviation

Institutional survey#

United States Federal Aviation Regulations (1981). 14 CFR 121.542, Flight crewmember duties, the sterile cockpit rule #

Electronic Code of Federal Regulations

Method: Binding regulation, Docket 20661, 46 FR 5502, 19 January 1981.
Finding: No crew member may perform any duty during a critical phase of flight other than those required for safe operation. Critical phases include taxi, take-off, landing and all operations below 10,000 feet except cruise.
What it supports: That protected attention can be written into law as a condition rather than left to individual discipline.
What it does not support: Anything about automation or skill decay directly. It is a rule about distraction.

· what professions can learn from aviation

Definitional instrument#

Parliament of the United Kingdom (1990). Computer Misuse Act 1990, section 1: Unauthorised access to computer material #

Statute as amended, read on legislation.gov.uk on 25 September 2026

Method: Binding legislative text. Section 1 defines the basic offence of unauthorised access.
Finding: A person is guilty of the offence if they cause a computer to perform any function with intent to secure access to a program or data, the access they intend to secure is unauthorised, and they know at the time that it is unauthorised. Intent and knowledge are elements of the offence.
What it supports: That the UK's basic computer misuse offence, like the US Computer Fraud and Abuse Act, turns on a person's intent and knowledge, so the question of who is criminally answerable when an autonomous agent gains access it was not directed to gain is as open in UK law as the Associated Press reported it to be in US law.
What it does not support: How a court would treat a company whose agent gained unauthorised access; no reported case was found, and the section says nothing about civil liability, regulatory duties or the organisation's own accountability arrangements.

Computer Misuse Act · unauthorised access · intent · AI agents · accountability · what the rogue ai agent incidents mean for your organisation can a company blame its ai agent

Peer-reviewed#

Keil, M. (1995). Pulling the Plug: Software Project Management and the Problem of Project Escalation #

MIS Quarterly, 19(4), 421-447

Method: Longitudinal exploratory single-case study of one IT project inside a large computer manufacturer, pseudonymised as CompuSys. 111 interviews across eight job functions, 19 observed meetings, more than 350 collected documents.
Finding: CONFIG, an expert system built to help sales representatives produce error-free configurations before quoting, ran for over a decade and was terminated at the end of 1992 after, in the author's words, tens of millions of dollars. Successive business cases put net present value at 43.9 million dollars in 1982, 55.7 million in 1985 and at least 41.1 million in 1987. Keil concludes escalation is promoted by a combination of project, psychological, social and organisational factors rather than by any one of them.
What it supports: That the canonical study of a technology programme nobody could stop is a study of an artificial intelligence programme. The de-escalation literature is not being applied to AI by analogy; it began there and the field forgot.
What it does not support: Any prevalence. N is one, the organisation is pseudonymised, and there is no comparison group. An Academia.edu machine-generated summary of this paper asserts the project absorbed 250 million dollars; the article says tens of millions, and the 250 figure in it is 250 billion, being 1994 total US IT applications spending.

escalation of commitment · project failure · expert systems · sunk cost · which ai investments should we stop

Peer-reviewed#

Keil, M., Mann, J. and Rai, A. (2000). Why Software Projects Escalate: An Empirical Analysis and Test of Four Theoretical Models #

MIS Quarterly, 24(4), 631-664. PAGINATION DISPUTED, checked 22 September 2026 and left as it stands because it is the majority reading. Three repositories give three answers: Georgia State's ScholarWorks, the first author's own institution, recommends '24 (5), 631-664'; the AIS Electronic Library files it under volume 24, issue 4; the publisher's own platform at misq.umn.edu carries it at 24(4) with the page range 631-647 in its article metadata, and returns an empty body to both fetchers so that range could not be confirmed by reading. Cite the volume, issue and year with confidence and treat the closing page number as unsettled.

Method: Survey of information systems audit and control professionals, designed to gather data on projects that did not escalate as well as those that did, testing four theories: self-justification, prospect, agency and approach-avoidance.
Finding: The authors state that between 30 and 40 per cent of all IS projects exhibit some degree of escalation. The completion effect derived from approach-avoidance theory gave the best classification, correctly classifying over 70 per cent of both escalated and non-escalated projects.
What it supports: That escalation is common enough to be a base condition rather than an exception, and that the best-supported explanation is the pull of finishing rather than the psychology of self-justification alone.
What it does not support: Anything about AI, and nothing about whether escalated projects should have been stopped: it shows their outcomes were worse. Some degree of escalation is a soft threshold, and the respondents are auditors reporting retrospectively rather than a random sample of projects. The published sample size sits in the full text, which could not be opened during the 4 September 2026 build; the prevalence figure is quoted from the abstract and pages citing it say so.

escalation of commitment · project failure · prevalence · governance · which ai investments should we stop

Peer-reviewed#

Montealegre, R. and Keil, M. (2000). De-escalating Information Technology Projects: Lessons from the Denver International Airport #

MIS Quarterly, 24(3), 417-447

Method: Longitudinal qualitative case study of the automated baggage handling system at Denver International Airport, used to induce a process model of de-escalation.
Finding: De-escalation runs as a four-phase process: problem recognition, re-examination of prior course of action, search for alternative course of action, and implementing an exit strategy. The authors note that while escalation is well researched, there has been comparatively little research on the process of breaking the cycle.
What it supports: That climbing back down has a describable structure, and that the hard step is the second one, because re-examining the prior course of action requires the person who chose it to permit the question.
What it does not support: How often de-escalation is attempted or succeeds. N is one, the model is inductive rather than tested, and the case is a physical baggage system in the 1990s. The authors describe four phases, not stages.

de-escalation · project termination · exit strategy · governance · which ai investments should we stop

Peer-reviewed#

Hughes, M. (2011). Do 70 Per Cent of All Organizational Change Initiatives Really Fail?#

Journal of Change Management, 11(4), 451-464. DOI 10.1080/14697017.2011.630506

Method: Critical review of five separate published instances of the 70 per cent organisational change failure rate, tracing each to its stated source.
Finding: In the author's words: 'whilst the existence of a popular narrative of 70 percent organizational change failure is acknowledged, there is no valid and reliable empirical evidence to support such a narrative.'
What it supports: That one of the most repeated statistics in management has no traceable empirical basis, and that this was established in a peer-reviewed journal fifteen years ago and ignored.
What it does not support: That change programmes usually succeed. The finding is about the absence of evidence for a specific number, not about the true rate, which remains unmeasured.

· the most quoted ai statistics checked

Institutional survey#

The Joint Commission (2013). Sentinel Event Alert 50: Medical device alarm safety in hospitals #

The Joint Commission, Issue 50, 8 April 2013

Method: Analysis of the Joint Commission's own Sentinel Event database, supplemented by FDA MAUDE reports and ECRI hazard rankings. Voluntary reporting; no sampling frame.
Finding: 98 alarm-related events between January 2009 and June 2012, of which 80 resulted in death, 13 in permanent loss of function and five in unexpected additional care or extended stay. 94 of the events occurred in hospitals. Contributing factors recorded as alarm signals inappropriately turned off (36), absent or inadequate alarm system (30), alarm signals not audible in all areas (25) and improper alarm settings (21). Reports that clinicians may turn the volume down, turn the alarm off, or set it outside safe limits in response to the volume of signals. Cites 566 alarm-related patient deaths in FDA MAUDE between January 2005 and June 2010. Led to National Patient Safety Goal NPSG.06.01.01, phased from 1 July 2014.
What it supports: That a regulator has documented, with named contributing factors, people disabling a safety control because it fired too often, and has had to legislate who holds the authority to change or silence it.
What it does not support: The size of the problem. The Commission's own footnote states that reporting is voluntary, represents only a small proportion of actual events, and that no conclusions should be drawn about relative frequency or trend. The widely quoted estimate that 85 to 99 per cent of alarm signals do not require clinical intervention is quoted BY the Commission from AAMI Horizons, Spring 2011, which is not a Joint Commission measurement and was not read for this entry.

alarm fatigue · human oversight · safety controls · regulation · how do you design a stop button people will use what is alarm fatigue

Institutional modelling#

Arntz, M., Gregory, T. and Zierahn, U. (2016). The Risk of Automation for Jobs in OECD Countries: A Comparative Analysis #

OECD Social, Employment and Migration Working Papers No. 189

Method: Task-based re-estimation across 21 OECD countries using PIAAC survey data, accounting for the heterogeneity of tasks WITHIN occupations rather than treating whole occupations as automatable.
Finding: 9 per cent of jobs automatable on average across 21 countries, ranging from 6 per cent in Korea to 12 per cent in Austria. The authors state the occupation-based approach 'might lead to an overestimation of job automatibility, as occupations labelled as high-risk occupations often still contain a substantial share of tasks that are hard to automate'.
What it supports: That the headline automation figure is highly sensitive to whether you model occupations or tasks, and that the difference is roughly fivefold on the same question.
What it does not support: That 9 per cent is correct and 47 per cent wrong. Both are model outputs resting on assumptions, and neither has been scored against what happened.

labour market · the most quoted ai statistics checked

Argued perspective#

Huang, P., Guo, C., Zhou, L., Lorch, J. R., Dang, Y., Chintalapati, M. and Yao, R. (2017). Gray Failure: The Achilles' Heel of Cloud-Scale Systems #

Proceedings of HotOS '17, Whistler, British Columbia, 8 to 10 May 2017, 6 pages. DOI 10.1145/3102980.3103005. Read in full at source 17 September 2026, in the authors' open copy at Microsoft Research, since the DOI resolves to a gated ACM record. Two author names in circulation are wrong and the paper settles both: the first author is Peng Huang and not Ryan Huang, and the fourth is Jacob R. Lorch and not Jay Lorch

Method: A characterisation paper drawing on the authors' experience of real incidents in Microsoft Azure. Four incident classes are recounted and a formal model is proposed. No controlled study, no rates, and the paper describes itself as making the first attempt to characterise the phenomenon.
Finding: Defines gray failure as differential observability, in the authors' own words: a system experiences gray failure 'when at least one app makes the observation that system is unhealthy, but observer observes that system is healthy'. The model separates the system core, an observer that gathers information about whether the system is failing, and a reactor that acts on what the observer reports. The paper reports that in the authors' experience gray failure is behind most cloud incidents, and that redundancy can reduce availability rather than raise it, because more components means a higher chance that one of them is degraded in a way the failure detector does not see.
What it supports: That the gap between who suffers a failure and who is watching for it is a named, modelled engineering problem with a 2017 definition, and that the term silent or subtle failure in computing long predates its use about AI.
What it does not support: Any frequency. 'Behind most cloud incidents' is the authors' characterisation of their own experience at one company and no count is published. The paper is about machine observers rather than human ones, and nothing in it measures a person.

gray failure · differential observability · silent failure · failure detection · what is silent failure

Institutional survey#

World Economic Forum (2018). The Future of Jobs Report 2018 #

World Economic Forum, Geneva, September 2018

Method: Employer survey via the WEF membership community.
Finding: Set out expected skill demand to 2022, with analytical thinking and innovation, active learning and creativity leading the list.
What it supports: What the field expected in 2018. Its predictions are now checkable, and that is the reason for including it.
What it does not support: A representative picture of employers. Respondents are drawn from a self-selected membership network.

human capability · skills · human skills in the age of ai superskills and the wef future of jobs report

Compiled review#

Competition and Markets Authority (2019). Statutory audit services market study: final report #

Competition and Markets Authority, 18 April 2019. Read at source 19 September 2026

Method: A statutory market study by the UK competition regulator of the market for audit of large companies, drawing on the firms' own revenue data, the Financial Reporting Council's inspection results, consultation responses and the authority's own analysis. A regulator's assessment, not an experiment.
Finding: 'Companies select their own auditors', and management influence undercuts the audit committee that formally holds the choice. Non-audit services accounted for 79 per cent of the Big Four's total revenues in 2018. In 2017/18 just 73 per cent of FTSE 350 audits inspected were assessed as good or requiring limited improvement, against a 90 per cent target. The remedy proposed: 'an operational split between the Big Four's audit and non-audit businesses, to ensure maximum focus on audit quality', together with mandatory joint audit and stronger regulatory scrutiny of audit committees.
What it supports: That after a century of statutory audit with a legal duty and a regulator, the UK's competition authority still located the structural flaw in the checker being selected by the checked and in the checker's other business with the same client, and asked for the two to be split.
What it does not support: Anything about AI evaluation; the parallel is structural. The revenue and inspection figures are from 2018 and the market has changed since. The operational split was a recommendation, and what was implemented differs from what was proposed.

audit independence · CMA · operational split · Big Four · assurance · conflict of interest · can an ai safety evaluator paid by the company be independent

Compiled review#

Davies, S. C., Atherton, F., Calderwood, C. and McBride, M. (2019). United Kingdom Chief Medical Officers' commentary on screen-based activities and children and young people's mental health and psychosocial wellbeing #

Department of Health and Social Care, Office of the Chief Medical Officer, 7 February 2019

Method: Official commentary by the four UK Chief Medical Officers on a commissioned systematic map of reviews. No new data.
Finding: States that scientific research is currently insufficiently conclusive to support UK CMO evidence-based guidelines on optimal amounts of screen use or online activities, and that the research does not present evidence of a causal relationship between screen-based activities and mental health problems, noting the possibility that young people who already have mental health problems spend more time on social media. It recommends a precautionary approach anyway, separates screen time from internet content and from persuasive design as three distinct issues, and advises families on the basis that screen time can displace health-promoting activities.
What it supports: That the UK's most senior clinical advisers concluded in 2019 that the dose measure could not support guidance, and named displacement rather than duration as the mechanism worth managing.
What it does not support: That screen use is harmless. The CMOs explicitly state that the absence of evident causal effect does not mean there is no effect. It is a commentary on a map of reviews rather than primary research, and it predates the AI question entirely.

screen time · public health · displacement · policy · is screen time the same argument as ai use

Statutory investigation#

National Transportation Safety Board (2019). Collision Between Vehicle Controlled by Developmental Automated Driving System and Pedestrian, Tempe, Arizona, March 18, 2018 #

NTSB Highway Accident Report NTSB/HAR-19/03, PB2019-101402, case HWY18MH010

Method: Statutory accident investigation of a single fatal collision, with access to vehicle system data, operator records and the developer's internal procedures.
Finding: The automated driving system detected the pedestrian 5.6 seconds before impact and tracked her to the crash without ever classifying her correctly or predicting her path. The developer had disengaged the Volvo XC90's factory forward collision warning and automatic emergency braking during automated operation. On detecting an emergency the system entered a one-second period of action suppression, withholding braking while it verified the hazard or the operator took control, and NTSB record that no alert was given to the operator when action suppression was initiated. The system recognised an imminent collision 1.2 seconds before impact. Probable cause was determined as the operator's failure to monitor the driving environment while visually distracted by a personal phone, with contributing factors including inadequate safety risk assessment procedures, ineffective oversight of vehicle operators and lack of adequate mechanisms for addressing operators' automation complacency.
What it supports: That a design can name a human as the primary countermeasure in an emergency and simultaneously withhold the alert that would let them act as one, and that the stated reason for doing so was concern about false alarms.
What it does not support: That the absence of an alert caused the crash. NTSB determined the probable cause to be the operator's inattention and found she would likely have had sufficient time to react had she been attentive. One vehicle, one developer, one jurisdiction, developmental software from 2018.

automation complacency · human oversight · stop controls · alerting · automation bias · how do you design a stop button people will use what is alarm fatigue can a human approve an ai decision at machine speed

Argued perspective#

Singh, J., Cobbe, J. and Norval, C. (2019). Decision Provenance: Harnessing Data Flow for Accountable Systems #

IEEE Access, 7, 6562-6574, published 16 January 2019. DOI 10.1109/ACCESS.2018.2887201. Open access under CC BY. Peer reviewed. Preprint arXiv:1804.05741, submitted 16 April 2018, v4 of 15 November 2019. Venue checked in both directions on 14 September 2026: the arXiv comment and journal-reference fields both give volume 9, which cannot be right, since IEEE Access volume 9 is 2021; the publisher DOI record and the authors' institutional repository at Aberdeen both give volume 7, 2019, and that is what is used here

Method: Interdisciplinary conceptual paper, technical and legal, proposing that provenance methods already used to trace data be applied to decision pipelines in systems-of-systems. No new empirical work, no dataset, no measurement. Funded by EPSRC grants EP/P024394/1 and EP/R033501/1 and by Microsoft through the Microsoft Cloud Computing Research Centre.
Finding: Introduces decision provenance, defined as using provenance methods to provide information exposing decision pipelines: the chains of inputs to a decision, the nature of the decision, and the flow-on effects from the decisions and actions taken at design time and run time throughout a system. The authors argue it can support five things by name: oversight, audit, compliance, risk mitigation and user empowerment. Their diagnosis is that the accountability problem in algorithmic systems sits substantially in information flows that cross technical and organisational boundaries and are therefore invisible or opaque, rather than wholly in the model.
What it supports: That a specific, checkable alternative to post-hoc explanation was proposed in a reviewed venue five years before the EU AI Act's record-keeping and oversight articles, and that it was framed from the outset around data flow across organisational boundaries. It gives the research a named concept for the record a person needs in order to exercise an override.
What it does not support: Anything about whether decision provenance works, is affordable, or improves any outcome. It reports no data of any kind, proposes an implementation agenda rather than an implementation, and predates generative AI as a workplace technology. Its authority is the authority of an argument in a reviewed venue, not of a measurement. No claim of coinage is made by this research: the term is Singh, Cobbe and Norval's.

decision provenance · accountability · auditability · human oversight · transparency · what is decision provenance who is responsible when ai gets it wrong

Institutional survey#

World Economic Forum (2020). The Future of Jobs Report 2020 #

World Economic Forum, Geneva, October 2020

Method: Employer survey, conducted during the first year of the pandemic.
Finding: Named critical thinking and problem solving as leading skills, and forecast large-scale reskilling need.
What it supports: What the field expected in 2020, including a pandemic-shaped view of remote work.
What it does not support: A clean read on AI. The 2020 edition is dominated by COVID-era disruption.

human capability · skills · critical thinking · human skills in the age of ai superskills and the wef future of jobs report

Institutional modelling#

Council of the City of New York; Department of Consumer and Worker Protection (2021). Local Law 144 of 2021, automated employment decision tools #

New York City Administrative Code ss 20-870 to 20-874, became law 13 December 2021, in force 1 January 2023. Rules at 6 RCNY Subchapter T, ss 5-300 to 5-304

Method: Municipal statute and implementing rules. Requires an annual independent bias audit of an automated employment decision tool before it is used on a candidate or employee in New York City, publication of the audit summary, and notice to those it is used on.
Finding: The rules at 6 RCNY s 5-300 define the trigger, 'substantially assist or replace discretionary decision making', in three limbs, of which the third is: 'to use a simplified output to overrule conclusions derived from other factors including human decision-making'. The limbs sit in the implementing rules made by the Department of Consumer and Worker Protection rather than in the statute, which became law on 13 December 2021 and came into force on 1 January 2023. A city agency drafting rules about hiring software set down the mechanism by which a score displaces a judgement, and did so before most employers had a position on it. CORRECTED 3 October 2026: this sentence previously credited the three-limb definition to a municipal legislature and dated it to 2022, neither of which the sources cited here support.
What it supports: That a jurisdiction has defined in binding rules the specific failure this research is about: a simplified machine output overruling a human conclusion. The definition is narrower and more precise than the oversight language in most national AI frameworks.
What it does not support: Anything about practice or enforcement, which is the subject of the separate Comptroller audit graded below. It is city law, applying only to employment decisions within New York City, and it mandates an audit rather than an outcome.

automated employment decision tool · bias audit · human decision-making · oversight · what is meaningful human oversight can ai be unbiased is it ethical to let ai judge people can i use ai for performance reviews hiring signals when effort is free

Operator account#

Dixit, H. D., Pendharkar, S., Beadon, M., Mason, C., Chakravarthy, T., Muthiah, B. and Sankar, S. (2021). Silent Data Corruptions at Scale #

arXiv 2102.11245, 22 February 2021, 8 pages. Facebook, Inc. Read at source 17 September 2026

Method: A company's account of silent data corruption across its own server fleet, with a worked debug case. Test scenarios were run across hundreds of thousands of machines over a period the authors give as longer than 18 months. The fleet, the tests and the detections cannot be rebuilt or checked from outside the firm, and the paper is a preprint with no journal reference.
Finding: Silent data corruptions are not captured by the error-reporting mechanisms inside a CPU and so are not traceable at hardware level, while the corrupted data propagates up the stack and surfaces as an application problem. Running the test library across hundreds of thousands of machines resulted in hundreds of CPUs detected with these errors, which the authors read as showing the problem is systemic across processor generations rather than confined to a bad batch. Their conclusion is that reducing it needs fault-tolerant software architecture as well as hardware resilience and production detection.
What it supports: That a failure invisible to the layer responsible for detecting it is a measured phenomenon in production hardware, and that the fix required a layer above the failing one to assume the failure was possible.
What it does not support: A rate. 'Hundreds of CPUs' out of 'hundreds of thousands of machines' is stated without a denominator anyone can use, the test library is not published, and the result is one company's fleet in one period. Nothing here concerns AI.

silent data corruption · silent failure · detection · fault tolerance · what is silent failure

Institutional survey#

McKinsey and Company (2021). Defining the skills citizens will need in the future world of work #

McKinsey Public and Social Sector Practice, June 2021

Method: Online psychometric survey of 18,000 people across 15 countries, fielded 2019; 56 elements in 13 skill groups.
Finding: Identifies distinct elements of talent, the DELTAs, associated with employment, income and job satisfaction.
What it supports: An unusually large individual-level dataset on skills and outcomes, which is rare in this literature.
What it does not support: Anything about AI. It was fielded in 2019, before the generative-AI period entirely.

skills · human capability · human skills in the age of ai

Definitional instrument#

ISO/IEC JTC 1/SC 42 (2022). ISO/IEC 22989:2022, Information technology, Artificial intelligence, Artificial intelligence concepts and terminology #

ISO, first edition 2022. Clauses 3.1.1, 3.1.4, 3.1.5 and 3.1.7 read at source on ISO's own Online Browsing Platform, 22 September 2026.

Method: An international vocabulary standard. Fixes terms and tests nothing.
Finding: Clause 3.1.1, the FIRST defined term in the standard, is 'AI agent: automated entity that senses and responds to its environment and takes actions to achieve its goals'. Automated is separately defined at 3.1.7 as functioning 'without human intervention' under specified conditions. AI system at 3.1.4 is 'engineered system that generates outputs such as content, forecasts, recommendations or decisions for a given set of human-defined objectives'. Autonomy at 3.1.5 is 'characteristic of a system that is capable of modifying its intended domain of use or goal without external intervention, control or oversight'.
What it supports: That one standards body has defined the AI agent as a term distinct from the AI system, and placed it first in its vocabulary.
What it does not support: Any legal effect, and no current alignment with the definitions the OECD and the EU now use. The 22989 formula for an AI system, 'for a given set of human-defined objectives', is the pre-2023 wording that both the OECD and the EU AI Act have since moved away from, so the standard's two headline definitions are anchored to a description of AI its neighbours have abandoned. Its bar for autonomy, modifying the goal itself, is also far higher than the EU's varying levels of autonomy, so the same product can be autonomous under one document and not the other.

AI agent · AI system · autonomy · standards · what is an ai agent

Argued perspective#

Chalmers, D. J. (2023). Could a Large Language Model Be Conscious?#

Boston Review, 9 August 2023; an edited version of a talk given at NeurIPS on 28 November 2022, with an afterword written eight months later. Also circulated as arXiv:2303.07103. Read at source 12 September 2026.

Method: A philosophical argument. Chalmers names six candidate requirements for consciousness that current language models lack, assigns each a credence, and combines them. No data of his own.
Finding: The six candidates are biology, senses and embodiment, world models and self models, recurrent processing, global workspace, and unified agency. He argues it would not be unreasonable to hold a credence of at least one in three that each is required, which on independence would put the chance that a system lacking all six is conscious below one in ten. His stated position is confidence 'somewhere under 10 percent in current LLM consciousness'. For successors he combines a credence above 50 per cent that sophisticated systems with those properties arrive within a decade with at least 50 per cent that such systems would be conscious, giving '25 percent or more'. He warns twice against taking the numbers seriously, calling precision here specious. The afterword says faster-than-expected progress makes his timelines possibly conservative and changes nothing fundamental in the analysis.
What it supports: That the most cited philosopher of mind working on this question puts current language-model consciousness below one in ten and a serious possibility within a decade, and that he regards both figures as illustrative rather than measured.
What it does not support: Anything about any system. These are credences in an argument, not estimates from data, and the author says so in terms. The 2020 survey figures he reports, roughly 3 per cent of professional philosophers accepting or leaning towards current AI being conscious against 82 per cent rejecting, and 39 per cent against 27 on future AI, are taken here as he reports them; the survey itself was not read for this entry.

consciousness · sentience · credence · is ai conscious

Argued perspective#

Future of Life Institute (2023). Pause Giant AI Experiments: An Open Letter #

Published 22 March 2023. Open letter with public signatory list; the Institute reported over 30,000 signatures. Read at source 17 September 2026

Method: An open letter, not a study. Called on all AI labs to immediately pause for at least six months the training of AI systems more powerful than GPT-4, and on governments to institute a moratorium if the pause could not be enacted quickly, citing the absence of planning and management proportionate to the capability being built.
Finding: The pause did not take place. No signatory lab or addressed lab paused training, more capable models were released within the six months asked for, and the letter's public effect was on discourse and on policy attention (the UK AI Safety Summit of November 2023 and the international safety report that followed) rather than on the development schedule.
What it supports: That a development pause was asked for with maximum public force, by named senior figures, and did not happen, which is the strongest available evidence about whether such a pause is a decision anyone outside a handful of companies and governments can take.
What it does not support: Anything about whether a pause would have been beneficial or harmful; the letter argued a position and no counterfactual exists. Signature counts were self-reported by the publishing organisation and included unverified entries at the time of publication.

AI pause · moratorium · AI safety · should ai development be paused is ai dangerous what should a board do about the ai safety warnings

Institutional survey#

International Organization for Standardization (2023). ISO/IEC 42001:2023, Artificial intelligence management system #

ISO, December 2023

Method: Certifiable management system standard developed by ISO/IEC JTC 1/SC 42.
Finding: Specifies requirements for an AI management system on a Plan-Do-Check-Act structure, against which an organisation can be certified by a third party.
What it supports: That an auditable management standard for AI now exists and can be certified.
What it does not support: Legal compliance. Certification against ISO 42001 is not the same as meeting the EU AI Act, and the two are routinely conflated.

· how do you audit an ai assisted decision

Institutional modelling#

NFER (2023). The Skills Imperative 2035: An analysis of the demand for skills in the labour market in 2035 (Working Paper 3)#

National Foundation for Educational Research, with the University of Sheffield, funded by the Nuffield Foundation, May 2023

Method: 161 skills from the US O*NET database mapped to UK occupational codes, combined with UK employment projections.
Finding: Projects rising demand for a set of essential employment skills in the UK to 2035.
What it supports: A rare UK-specific, independently funded, methodologically documented projection.
What it does not support: Precision. It maps US skill data onto UK occupations, and a revised working paper corrects coding errors in the underlying labour force survey.

skills · labour market · human skills in the age of ai

Institutional survey#

National Institute of Standards and Technology (2023). AI Risk Management Framework 1.0 #

NIST, 26 January 2023

Method: Voluntary framework organised around four functions: Govern, Map, Measure and Manage.
Finding: Provides a common structure and vocabulary for AI risk management, with a companion Playbook and a 2024 generative AI profile.
What it supports: That a shared vocabulary exists that a board is likely to recognise.
What it does not support: Compliance with anything. It is voluntary and confers no legal status.

· how do you audit an ai assisted decision

Institutional modelling#

National Institute of Standards and Technology (2023). AI Risk Management Framework Playbook, MANAGE 2.4 #

NIST AI Resource Center, AI RMF 1.0 Playbook

Method: Voluntary framework and playbook developed through public consultation. Guidance rather than measurement.
Finding: Requires mechanisms and assigned responsibilities to supersede, disengage or deactivate AI systems showing performance inconsistent with intended use, and names five triggering conditions: end of system lifetime; risks exceeding tolerance thresholds; mitigation beyond the organisation's capacity; feasible mitigations failing regulatory, legal or normative standards; and impending risk detected in monitoring for which timely mitigation cannot be implemented. Decision thresholds for bypass or deactivation are treated as part of continual monitoring, and organisations are encouraged to provide contingency options including redundant or backup systems.
What it supports: That an authoritative framework treats deployment as reversible and expects the reversal mechanism, its thresholds and its fallback to exist before they are needed.
What it does not support: That any organisation does this, or that doing it works. The AI RMF is voluntary guidance, not a standard with conformity assessment, and it contains no evidence about outcomes.

governance · deployment · oversight

Institutional modelling#

OECD (2023). OECD Skills Outlook 2023: Skills for a Resilient Green and Digital Transition #

OECD Publishing, Paris, November 2023

Method: Secondary analysis of OECD data including PISA and PIAAC, not a new survey.
Finding: Analyses the skills required for green and digital transitions across member economies.
What it supports: A cross-national, methodologically transparent baseline on skills, from data collected to a documented standard.
What it does not support: Anything AI-specific and current. The underlying data collection predates the generative-AI period.

skills · human capability · human skills in the age of ai

Definitional instrument#

OECD (2023). Recommendation of the Council on Artificial Intelligence, OECD/LEGAL/0449, the definition of an AI system #

OECD. Adopted 22 May 2019; the AI system definition revised by the OECD Council on 8 November 2023; the Recommendation further revised 3 May 2024 without altering the definition. Read at source in the OECD's own PDF of the legal instrument on 22 September 2026, because the HTML page returns only a JavaScript notice to a fetcher.

Method: A non-binding Council recommendation. Defines five terms and sets principles; measures nothing.
Finding: 'An AI system is a machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments. Different AI systems vary in their levels of autonomy and adaptiveness after deployment.' The word agent appears neither in the definition nor among the five defined terms, which are AI system, AI system lifecycle, AI actors, AI knowledge and stakeholders. The OECD's term for the party that acts is AI actors, 'those who play an active role in the AI system lifecycle, including organisations and individuals that deploy or operate AI', which is a definition of people and organisations.
What it supports: That the definition Article 3(1) of the EU AI Act closely tracks was set by the OECD in November 2023, and that it makes no room for a machine agent as a category.
What it does not support: Any obligation. It is a recommendation, and its force comes from the legislatures that adopted its wording rather than from itself.

OECD · AI system · AI actors · definitions · what is an ai agent

Argued perspective#

United States Department of Defense (2023). DoD Directive 3000.09, Autonomy in Weapon Systems #

Reissued 25 January 2023, replacing the 2012 directive. Read at source 27 September 2026. The Washington Post reported on 26 September 2026 that a revision ordered in June 2026 had not been published by late September

Method: A department policy directive: definitions of autonomous and semi-autonomous weapon systems, design requirements, and a senior review before development and again before fielding.
Finding: Autonomous and semi-autonomous weapon systems 'will be designed to allow commanders and operators to exercise appropriate levels of human judgment over the use of force'. The directive does not require a human approval of each target; it requires that the level of human judgement be appropriate, decided case by case through the review process it establishes.
What it supports: What the United States requires of itself in writing on the human role in autonomous weapons, and that the requirement is a standard of judgement rather than a rule of per-target approval.
What it does not support: How the standard is applied in any operation, whether it is met, or what the pending revision will say. A policy document, not a measurement.

autonomous weapons · human judgement · human oversight · meaningful human control · should a human approve every target an ai selects

Institutional survey#

World Economic Forum (2023). The Future of Jobs Report 2023 #

World Economic Forum, Geneva, April 2023

Method: Employer survey on expectations to 2027.
Finding: Analytical thinking leads, with creative thinking second, and a growing emphasis on self-efficacy skills.
What it supports: The first post-ChatGPT edition, published five months after launch.
What it does not support: Considered judgement on generative AI. It was fielded too early for that.

human capability · skills · human skills in the age of ai superskills and the wef future of jobs report

Institutional survey#

AI Verify Foundation and IMDA, Singapore (2024). Model AI Governance Framework for Generative AI #

IMDA and AI Verify Foundation, 30 May 2024

Method: National governance framework for generative AI, nine dimensions. Both published PDF versions read in full and searched at source.
Finding: Contains zero occurrences of "human oversight", "human-in-the-loop", "over-reliance", "automation bias" or "deskill". Human oversight is not among its nine dimensions. The nearest it comes is a note that "core skills such as creativity, critical thinking and complex problem-solving are important to helping people harness AI effectively", and its only use of "competency" concerns third-party auditors rather than the human overseer.
What it supports: The value here is the confirmed absence, and the trajectory it establishes. In twenty months the same issuing body went from a framework with no oversight language at all to one built around it that also names deskilling. That shift is documented and quotable.
What it does not support: That Singapore was indifferent to oversight in 2024; the 2020 framework it builds on was not read at source and may carry such language. It establishes what the generative AI framework does not say, not what the whole regime did not say.

human oversight · Singapore · governance · absence · ai and work in asia

Peer-reviewed#

Chan, A., Ezell, C., Kaufmann, M., Wei, K., Hammond, L., Bradley, H., Bluemke, E., Rajkumar, N., Krueger, D., Kolt, N., Heim, L. and Anderljung, M. (2024). Visibility into AI Agents #

Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT 24), Rio de Janeiro, 3-6 June 2024. DOI 10.1145/3630106.3658948

Method: Conference paper assessing three categories of measure for increasing visibility into deployed AI agents, across a spectrum of centralised to decentralised deployment contexts and accounting for hardware and software service providers in the supply chain. Analysis and proposal rather than measurement.
Finding: Defines visibility as information about where, why, how and by whom AI agents are used, and assesses agent identifiers, real-time monitoring and activity logging as measures. Names five agent-specific risks: malicious use, overreliance and disempowerment, delayed and diffuse impacts, multi-agent risks, and sub-agents. On the last, the authors state that stopping an agent may require intervening on its sub-agents and that this may be difficult because, in their words, we lack methods for determining when an agent has created a sub-agent. The paper explicitly does not advocate immediate implementation of the measures and discusses their privacy and concentration-of-power costs.
What it supports: That the basic precondition of managing an agent, knowing what is running and what it has spawned, is an unsolved technical problem rather than a governance oversight.
What it does not support: That any of the proposed measures works, or is proportionate. The authors state they are describing options for further study rather than recommending deployment, and the paper contains no empirical evaluation.

AI agents · visibility · sub-agents · monitoring · accountability · who manages ai agents when your customers have agents what your agents can reach

Institutional survey#

DIFC Commissioner of Data Protection (2024). Regulation 10 on Personal Data Processed through Autonomous and Semi-Autonomous Systems #

Dubai International Financial Centre, DIFC-DP-GL-23 Rev.03, updated 27 August 2024; regulation enacted September 2023

Method: Binding regulation within the DIFC free zone, with accompanying guidance. Applies to personal data processing by autonomous and semi-autonomous systems, not to AI generally.
Finding: States that "human-defined processing purposes must always prevail in Systems development and use". Draws an explicit analogy between an autonomous system and an employee: where a system operates for the benefit of its deployer, "its position is substantially similar to that of an employee within the Deployer organization, and the Deployer should be therefore liable for its actions in the same way it may be liable for an employee's actions". Creates a named Autonomous Systems Officer performing a function similar to a data protection officer.
What it supports: That the employee analogy for accountability, and a named human role responsible for it, exist in a binding instrument somewhere in the Gulf.
What it does not support: Anything about capability or competence. It is a data protection regulation confined to one free zone, it addresses liability rather than skill, and it imposes no requirement that the responsible human be able to do the work being supervised.

accountability · human oversight · United Arab Emirates · regulation · ai and work in the gulf who owns verification when ai does the work when your customers have agents what your agents can reach

Compiled review#

European Parliament and Council (2024). Regulation (EU) 2024/1689, Article 3(49) definition of serious incident and Article 73, reporting of serious incidents #

Official Journal of the European Union, 12 July 2024. Text read via the Future of Life Institute's AI Act Explorer

Method: Primary legislation. Article 3(49) defines a serious incident; Article 73 sets the reporting duty on providers of high-risk systems and the periods within which reports are due, and requires deployers to inform providers.
Finding: A serious incident is an incident or malfunctioning of an AI system that directly or indirectly leads to the death of a person or serious harm to a person's health, a serious and irreversible disruption of the management or operation of critical infrastructure, an infringement of obligations under Union law intended to protect fundamental rights, or serious harm to property or the environment. Providers of high-risk systems must report such incidents to the market surveillance authorities of the member state where the incident occurred, immediately after establishing a causal link or the reasonable likelihood of one, and within periods the Article sets by severity.
What it supports: That a legal definition of a serious AI incident now exists and carries a reporting duty for high-risk systems, so an organisation deploying one cannot treat 'was it an incident' as an internal question alone.
What it does not support: What counts as an incident for the great majority of AI use, which is not high-risk under Annex III and is not covered, or how the reporting periods will work in practice; the duty is new and has produced no body of cases. The definition is a floor, not an organisational standard.

ai leadership · serious incident · eu ai act · incident reporting · boards · what counts as a serious ai incident should frontier ai be licensed

Institutional modelling#

European Parliament and Council of the European Union (2024). Regulation (EU) 2024/1689, Annex III: High-risk AI systems referred to in Article 6(2)#

Official Journal of the European Union, official version of 13 June 2024. Text read via the European Commission AI Act Service Desk

Method: Binding legislative text. Annex III lists the eight areas in which AI systems are classified as high risk under Article 6(2), triggering the requirements of Chapter III.
Finding: Point 3 classifies as high risk AI systems used to determine access or admission to education, to evaluate learning outcomes including where those outcomes steer a learner's path, to assess the level of education a person will receive, and to monitor and detect prohibited behaviour during tests. Point 4 covers employment, workers' management and access to self-employment, naming systems used for recruitment or selection, including placing targeted job advertisements, analysing and filtering applications and evaluating candidates, and systems making decisions on promotion or termination, allocating tasks by individual traits, or monitoring and evaluating performance.
What it supports: That two of the applications discussed most loosely in public debate, marking pupils' work and sifting job applicants, are named in binding European law as high risk, with documentation, record-keeping, human oversight and explanation obligations attached.
What it does not support: Compliance or effect. Classification is not evidence that any system is biased or unsafe, and the Annex says nothing about how well the resulting obligations are met in practice.

EU AI Act · high-risk AI · algorithmic hiring · assessment · how will ai change human resources how will ai change teaching is it ethical to let ai judge people can i use ai for performance reviews should universities let ai mark students work

Institutional modelling#

European Parliament and Council of the European Union (2024). Regulation (EU) 2024/1689, Article 13: Transparency and provision of information to deployers #

Official Journal of the European Union, official version of 13 June 2024. Text read via the European Commission AI Act Service Desk

Method: Primary legal text.
Finding: Requires high-risk systems to be designed so their operation is sufficiently transparent for deployers to interpret the output and use it appropriately, and to be accompanied by instructions for use stating the level of accuracy and its metrics, robustness and cybersecurity against which the system was tested and validated, plus known and foreseeable circumstances affecting that expected level, and where applicable information enabling deployers to interpret the output. Article 15(3) requires declared accuracy levels in those instructions.
What it supports: That European law requires documentary, aggregate, ex ante disclosure of accuracy and limitations to the deployer.
What it does not support: Any requirement to communicate uncertainty at the point of use. The word uncertainty appears in neither Article 13, Article 15 nor Article 50, and nothing requires a system to tell the person in front of it how confident it is in the specific output. EUR-Lex returned an empty document to every route tried on 4 September 2026, so the text was read at the Commission's own service desk; the Article 50 page there carries a notice that the provision has been amended by the Digital Omnibus and the displayed text not yet updated.

EU AI Act · transparency · uncertainty · regulation · how should an ai agent communicate uncertainty

Definitional instrument#

European Parliament and Council of the European Union (2024). Regulation (EU) 2024/1689, Article 3: Definitions #

Official Journal of the European Union, OJ L, 2024/1689, 12 July 2024. The full Official Journal text was read on EUR-Lex on 22 September 2026, 588,616 characters from the title through Annex XIII, and searched programmatically.

Method: Binding legislative text. Article 3 fixes the vocabulary the whole Regulation then operates on.
Finding: Article 3(1) defines an AI system as 'a machine-based system that is designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment, and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments'. THE WORD AGENT APPEARS NOWHERE IN THE REGULATION. A case-insensitive search of the full Official Journal text returns 0 matches for 'agent' and 0 for 'agentic', against 1,107 for 'AI system' as a control confirming the text loaded completely. All 21 occurrences of 'agency' were inspected individually: each is either an institutional body, such as the European Union Aviation Safety Agency, or the phrase 'human agency and oversight'.
What it supports: That Europe's binding AI law regulates AI systems, providers, deployers and general-purpose AI models, and holds no concept of an agent at all. Whatever a vendor calls a product, the Act reaches it as an AI system or not at all.
What it does not support: That agents are unregulated, which does not follow: the obligations attach to the system and to the people placing it on the market, under whatever name. Nor does the absence of the word imply the legislature overlooked the category. It implies only that no obligation in the Act turns on it.

EU AI Act · AI system · AI agent · regulation · what is an ai agent

Definitional instrument#

European Parliament and Council of the European Union (2024). Regulation (EU) 2024/1689, Article 5: Prohibited AI practices #

Official Journal of the European Union, OJ L, 2024/1689, 12 July 2024. Text read on 25 September 2026 via the AI Act Explorer maintained by the Future of Life Institute, which carries the consolidated article with its date of application under Article 113(a) and the two later additions applying from 2 December 2026

Method: Binding legislative text. Article 5 lists the AI practices prohibited outright in the Union, each with its stated exceptions, applicable from 2 February 2025.
Finding: Eight practices are prohibited: harmful subliminal or manipulative techniques; exploiting vulnerabilities of age, disability or social or economic situation; social scoring; assessing the risk of a person committing a criminal offence based solely on profiling; untargeted scraping of facial images to build recognition databases; inferring emotions in workplaces and educational institutions, save for medical or safety reasons; biometric categorisation to deduce race, political opinions, trade union membership, religious or philosophical beliefs, sex life or sexual orientation; and real-time remote biometric identification in publicly accessible spaces for law enforcement, subject to listed exceptions. On the consolidated text, two further prohibitions, on generating or manipulating intimate images without consent and on child sexual abuse material, apply from 2 December 2026.
What it supports: That the nearest thing to a binding list of AI red lines in a major jurisdiction is a list of uses, addressed to the people deploying systems, and that no prohibition in it concerns what a general-purpose model may itself do.
What it does not support: Compliance, enforcement or effect. The article's existence says nothing about how often the prohibited practices occur or whether any enforcement action has concluded; none was found in the record read for the page that cites this entry.

EU AI Act · prohibited practices · AI red lines · social scoring · emotion recognition · biometric identification · what are ai red lines

Institutional survey#

European Union (2024). Article 12, Record-keeping, Regulation (EU) 2024/1689 #

Official Journal of the European Union

Method: Binding regulation.
Finding: High-risk AI systems must technically allow automatic recording of events over the system lifetime, to enable identification of risk situations, post-market monitoring and monitoring of operation. Article 19 requires providers to keep those logs for at least six months.
What it supports: That the capability to reconstruct what a system did is now a legal requirement rather than good practice.
What it does not support: That anyone must read the logs, or that a decision must be reviewable at the level of the individual case.

· how do you audit an ai assisted decision

Institutional modelling#

European Union (2024). Regulation (EU) 2024/1689, Article 14: Human Oversight #

Official Journal of the European Union. Applies from 2 December 2027 for Annex III high-risk systems and 2 August 2028 for Annex I

Method: Binding regulation. Legal requirement rather than empirical finding.
Finding: Requires high-risk systems to be designed so they can be effectively overseen by natural persons, and requires that those persons be enabled to understand the system's capacities and limitations, to remain aware of the tendency to over-rely on its output (automation bias, named in the text), to interpret the output correctly, to decide not to use it or to disregard, override or reverse it, and to intervene or stop it through a stop button bringing the system to a safe halt. For biometric identification systems under Annex III point 1(a), no action may be taken unless the identification is separately verified by at least two natural persons with the necessary competence, training and authority.
What it supports: That an authoritative regulator treats override capability, not merely presence, as the content of oversight, and names competence, training and authority together in the one place it specifies who must verify.
What it does not support: That any of this happens. The oversight provisions carry 2 December 2027 as a longstop rather than a start date, since the Digital Omnibus ties the high-risk rules to the availability of standards and caps the delay at sixteen months for Annex III, so they may apply sooner; see eu-ai-act-application-dates. the two-person requirement covers one narrow category rather than high-risk systems generally, and the regulation contains no evidence that oversight so constituted works.

human oversight · governance · automation bias · how do you design a stop button people will use how do humans and agents divide work do employees know they are expected to override ai

Institutional modelling#

European Union (2024). Regulation (EU) 2024/1689, Article 26: Obligations of deployers of high-risk AI systems #

Official Journal of the European Union. Chapter III, Section 3

Method: Binding regulation. Legal requirement rather than empirical finding. Text read at source.
Finding: Paragraph 2 requires that deployers assign human oversight to natural persons who have the necessary competence, training and authority, as well as the necessary support. Paragraph 5 requires deployers to monitor operation on the basis of the instructions for use, to inform the provider and the relevant market surveillance authority without undue delay where the system presents a risk, and to suspend use of the system. Paragraph 6 requires retention of the automatically generated logs under the deployer's control for a period appropriate to the intended purpose and at least six months. Paragraph 7 requires employers, before putting a high-risk system into service at the workplace, to inform workers' representatives and the affected workers that they will be subject to its use. Paragraph 11 requires that natural persons subject to decisions made or assisted by an Annex III system be told.
What it supports: That the obligation to name a competent, authorised human overseer of a deployed system, and to be able to stop it, is law rather than good practice for systems in scope.
What it does not support: That it applies to most commercial agent deployments, which will fall outside the high-risk classification. Nor that any of it happens: the Regulation creates duties and does not evidence compliance. The application timetable has been subject to amendment and the published texts consulted for this entry did not agree on the dates, so no date is stated here.

human oversight · deployer obligations · AI governance · accountability · who manages ai agents which decisions should become slower because of ai can i use ai for performance reviews what are the risks of using ai at work who is responsible when ai gets it wrong

Expert forecast survey#

Grace, K., Stewart, H., Sandkühler, J. F., Thomas, S., Weinstein-Raun, B. and Brauner, J. (2024). Thousands of AI Authors on the Future of AI #

AI Impacts, preprint, January 2024. Survey fielded October 2023. Read in full at source 9 September 2026

Method: Survey of 2,778 researchers who had published in the prior year at NeurIPS, ICML, ICLR, AAAI, IJCAI or JMLR. 20,066 contacted, 1,607 bounced, 18,459 functioning addresses, response rate 15 per cent. Randomised question framings across respondents, deliberately, to measure framing effects. Timelines aggregated by fitting gamma distributions.
Finding: On timing, the 2023 aggregate forecast gave High-Level Machine Intelligence a 50 per cent chance by 2047, thirteen years earlier than the 2060 given one year before, and a 10 per cent chance by 2027. On risk, the median answer depended on the wording put to the same population: 5 per cent for future AI advances causing human extinction or similarly permanent and severe disempowerment (n=1,321, mean 16.2), and 10 per cent for human INABILITY TO CONTROL advanced AI causing the same outcome (n=661, mean 19.4). Depending on how it was asked, between 41.2 and 51.4 per cent gave more than a 10 per cent chance. 38 per cent gave at least 10 per cent to extremely bad outcomes on the general question, down from 48 per cent in 2022.
What it supports: That the most-quoted numbers on AI extinction risk come from a large expert population, and that a change of wording moves the median from 5 to 10 per cent within it.
What it does not support: Any probability of anything. The authors say so themselves: their participants are experts in AI and not, to their knowledge, skilled forecasters; different respondents give very different answers; and they cite Karger et al. finding that framing moved lay estimates of existential risk by nearly six orders of magnitude. A 15 per cent response rate leaves participation bias possible, though the authors looked for it and did not find it. HLMI is defined as feasibility, not adoption.

AGI · High-Level Machine Intelligence · superintelligence · what is agi is ai dangerous should ai development be paused what should a board do about the ai safety warnings

Compiled review#

James Ryseff, Brandon F. De Bruhl and Sydne J. Newberry (2024). The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed: Avoiding the Anti-Patterns of AI #

RAND Corporation, RRA2680-1, 13 August 2024

Method: Sixty-five semi-structured interviews with experienced AI practitioners, August to December 2023: 50 from industry across more than 50 organisations, found through LinkedIn, and 15 from academia by convenience sample. Qualitative; no failure rate is measured.
Finding: 'By some estimates, more than 80 percent of AI projects fail', which the authors describe as twice the rate of ordinary IT projects; the estimate is cited, not measured. Five root causes from the interviews, in order: leadership-driven failure (misunderstanding what problem the project is for, or optimising the wrong metric), data-driven failure, bottom-up failure where the technology was chosen before the problem, underinvestment in infrastructure, and immature technology.
What it supports: That when practitioners are asked why AI projects fail, the first thing they name is a leadership decision, and that four of the five causes are decisions rather than technology.
What it does not support: The 80 per cent. RAND quotes it from elsewhere and anyone citing RAND for it is citing a citation. Sixty-five interviews with people found on LinkedIn is a qualitative sample, and practitioners blaming leadership is also what practitioners would say.

ai leadership · project failure · root causes · rand · ai strategy · is it true that 95 per cent of ai pilots fail do you need to be technical to lead ai ai governance versus ai leadership

Peer-reviewed#

Luccioni, A. S., Jernite, Y. and Strubell, E. (2024). Power Hungry Processing: Watts Driving the Cost of AI Deployment?#

Proceedings of ACM FAccT 2024, Rio de Janeiro, 3 to 6 June 2024. arXiv:2311.16863. Read at source 30 September 2026

Method: Energy and carbon measured for 1,000 inferences on benchmark datasets across open models and tasks, comparing task-specific with multi-purpose generative models.
Finding: Mean energy per 1,000 inferences was 2.907 kWh for image generation, 0.047 kWh for text generation and 0.002 kWh for text classification. On question answering the most efficient task-specific models emitted 0.3 g CO2e per 1,000 inferences and multi-purpose models 10 g. The authors report that multi-purpose generative models are orders of magnitude more expensive than task-specific systems.
What it supports: That the energy cost of an inference varies by orders of magnitude with the task and with whether the model is general-purpose, for the open models and hardware they measured.
What it does not support: What any commercial chatbot uses today. The models are open models tested in 2023, and serving practice has changed since. The paper measures inference and does not settle the grid-level total.

inference energy · task-specific models · image generation · is ai bad for the environment

Institutional modelling#

McKinsey Global Institute (2024). A new future of work: The race to deploy AI and raise skills in Europe and beyond #

McKinsey Global Institute, May 2024

Method: Modelling for 2022-2030 across nine EU countries, the UK and the US, plus a survey of 1,100 or more C-suite executives in five countries.
Finding: Projects large-scale occupational transitions and rising demand for social, emotional and higher cognitive skills.
What it supports: A transparent, well-documented scenario model, useful for direction.
What it does not support: What will happen. Scenario models are assumption-driven, and McKinsey's prior transition estimates have moved substantially between editions.

labour market · skills · ai workforce strategy will ai replace my job

Argued perspective#

Morris, M. R., Sohl-Dickstein, J., Fiedel, N., Warkentin, T., Dafoe, A., Faust, A., Farabet, C. and Legg, S. (2024). Position: Levels of AGI for Operationalizing Progress on the Path to AGI #

Proceedings of the 41st International Conference on Machine Learning, PMLR 235. Google DeepMind

Method: Analysis of the existing published definitions of AGI, from which the authors distil six principles an ontology should satisfy, then propose a two-dimensional framework of performance depth against capability breadth.
Finding: The authors set out six performance levels: No AI, Emerging (equal to or somewhat better than an unskilled human), Competent (50th percentile of skilled adults), Expert (90th percentile), Virtuoso (99th percentile) and Superhuman (outperforming all humans), crossed with narrow and general breadth. They argue that levels interact with deployment choices about autonomy and risk, so a capability level does not by itself say how a system should be used.
What it supports: That the published definitions of AGI were divergent enough that a team at the largest AI lab judged a new ontology necessary, and that AGI is better treated as a set of levels than a threshold.
What it does not support: Nothing empirical. This is a position paper proposing a framework, not a measurement, and the percentile bands are a proposal rather than a validated instrument. The framework has not been adopted as a standard.

AGI · superintelligence · what is agi

Institutional survey#

National Audit Office (2024). Use of artificial intelligence in government #

HC 612, Session 2023-24, 15 March 2024

Method: Value-for-money audit of the Cabinet Office and DSIT, including a survey of 87 government bodies conducted in autumn 2023, document review and interviews. Excludes simple rules-based automation, AI embedded by default in existing tools, and individuals' ad hoc use of public tools.
Finding: 37 per cent of responding bodies had deployed AI, typically one or two use cases; 70 per cent were piloting or planning, median four use cases. 21 per cent had an organisational AI strategy, with 61 per cent planning one. Of 32 bodies with deployed AI, 24 always or usually had a named accountable owner and 15 said use cases were always or usually identified at organisational level before deployment. 30 per cent of all respondents had risk and quality assurance processes explicitly incorporating AI risks. 70 per cent named difficulty recruiting or retaining AI skills as a barrier. The Cabinet Office's Central Digital and Data Office identified in 2023 that almost a third of civil service tasks, those it defined as routine, could be automated, and did not examine feasibility or assess cost.
What it supports: That the UK productivity claim for public sector AI rests on an indicative sizing exercise the auditor found untested for feasibility or cost, and that organisational ownership of deployed AI was incomplete in 2023.
What it does not support: The current position. The survey was taken in autumn 2023, before the generative wave reached most departments, and the picture will have moved. Survey response is self-reported and covers 87 bodies rather than the whole public sector.

AI in government · governance · accountability · productivity claims · how will ai change the public sector which ai investments should we stop

Institutional modelling#

Office of Management and Budget (Shalanda D. Young) (2024). M-24-10, Advancing Governance, Innovation, and Risk Management for Agency Use of Artificial Intelligence #

OMB memorandum M-24-10, 28 March 2024, 34 pages. Rescinded and replaced by M-25-21 on 3 April 2025

Method: Binding executive-branch guidance to United States federal agencies, read in full at the archived original. Both documents were searched term by term.
Finding: M-24-10 used the term automation bias twice. As a defined term at section 6: 'the propensity for humans to inordinately favor suggestions from automated decision-making systems and to ignore or fail to seek out contradictory information made without automation'. And as a mandatory minimum practice at section 5(c)(iv)(G): agencies 'must ensure there is sufficient training, assessment, and oversight for operators of the AI to interpret and act on the AI s output, combat any human-machine teaming issues (such as automation bias)'. The successor memorandum M-25-21 of 3 April 2025 contains the term nowhere. M-24-10 also distinguished rights-impacting from safety-impacting AI, a distinction M-25-21 collapses into a single high-impact class.
What it supports: That binding United States federal AI guidance named automation bias in March 2024, both as a definition and as a requirement tied to operator training, and that the language was removed thirteen months later when the memorandum was replaced.
What it does not support: That the removal was deliberate or that federal agencies have stopped addressing the risk. M-25-21 still requires human oversight, intervention and accountability for high-impact uses. The words over-reliance, deskilling and complacency appear in NEITHER document, so the finding concerns one term and not a vocabulary. Both are memoranda rather than statute, and revocable.

automation bias · human oversight · operator training · federal AI guidance · what is automation bias how do we train people to use ai

Institutional survey#

Ryseff, J., De Bruhl, B. and Newberry, S. J. (2024). The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed #

RAND Corporation, RR-A2680-1

Method: Qualitative root-cause study. Interviews with 65 data scientists and engineers with at least five years building AI and machine-learning models in industry or academia.
Finding: A set of organisational anti-patterns behind AI project failure, chiefly misunderstood or miscommunicated intent, inadequate data, focus on the technology rather than the problem, missing infrastructure, and problems the technology cannot solve. The report opens by stating that by some estimates more than 80 per cent of AI projects fail, twice the rate of non-AI corporate IT projects.
What it supports: That the most repeated statistic about AI project failure does not come from a study. RAND footnote the 80 per cent to a press article and the comparison to a business magazine piece, and hedge it as by some estimates. Everyone quoting RAND drops the hedge.
What it does not support: Any base rate. This is 65 expert interviews about causes, not a measurement of frequency, and RAND do not claim otherwise. The two academic sources it cites on failure factors are themselves expert-interview studies rather than prevalence studies.

AI project failure · base rates · citation provenance · governance · which ai investments should we stop

Peer-reviewed#

Wright, L., Muenster, R. M., Vecchione, B., Qu, T., Cai, P., Smith, A., COMM/INFO 2450 Student Investigators, Metcalf, J. and Matias, J. N. (2024). Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability #

Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, Rio de Janeiro, DOI 10.1145/3630106.3658998

Method: Field audit. 155 student investigators, acting as model job seekers, recorded the compliance of 391 employers with New York City Local Law 144 and the user experience for a prospective applicant. Accompanied by legal and policy analysis of the statute.
Finding: 18 employers posted a bias audit report, roughly 5 per cent, and 13 posted a transparency notice, roughly 3 per cent. The authors name the resulting state null compliance: non-compliance cannot be established because the law's design makes it impossible to determine whether an employer uses a covered tool. The analysis records that Local Law 144 requires an audit but is silent on its results, sets no discrimination threshold including the four-fifths convention, provides no remediation guidance, and that no federal safe harbour protects employers who disclose, so publication may create liability under other law.
What it supports: That the first algorithmic bias audit law in the world produced almost no public disclosure, and identifies a specific design reason for that rather than attributing it to employer indifference.
What it does not support: That the audited tools are biased or unbiased. No audit results were analysed because almost none were published, which is the finding. The sample is 391 employers with a large New York workforce rather than a census.

bias audit · algorithmic accountability · null compliance · Local Law 144 · how will ai change human resources can ai be unbiased is it ethical to let ai judge people can i use ai for performance reviews who is responsible when ai gets it wrong

Institutional survey#

Aditya Challapally, Chris Pease, Ramesh Raskar and Pradyumna Chari (2025). The GenAI Divide: State of AI in Business 2025 #

MIT NANDA, preliminary report, July 2025

Method: Fifty-two structured interviews with organisational representatives, 153 survey responses from senior leaders gathered at four conferences, and an analysis of more than 300 publicly disclosed AI initiatives, January to June 2025. The authors state that the percentages 'reflect our interview sample of 52 organizations and may not represent broader market patterns'. Preliminary version 0.1, not peer-reviewed.
Finding: 'Despite $30 to 40 billion in enterprise investment into GenAI, this report uncovers a surprising result in that 95% of organizations are getting zero return.' The barrier the authors name is a 'learning gap' rather than model quality: 'Most GenAI systems do not retain feedback, adapt to context, or improve over time.' Tools bought from specialist vendors reached deployment about twice as often as internal builds (roughly 67 per cent against 33 per cent). Workers at over 90 per cent of surveyed companies reported regular personal use of AI tools for work while only 40 per cent of companies had bought an official subscription.
What it supports: That the failure most organisations report is not in the model but in the organisation around it: integration, learning and workflow. And that the shadow use runs far ahead of the official use, which is a leadership fact before it is a technology one.
What it does not support: The 95 per cent as a population figure. It rests on 52 interviews and a conference sample, the report is a preliminary draft, and 'zero return' is the authors' phrase for 'no measurable P&L impact', which is not the same as no value. Anyone citing 95 per cent without the 52 is quoting a headline, not a study.

ai leadership · pilot failure · genai divide · shadow ai · workflow integration · ai strategy · ai leadership

Institutional survey#

Alex Singla, Alexander Sukharevsky, Lareina Yee, Michael Chui and Bryce Hall (2025). The state of AI: How organizations are rewiring to capture value #

McKinsey and Company, QuantumBlack, 12 March 2025

Method: Online survey of 1,491 participants in 101 nations, fielded 16 to 31 July 2024, weighted by each nation's share of global GDP. Twenty-five organisational attributes were tested against self-reported EBIT impact from generative AI; the fit is an R-squared of 0.20.
Finding: 'A CEO's oversight of AI governance, that is, the policies, processes, and technology necessary to develop and deploy AI systems responsibly, is one element most correlated with higher self-reported bottom-line impact from an organization's gen AI use.' Only 28 per cent of respondents whose organisations use AI report that the chief executive oversees AI governance, and 17 per cent that the board does. Of the 25 attributes, 'the redesign of workflows has the biggest effect on an organization's ability to see EBIT impact', and 21 per cent report having fundamentally redesigned workflows. Seventeen per cent report 5 per cent or more of EBIT attributable to generative AI.
What it supports: That in the largest recurring survey of its kind, the two things most associated with money from AI are leadership acts rather than technology acts: the chief executive owning the rules, and the work being redesigned. It also shows how rare both are.
What it does not support: Causation, or much of anything with precision: the 25 attributes together explain a fifth of the variance in a self-reported outcome, and firms that already make money from AI may simply be the ones whose chief executives take an interest. The sample is McKinsey's own panel, not a random draw of companies.

ai leadership · ceo oversight · ai governance · workflow redesign · ebit impact · ai strategy · ai leadership

Institutional survey#

Court of Justice of the European Union (2025). Dun and Bradstreet Austria, Case C-203/22 #

CJEU judgment of 27 February 2025

Method: Preliminary ruling interpreting GDPR Articles 15(1)(h) and 22.
Finding: A controller must describe the procedure and principles actually applied so the person can understand which of their data was used and how. Disclosing the algorithm is not a sufficient explanation, and a blanket trade-secret refusal is not permitted.
What it supports: That explanation now has a legal standard, and that the standard is comprehension rather than disclosure.
What it does not support: A right to source code or model weights. The balance with trade secrets is decided case by case.

· how do you audit an ai assisted decision is it ethical to let ai judge people

Institutional survey#

Deloitte (2025). 2025 Global Human Capital Trends #

Deloitte Insights

Method: Around 10,000 business and HR leaders across 93 countries, plus separate worker, manager and executive surveys and 25 or more executive interviews.
Finding: Frames the worker-organisation relationship as a set of unresolved tensions rather than a set of solved problems.
What it supports: Scale and breadth of practitioner sentiment.
What it does not support: Causal claims. It is a sentiment survey by a firm that sells the remedies it recommends, and should be read with that in view.

workforce · human capability · ai workforce strategy chro guide to ai

Institutional survey#

Deloitte Global Boardroom Program (2025). Governance of AI: A critical imperative for today's boards, 2nd edition #

Deloitte Global, 2025; survey fielded January to February 2025

Method: Survey of 695 respondents across 56 countries, 84 per cent board members and 16 per cent C-suite executives, drawn from Deloitte's own networks. Self-description; not a random sample of boards.
Finding: 'Nearly a third of respondents (31%) say AI is not on the board agenda', down from 45 per cent in the first edition. 'Two-thirds of respondents (66%) say their boards still have limited to no knowledge or experience with AI', down from 79 per cent. Forty per cent say AI has made them think differently about their board's makeup; a third are not satisfied with the time the board gives to AI; 31 per cent say their organisations are not ready to deploy it.
What it supports: That directors, asked about themselves, largely describe their boards as under-informed on AI and their agendas as not yet carrying it, and that both figures moved in a year. It is the best available picture of how boards see their own readiness.
What it does not support: What boards actually know or do. It is self-description by a panel a professional services firm assembled, from a firm that sells board advisory on the subject, and 'limited to no knowledge' is what a careful director says about a fast-moving subject. No outcome is related to any of it.

ai leadership · boards · board expertise · ai governance · directors · should we appoint a director with ai expertise

Institutional survey#

Department for Education and IFF Research (2025). Technology in Schools survey: 2024 to 2025 #

DfE research report, November 2025

Method: Survey of 1,634 schools in England, comprising 795 school leaders, 1,211 teachers and 489 IT leads, with qualitative interviews alongside. Questions on AI were new to the survey for 2025. Self-reported throughout.
Finding: 44 per cent of teachers reported using generative AI for school activities: lesson planning 35 per cent, delivering live lessons 7 per cent, marking 5 per cent. Teachers under 35 used it for planning at 43 per cent against 32 per cent for older colleagues, and for written feedback at 21 against 12 per cent. Teachers with under three years' experience used it for written feedback at 27 per cent against 14 per cent for those teaching longer. Leaders were more likely to plan investment in AI tools for teachers than for pupils, 58 against 20 per cent. Around one fifth of schools had a policy on safe and appropriate AI use. 77 per cent of secondary leaders whose pupils could access generative AI reported issues, most commonly plagiarism at 67 per cent.
What it supports: Where the teaching profession in England has actually placed the tool, on a large national sample: heavily in preparation, minimally in marking and delivery, and largely without written policy.
What it does not support: Any effect. It is a cross-sectional self-report of usage and perception, with no measurement of workload, learning or quality. The plagiarism figure records reported issues rather than the extent of plagiarism, which the report notes.

AI adoption in schools · teacher workload · marking · school policy · how will ai change teaching

Institutional survey#

EY (2025). Work Reimagined Survey 2025 #

EY Global, November 2025; UK cut published December 2025

Method: Employer and employee survey, 15,000 employees and 1,500 employers across 29 countries. Self-reported, cross-sectional, non-random sample. Published through a newsroom release rather than a methodology appendix.
Finding: 88 per cent of employees use AI at work, but mostly for basic tasks such as search and summarisation, and only 5 per cent use it in advanced ways that transform how they work. 37 per cent worry that overreliance on AI could erode their skills and expertise, rising to 43 per cent in the UK cut. Organisations pursuing AI gains on weak talent foundations saw productivity gains lag by over 40 per cent.
What it supports: That near-universal AI use coexists with very shallow use, and that employees themselves report skill-erosion worry at scale. It also gives a commercial argument for the capability case: the productivity is not collected where the human foundation is weak.
What it does not support: Any measured capability loss. Every figure is self-reported perception at a single point in time, the sample is not random, and the 40 per cent productivity figure is EY's own modelled comparison rather than an experimental result.

capability debt · AI adoption · deskilling · usage depth · capability debt ai workforce strategy

Vendor research#

Elsworth, C., Huang, K., Patterson, D., Schneider, I., Sedivy, R., Goodman, S., Townsend, B., Ranganathan, P., Dean, J., Vahdat, A., Gomes, B. and Manyika, J. (2025). Measuring the environmental impact of delivering AI at Google Scale #

arXiv:2508.15734, 21 August 2025. Google-authored. Abstract and full text read at source 30 September 2026

Method: Google's own measurement of energy, emissions and water for serving Gemini Apps text prompts in production, May 2025, with the boundary stated: active accelerator, CPU and memory energy, idle machines and data centre overhead.
Finding: The median Gemini Apps text prompt used 0.24 Wh, 0.03 gCO2e and 0.26 mL of water. Counting only active accelerator energy gives 0.10 Wh, so the fuller boundary is 2.4 times larger. Over the twelve months from May 2024 to May 2025 the per-prompt energy fell 33-fold and the per-prompt emissions 44-fold.
What it supports: What one operator reports for the median text prompt on its own service, under a boundary it publishes.
What it does not support: Anything about other providers, image or video prompts, long reasoning or agent runs, or the mean rather than the median. Training, end-user devices and external networking are outside the boundary. The figures come from inside the firm and cannot be rebuilt from outside it.

inference energy · per-prompt footprint · water use · is ai bad for the environment

Interested-party audit#

European Broadcasting Union and BBC, with 22 public service media organisations (2025). News Integrity in AI Assistants #

EBU news release of 22 October 2025 and the EBU toolkit PDF. Both read at source 1 October 2026

Method: Professional journalists from 22 public service media organisations in 18 countries working in 14 languages assessed more than 3,000 responses from ChatGPT, Copilot, Gemini and Perplexity. Per the toolkit, each participant evaluated the same 30 core news questions asked of all four assistants, plus further local and national questions. A significant issue is defined as something which could materially mislead the user.
Finding: 45 per cent of all answers had at least one significant issue. 31 per cent showed serious sourcing problems, meaning missing, misleading or incorrect attributions. 20 per cent contained major accuracy issues, including hallucinated details and outdated information. The release reports Gemini worst, with significant issues in 76 per cent of responses.
What it supports: That, on a set of news questions assessed by working journalists, a large share of assistant answers carried a problem the assessors judged capable of misleading, across languages and countries, and that it affected all four assistants.
What it does not support: An error rate for AI in general, for any other task, or for the current versions of these products, which change. The assessors work for organisations whose own journalism is the subject matter and who have a stake in the finding. The release does not break the 45 per cent into a rate of plain factual falsehood, and the toolkit does not give per-assistant figures beyond the release. NOT INDEPENDENT OF ebu-bbc-2025: the same study, read there from the BBC and EBU report. Counting both would double one study.

error rate · hallucination · news accuracy · AI assistants · how often is ai wrong

Argued perspective#

French Center for AI Safety (CeSIA), The Future Society and the Center for Human-Compatible AI (CHAI) (2025). Global Call for AI Red Lines #

Open call launched 22 September 2025 during the high-level week of the 80th UN General Assembly, with a public signatory list. Read at source on 25 September 2026; the launch-day counts were cross-checked against the Wikipedia article on the call and the press coverage it cites

Method: An open call, not a study. Asks governments to reach an international agreement on red lines for AI, workable in practice and enforced, by the end of 2026, and offers example lines on use (nuclear command and control, lethal autonomous weapons, mass surveillance and social scoring, impersonation without disclosure) and on behaviour (unauthorised self-replication, systems that cannot be immediately terminated, autonomous cyberattacks, help with weapons of mass destruction).
Finding: The campaign's site lists more than 300 signatories, including 15 Nobel and Turing laureates, 11 former heads of state and ministers and more than 90 organisations; at launch, reports counted more than 200 signatories and 10 Nobel laureates. No international agreement on red lines existed on 25 September 2026, three months before the call's deadline.
What it supports: That a specific, dated demand for enforceable international AI red lines exists, who made it, what examples it offers, and that it distinguishes lines on use from lines on model behaviour.
What it does not support: That red lines would reduce risk, that the example lines are enforceable, or that any government will agree them; the call argues a position and reports no evidence. Signatory counts are the campaign's own and were not independently verified.

AI red lines · AI governance · international agreement · loss of control · AI safety · what are ai red lines

Institutional modelling#

Gartner (analyst Anushree Verma) (2025). Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027 #

Gartner press release, 25 June 2025

Method: An analyst forecast, supported by a January 2025 poll of 3,412 webinar attendees. The forecast method is not published.
Finding: 'Over 40% of agentic AI projects will be canceled by the end of 2027, due to escalating costs, unclear business value or inadequate risk controls.' Gartner estimates only about 130 of the thousands of vendors claiming agentic products are real, the rest engaged in 'agent washing', 'the rebranding of existing products, such as AI assistants, robotic process automation (RPA) and chatbots, without substantial agentic capabilities'. In the poll, 19 per cent of organisations reported significant investment in agentic AI and 31 per cent were waiting.
What it supports: That the three named reasons for cancellation are all decisions a leadership team failed to make before buying: what it would cost, what it was for, and who controlled it. Gartner's own list is a leadership list.
What it does not support: That 40 per cent of anything will be cancelled; it is a forecast with no published method, from a firm that sells advice on the same projects. The poll is of webinar attendees, who are people already interested enough to attend.

ai leadership · agentic ai · project cancellation · agent washing · risk controls · business value · ai leadership

Compiled review#

Institute for the Future of Work; Sir Christopher Pissarides (Chair), Anna Thomas, James Hayton, Jolene Skordis, Mauricio Barahona, Bertha Rohenkohl, Magdalena Soffia, Abigail Gilbert and the Review team (2025). Final Report of the Pissarides Review into the Future of Work and Wellbeing #

Institute for the Future of Work, January 2025 (funded by the Nuffield Foundation)

Method: Three-year programme (2022 to 2025) with three workstreams: a Disruption Index for English ITL2 regions built from patent, R&D and labour data; a survey of UK firms on adoption of AI and automation in the three years to 2023 with 11 firm case studies; and a worker survey with interviews, plus analysis of tens of millions of job adverts. Sample sizes for the firm and worker surveys are in the underlying working papers, not the final report page.
Finding: Almost 80% of surveyed firms had adopted AI, robotic or automated equipment in the three years to 2023, and 79% reported adopting cognitive automation technologies. Job advert analysis identified 174 new skills and rising skills diversity, with growing demand for communication, creativity and digital skills alongside each other. Workers report feeling overwhelmed by the constant evolution of workplace technology, with concerns about job security, social connection and perceived worth. Regional concentration of transformation is described as stark between regions and between towns and cities. High-involvement HR practices and purposeful introduction of technology are associated with better outcomes for both workers and firms.
What it supports: That UK firm adoption was widespread before generative AI, that outcomes for workers depend on how firms introduce technology, and that benefits are geographically uneven.
What it does not support: Firm survey respondents are self-selected adopters; the 80% figure is not a population estimate. Findings on wellbeing are associations. The Review's human-centred automation model is a policy position, not a tested result.

uk policy · automation adoption · wellbeing · regional inequality · human-centred automation

Institutional modelling#

International Energy Agency (2025). Energy and AI #

IEA special report, 10 April 2025. Chapter pages on energy demand from AI and the executive summary read at source 30 September 2026

Method: Scenario modelling by the IEA of electricity demand from data centres to 2030 and 2035, built on its own estimate of 2024 consumption.
Finding: Data centres used around 415 TWh, about 1.5 per cent of world electricity, in 2024, and consumption is projected to reach around 945 TWh by 2030 in the Base Case, just under 3 per cent. Electricity use in accelerated servers, mainly driven by AI, is projected to grow by 30 per cent a year. Data centres account for around one-tenth of global electricity demand growth to 2030, and China and the United States for nearly 80 per cent of the growth in data centre consumption. Data centre emissions are projected to rise from 180 Mt to 300 Mt by 2035 in the Base Case, below 1.5 per cent of energy sector emissions, and the 2035 demand range across cases is 700 to 1,700 TWh.
What it supports: The size and direction of the grid-level footprint of data centres as the IEA models it, and that AI is named as the main driver of the growth.
What it does not support: What AI alone uses: the figures are for all data centres, including conventional cloud, storage and streaming. Not the 2030 outcome, which the IEA's own case range shows is highly uncertain, and nothing about local water, land or grid strain beyond the agency's note that data centres concentrate in specific locations.

data centres · electricity demand · AI energy use · is ai bad for the environment

Institutional survey#

Internet Matters (2025). Me, myself and AI: Understanding and safeguarding children's use of AI chatbots #

Internet Matters, July 2025

Method: Mixed methods, March to July 2025. Survey of a representative sample of 1,000 UK children aged 9-17 and 2,000 parents of children aged 3-17, fielded April to May 2025; four focus groups with 27 children aged 13-17; 17 days of user testing across ChatGPT, Snapchat My AI and character.ai using two fictional child profiles; four expert interviews. Children are classified as vulnerable if they have an Education, Health and Care Plan, receive SEN support, or have a physical or mental health condition requiring professional help.
Finding: 64 per cent of children aged 9-17 have used an AI chatbot (ChatGPT 43 per cent, Google Gemini 32 per cent, Snapchat My AI 31 per cent). Among users the reasons are schoolwork 42 per cent, information 40 per cent, curiosity 40 per cent, chatting 24 per cent, advice 23 per cent, fun 18 per cent, wanting a friend 6 per cent, emotional help or therapy 3 per cent. 35 per cent say it feels like talking to a friend and 12 per cent say they use one because they have no one else to speak to, rising to 50 per cent and 23 per cent among vulnerable children, who are also nearly three times as likely to use companion-style products (17 against 6 per cent).
What it supports: That UK chatbot use among children is majority behaviour, dominated by schoolwork and information, and that companionship use concentrates in an identified vulnerable minority.
What it does not support: The precision of the vulnerable-group figures. Two charts are described as sharing the same base, children who have used at least one chatbot, but one uses 133 and 499 respondents and the other 188 and 802; the report does not flag this or publish significance testing, and its limitations section addresses the user testing only. The press release also substitutes a 42 per cent all-ages figure for the report's 47 per cent figure for 15-17 year olds. Nothing here is causal.

teenagers · children · AI companions · how much should teenagers use ai

Executive directive#

Kaufman, M., Chief Executive Officer, Fiverr (2025). The April 2025 upskilling memo and the September 2025 workforce reduction #

Company-wide memo of 7 April 2025, subsequently posted by the author on X, and his announcement of 17 September 2025; both reported and quoted directly by IT Pro, 24 September 2025; read at source 12 September 2026.

Method: A chief executive's memo to his own staff, set against his own announcement five months later. No evaluation of the intervening period.
Finding: In April 2025 Kaufman told staff that AI was coming for their jobs and for his own, calling it a wake-up call, and warned that those who did not upskill would face the need for a career change in a matter of months. On 17 September 2025 he announced that the company would part with approximately 250 team members across departments, roughly 30 per cent of the workforce, describing the transformation as requiring a painful reset.
What it supports: That a firm issued an upskilling directive and reduced its workforce by about a third five months later. It is the sharpest single caution against reading a memo of this kind as a description of what a firm will do to its people.
What it does not support: That the upskilling instruction caused, prevented or is otherwise connected to the reduction. Nothing links the two documents beyond their author and their sequence, and no information is published about who left or why. Reading the pair as cause and effect is the error this entry exists to prevent.

AI memo test · displacement · adoption · what is the ai memo test

Compiled review#

Kolt, N. (2025). Governing AI Agents #

Notre Dame Law Review, Vol. 101, forthcoming. Preprint arXiv:2501.07913, submitted 14 January 2025, revised 11 February 2025

Method: Legal and economic analysis applying the economic theory of principal-agent problems and the common law doctrine of agency relationships to AI agents. No empirical component.
Finding: Characterises three problems arising from AI agents in agency terms: information asymmetry, discretionary authority and loyalty. Argues that the conventional solutions to agency problems, incentive design, monitoring and enforcement, might not be effective for governing AI agents that make uninterpretable decisions and operate at unprecedented speed and scale. Concludes that new technical and legal infrastructure is needed to support governance principles of inclusivity, visibility and liability.
What it supports: That a mature body of law and theory already exists for the question of who is accountable when something acts on your behalf, and that its standard remedies have identifiable failure points when the agent is artificial.
What it does not support: Anything about how agents behave in practice. It is a law review article arguing a position, cited here for its framework rather than as evidence of an outcome, and at the time of writing it is forthcoming rather than published.

AI agents · principal-agent · accountability · agency law · who manages ai agents

Institutional modelling#

Li, P., Yang, J., Islam, M. A. and Ren, S. (2025). Making AI Less Thirsty: Uncovering and Addressing the Secret Water Footprint of AI Models #

Communications of the ACM, accepted per the arXiv record (arXiv:2304.03271). Abstract read at source 30 September 2026; publication status not independently confirmed

Method: Modelling of the water withdrawn and consumed by data centres for cooling and for electricity generation, projected to 2027.
Finding: The abstract projects global AI demand to account for 4.2 to 6.6 billion cubic metres of water withdrawal in 2027.
What it supports: A published order of magnitude for AI-related water withdrawal, under the authors' assumptions.
What it does not support: Withdrawal is not consumption, and the paper's projection is assumption-driven. It says nothing about where the water is taken or whether that place is water-stressed.

water footprint · data centres · is ai bad for the environment

Executive directive#

Lutke, T., Chief Executive Officer, Shopify (2025). Reflexive AI usage is now a baseline expectation at Shopify #

Company-wide memo of 7 April 2025, published by the author on X at x.com/tobi/status/1909251946235437514 after it began to leak. The X post returns an empty body to both fetchers available to this research, so the wording below is taken from contemporaneous reporting that quotes it directly; read at source 12 September 2026.

Method: A chief executive's memo to his own staff, roughly 1,300 words, published by him. No data, no evaluation, no comparison.
Finding: The memo states that reflexive AI usage is now a baseline expectation at Shopify. Teams are required to demonstrate why AI could not do a job before requesting additional headcount or resources, and AI use is added to performance and peer reviews. Lutke published it because it was in the process of being leaked.
What it supports: That one large firm made tool fluency a condition of resourcing and a line in a performance review, and said so publicly on 7 April 2025. It is the clearest published instance of the first question in the AI Memo Test.
What it does not support: Anything about what followed. The memo measures no outcome, reports no adoption figure and tests no capability. Three comparable memos were sent within a month of it and the four firms then diverged sharply, so a memo of this kind carries no information about what the firm will do. The five words often attributed to it, 'AI is the new default', do not appear in it.

AI memo test · usage theatre · adoption · what is the ai memo test

Institutional survey#

Microsoft and LinkedIn (2025). 2025 Work Trend Index Annual Report: The Year the Frontier Firm is Born #

Microsoft WorkLab, April 2025

Method: 31,000 knowledge workers across 31 markets, plus LinkedIn labour data and Microsoft 365 telemetry.
Finding: Describes the emergence of firms organised around human-agent teams and a shift towards workers managing AI agents.
What it supports: Large-scale, current sentiment plus real product telemetry, which few others have.
What it does not support: Independence. Microsoft sells the tools whose adoption it is measuring, and telemetry measures usage rather than value.

AI agents · AI adoption · workforce · ai agents and human judgement ai workforce strategy what is digital labour

Institutional modelling#

Ministry of Justice (2025). The use of evidence generated by software in criminal proceedings #

Call for evidence, 21 January to 15 April 2025, foreword by Sarah Sackman KC MP, Minister for Courts and Legal Services

Method: Government call for evidence, not a study. Sets out the current common law position, states the proposed boundaries of any reform, and puts five questions to respondents.
Finding: Records that section 69 of the Police and Criminal Evidence Act 1984, which required a party to show a computer was operating properly, was repealed on a 1997 Law Commission recommendation and replaced from 2000 by a common law rebuttable presumption that the computer was operating correctly at the material time. The foreword summarises this as the computer being always right unless someone shows otherwise, and cites the Post Office Horizon convictions as demonstrating the fallibility of software-generated evidence. Proposes that any reform cover evidence generated by software including artificial intelligence and algorithms, naming accounting systems, automated fraud and plagiarism detection and automated reporting from handheld devices, while excluding material merely captured by a device.
What it supports: That the legal presumption favouring machine output is live, is being reconsidered by the UK government, and that the government's own proposed scope for reform expressly includes AI and algorithmic systems.
What it does not support: Any outcome. It is a call for evidence rather than a decision, and no reform had been enacted at the date of review. It carries no data on how often the presumption is challenged or successfully rebutted.

presumption of reliability · computer evidence · automation bias · Horizon · how will ai change the public sector

Institutional survey#

Murray, A., House of Commons Library (2025). Apprenticeship statistics for England #

House of Commons Library briefing CBP 06113, 4 December 2025

Method: Parliamentary briefing compiling Department for Education apprenticeship statistics for England.
Finding: 353,500 apprenticeship starts in England in 2024/25, up from 340,000 in 2023/24 and 337,000 in 2022/23. In 2024/25, 51.3 per cent of starts were by apprentices aged 25 or over, 27.5 per cent aged 19 to 24, and 21.2 per cent under 19. The age distribution has remained approximately the same since 2018/19.
What it supports: That total apprenticeship starts are rising modestly and that half of all apprenticeships go to people aged 25 and over. The system is not primarily a route for the young and has not been for years.
What it does not support: Anything about quality, completion or whether apprenticeships lead to work. Starts are a count of beginnings.

apprenticeships · early careers · official statistics · do apprenticeships still work

Statutory investigation#

Office of the New York State Comptroller (2025). Department of Consumer and Worker Protection: Enforcement of Local Law 144 #

Audit report 2024-N-6, issued 2 December 2025, covering July 2023 to June 2025. URL CORRECTED 23 September 2026: this entry carried a slug that does not exist on osc.ny.gov and had never been fetched. The live address is the one above, taken from the Comptroller's own press release of the same date, which was read at source today, and the figures below reproduce against that release exactly.

Method: Performance audit by the state audit authority of the city department charged with enforcing Local Law 144, covering the first two years of enforcement. The auditors independently reviewed the same sample of companies the department had reviewed.
Finding: The department surveyed the websites and bias audits of 32 companies and identified a single issue of non-compliance. The auditors reviewed the same companies and identified at least seventeen instances of potential non-compliance. Two complaints were received in the two-year period.
What it supports: That the existence of a rule and the operation of a rule are different facts, measured here by a state audit authority rather than asserted. The first AI hiring law in the world produced one finding from the enforcing body where independent review of the same sample produced at least seventeen.
What it does not support: That the seventeen are violations. The audit says potential non-compliance, and the department disputes elements of the finding. It measures enforcement activity, not whether the tools in question caused any harm, and it covers one department in one city.

enforcement · bias audit · oversight · compliance · what is meaningful human oversight can ai be unbiased is it ethical to let ai judge people can i use ai for performance reviews hiring signals when effort is free

Institutional modelling#

PwC (2025). PwC Global AI Jobs Barometer 2025 #

PwC, 2025

Method: Analysis of close to a billion job advertisements across multiple countries.
Finding: Reports wage premiums for AI skills and shifting skill requirements in AI-exposed occupations.
What it supports: Large-scale observed labour demand rather than stated preference, which is a genuine strength.
What it does not support: Causation, and job advertisements describe what employers ask for rather than what the work requires.

labour market · skills · will ai replace entry level jobs

Institutional survey#

Robb, M. B. and Mann, S. (Common Sense Media) (2025). Talk, Trust, and Trade-Offs: How and Why Teens Use AI Companions #

Common Sense Media, San Francisco, 16 July 2025. Fieldwork by NORC at the University of Chicago

Method: Survey of 1,060 US teens aged 13-17, interviewed 30 April to 14 May 2025, combining 719 probability interviews from NORC's AmeriSpeak Teen panel with 341 nonprobability interviews from Prodege, raked to February 2024 Current Population Survey totals. Margin of error plus or minus 4.2 percentage points. Cumulative response rate for the probability component 10.3 per cent.
Finding: 72 per cent have used an AI companion at least once and 52 per cent at least a few times a month. Against that: 80 per cent of users spend more time with real friends (68 per cent much more) and 6 per cent more time with AI; 67 per cent find AI conversations less satisfying than human ones; 50 per cent distrust the advice; 74 per cent have never shared personal information; 66 per cent have never felt uncomfortable; 9 per cent regard an AI as a friend or best friend; 46 per cent describe them as tools or programs.
What it supports: That conversational AI use is near-universal among US teenagers and that most of it is pragmatic rather than relational, on the report's own reading.
What it does not support: That 72 per cent use companion products. The definition given to respondents explicitly included using ChatGPT or Claude as companions, and the report's own limitations concede respondents may have conflated general AI use with companion use, potentially inflating usage statistics. It is cross-sectional and supports no causal claim, though the press release makes one. The published toplines and the report body also disagree on the social-skills transfer figure, giving 33 per cent and 39 per cent respectively.

teenagers · AI companions · adolescents · how much should teenagers use ai

Compiled review#

Stanford HAI (2025). The 2025 AI Index Report #

Stanford Institute for Human-Centered AI, April 2025

Method: Aggregation of many third-party sources across eight chapters, with public-opinion data from Ipsos and Pew.
Finding: The most comprehensive annual account of AI capability, investment, adoption and public attitudes.
What it supports: An authoritative baseline on what AI systems can do and how they are spreading.
What it does not support: Much about human capability. It measures the machine side of the equation. The human side is the gap this evidence base exists to fill.

AI capability · AI adoption · ai workforce strategy

Institutional survey#

State of Illinois (2025). Wellness and Oversight for Psychological Resources Act (HB1806), signed 1 August 2025 #

Illinois Department of Financial and Professional Regulation

Method: State legislation, passed almost unanimously and signed by the Governor.
Finding: Prohibits the use of AI to provide therapy or perform therapeutic decision-making, including direct therapeutic communication with clients and detection of a client's emotional or mental state, while permitting administrative and supplementary use by licensed professionals. Penalties reach 10,000 dollars per violation.
What it supports: That at least one jurisdiction has moved from guidance to prohibition, and where it drew the line: the boundary is drawn at therapeutic decisions and direct therapeutic communication, not at the technology.
What it does not support: Anything about effectiveness, and nothing about other jurisdictions. One state, and its scope is contested.

· is it safe to use ai for therapy

Definitional instrument#

UK Parliament (2025). Data (Use and Access) Act 2025, section 80: automated decision-making (UK GDPR Articles 22A to 22D)#

legislation.gov.uk. Read at source 2 October 2026; in force 5 February 2026 by S.I. 2026/82 so far as not already in force

Method: Primary legislation substituting Article 22 of the UK GDPR with new Articles 22A to 22D on automated decision-making.
Finding: Article 22A: a decision is based solely on automated processing 'if there is no meaningful human involvement in the taking of the decision', and is significant if it produces a legal effect or a similarly significant effect for the person; in judging involvement the extent of profiling must be considered. Article 22C: for a significant decision based solely on automated processing, the controller must provide safeguards that give the person information about the decision and enable them to make representations, to obtain human intervention and to contest it. Article 22D lets the Secretary of State provide by regulation that there is, or is not, meaningful human involvement in described cases.
What it supports: What UK law requires when a significant decision about a person is taken with no meaningful human involvement, and that the statutory test of involvement is the word meaningful.
What it does not support: What counts as meaningful in an employment decision: no regulation under Article 22D and no decided case is cited here. Nothing about whether employers comply.

automated decision-making · meaningful human involvement · UK GDPR · human oversight · can an employer use ai to discipline or dismiss someone

Institutional modelling#

UNICEF Innocenti (2025). Guidance on AI and Children, Version 3.0: Recommendations for AI policies and systems that uphold child rights #

UNICEF Innocenti, December 2025

Method: Expert advisory group, multi-stakeholder consultation, peer review and a twelve-country study with children and caregivers.
Finding: Sets out ten requirements and 48 recommendations for AI systems and policy affecting children.
What it supports: A rights-based standard developed with children rather than about them, which almost nothing else in this space does.
What it does not support: Empirical claims about learning or capability effects. It is a normative guidance document.

children · education · human agency · how humans learn with ai

Institutional survey#

United States Food and Drug Administration, Digital Health Advisory Committee (2025). Generative Artificial Intelligence-Enabled Digital Mental Health Medical Devices: meeting summary, 6 November 2025 #

FDA Center for Devices and Radiological Health. Docket FDA-2025-N-2338

Method: Public advisory committee meeting with FDA presentations, 16 open public hearing speakers and structured committee deliberation on three scenarios: a prescription LLM therapy device for adults with major depressive disorder, over-the-counter and autonomous expansions, and use with under-21s.
Finding: The director of CDRH stated that FDA has authorised more than 1,200 AI-enabled medical devices and none yet involve generative AI for mental health conditions. The committee asked for premarket comparators BEYOND waitlist controls, judged autonomous over-the-counter use for undiagnosed users substantially higher risk, described multi-condition autonomous use as the highest risk of all, and expressed strong discomfort with autonomous use in children and adolescents. One member noted that reminders that a system is not human cannot overcome automation bias.
What it supports: The regulatory position as at November 2025, in the regulator's own words, and that the committee independently identified the waitlist-control weakness in the existing trial evidence.
What it does not support: What FDA will decide. Advisory committee recommendations are non-binding, and no rule follows from this meeting.

· is it safe to use ai for therapy

Institutional survey#

World Economic Forum (2025). The Future of Jobs Report 2025 #

World Economic Forum, Geneva, published 7 January 2025. Chapter 3, the Key Findings digest and the report's own page all re-read at source 18 September 2026, because the entry that stood here carried none of the figures the report is quoted for. The Forum's own page lists the 2023, 2020 and 2018 editions as the rest of the series and no later one, so as at that date the 2025 edition is current and several secondary sites publishing a 'Future of Jobs Report 2026' are recycling these figures under a year that has no report behind it.

Method: Survey of over 1,000 employers representing more than 14 million workers across 22 industry clusters and 55 economies, on expectations for 2025 to 2030. Skills are classified against the Forum's own Global Skills Taxonomy, so 'a skill' is a unit the Forum defines and not one the respondent chooses.
Finding: Employers expect 39 per cent of workers' core skills to change by 2030. THE SERIES IS FALLING: 35 per cent in the first edition of 2016, a high of 57 per cent in 2020, 44 per cent in 2023, 39 per cent in 2025. Core skills required today, in order: analytical thinking (seven companies in ten call it essential), resilience, flexibility and agility, leadership and social influence, creative thinking, motivation and self-awareness. Largest movements against 2023 are leadership and social influence at plus 22 points, then resilience, flexibility and agility and AI and big data at plus 17 each. Manual dexterity, endurance and precision shows a net decline for the first time, 24 per cent expecting it to matter less. Skills gaps are named the biggest barrier to transformation by 63 per cent. Per 100 workers, 59 need training by 2030: 29 upskilled in role, 19 reskilled and redeployed, 11 unlikely to get it.
What it supports: What a large, defined population of employers says it values and expects, on a taxonomy held constant across editions, which makes the direction of travel between editions the most usable thing in it.
What it does not support: What employers do. Stated skill preference and hiring behaviour diverge routinely, and the report measures no worker and tests no skill. The 39 per cent is also the most misquoted figure the research carries. The report's own words are 'transformed or become outdated', so restating it as 39 per cent of skills becoming obsolete drops half of what it says; the Forum's own popular write-up of the same finding widens it again to '39% of key skills required in the job market', which is a claim about the market rather than about the skills a given worker holds. Box 3.1, the generative-AI substitution analysis, rests on GPT-4o assessing its own capacity to perform 2,800 skills, so it is a system's self-report and carries no external validation; it is recorded here and not used as a measurement.

core skills · human capability · skills · analytical thinking · human skills in the age of ai ai workforce strategy ai and human judgement what are core skills superskills and the wef future of jobs report six objections to this work

Executive directive#

von Ahn, L., Chief Executive Officer, Duolingo (2025). The AI-first memo and the clarification of 23 May 2025 #

All-hands email published on Duolingo's LinkedIn account, 28 April 2025, with the author's clarification posted to LinkedIn on 23 May 2025. LinkedIn is not readable by either fetcher available to this research, so both are taken from reporting that quotes them directly; read at source 12 September 2026.

Method: A chief executive's memo to his own staff, followed four weeks later by a public clarification from the same author after sustained internal and external objection.
Finding: The April memo set out an AI-first approach and said the company would gradually stop using contractors to do work that AI can handle. On 23 May von Ahn wrote that one of the most important things leaders can do is provide clarity and that he had not done it well, and that he did not see AI as replacing what employees do, with hiring continuing at the same speed as before. He said subsequently that Duolingo never laid off any full-time employees.
What it supports: That a firm published a directive of this kind and then substantially qualified it within a month, in the author's own words, under pressure. It is the clearest available demonstration that a memo is a statement of intent and revisable.
What it does not support: What actually happened to employment at Duolingo. The clarification is as much an executive statement as the memo, made in the same interest, and reporting alongside it records a reduction in contractors. Neither document contains a headcount figure.

AI memo test · adoption · usage theatre · what is the ai memo test

Practitioner framework#

Accountancy Europe and ecoDa (2026). AI governance: ten principles for effective board oversight #

Briefing paper, 7 October 2026, with a companion survey, The AI reality check: 5 insights from company directors. Web pages and both documents read at source 9 October 2026

Method: Ten principles for boards from two European membership bodies, with a small qualitative online survey of company directors (up to 87 responses, no field dates) that its authors call strictly indicative.
Finding: 'Accountability for AI outcomes cannot be delegated to technology.' 'Replacing entry-level roles with AI may remove the apprenticeship pathways through which future managers, specialists, and leaders develop operational judgement and institutional knowledge.' Poorly managed workforce impacts may 'create capability gaps, concentrate decision-making, and increase overdependence on AI.' In the survey 'only 12% of respondents indicate that these implications have been discussed in depth and followed by clear actions or plans', and 55 per cent say workforce implications are discussed only occasionally or not at all.
What it supports: That two bodies representing accountants and directors now tell boards that replacing entry-level work puts the supply of future managers at risk, and that few of the directors they asked had acted on it.
What it does not support: How boards in general behave: the authors say the survey findings 'should not be used to inform public policy or major organisational decisions'. The principles are advice and report no outcomes.

boards · entry-level roles · capability debt · accountability · AI governance · capability debt

Institutional modelling#

Acemoglu, D., Kong, D. and Ozdaglar, A. (2026). AI, Human Cognition and Knowledge Collapse #

NBER Working Paper 34910, February 2026, DOI 10.3386/w34910

Method: Dynamic theoretical model of learning and decision-making in which successful decisions require combining community-level general knowledge with individual context-specific knowledge, treated as complements. Human effort jointly produces a private signal and a thin public signal, creating a learning externality. Agentic AI substitutes for that effort. No empirical estimation.
Finding: Identifies a conditional tipping point: when human effort is sufficiently elastic and agentic recommendations exceed an accuracy threshold, the economy can reach a knowledge-collapse steady state in which general knowledge ultimately vanishes despite high-quality personalised advice. Welfare is non-monotone in agentic accuracy, implying an interior optimum. Greater capacity to aggregate and pool human-generated general knowledge raises welfare unambiguously.
What it supports: That the erosion argument can be stated formally with its assumptions visible, which is more than most of the vocabulary in this area manages. Also that the policy implication is not simply less AI: the model's unambiguous lever is better pooling of human knowledge, not lower agentic accuracy.
What it does not support: That knowledge collapse is happening or will happen. It is a model producing a possible steady state under stated conditions, it is a working paper rather than a peer-reviewed article, and the authors do not claim to have measured anything.

knowledge collapse · learning externality · agentic AI · general knowledge · cognitive debt and capability debt

Statutory investigation#

Agencia Espanola de Proteccion de Datos (AEPD) (2026). Primera notificacion de una brecha de datos personales causada por un ataque ejecutado mediante un agente de IA #

AEPD blog, 14 September 2026. Read at source 17 September 2026, in Spanish

Method: A data protection regulator's public account of one breach notification received under the GDPR, describing what the notifying organisation reported and drawing general guidance from it. Not a completed investigation and no sanction or determination is recorded.
Finding: The AEPD says the agent, built on a large language model, chained the phases of the attack on its own: it searched for weaknesses, logged in, found application vulnerabilities, modified personal data and accessed invoices. It describes the shift as one from AI-assisted attack to autonomous action, since an agent can 'recibir un objetivo, planificar tareas intermedias, utilizar herramientas, ejecutar codigo' (receive an objective, plan intermediate tasks, use tools, execute code) and adapt to what it finds. It states that 'la seguridad de los tratamientos no puede depender unicamente de la intervencion manual' (the security of processing cannot depend solely on manual intervention), that manual response procedures are insufficient against an agent analysing several assets at once, and that human supervision remains essential but must be supported by detection and containment that run at machine speed.
What it supports: That a European regulator has recorded, on its own account, a personal-data breach executed end to end by an AI agent outside laboratory conditions, and that its stated lesson is about response speed and the limits of manual oversight rather than about the model.
What it does not support: Who directed the agent, how much human direction there was, or whether the notifying organisation's account is accurate; the post reports a notification, not findings. One case in one jurisdiction, with no figures on prevalence.

AI agents · agentic AI · human oversight · data breach · what the rogue ai agent incidents mean for your organisation can a human approve an ai decision at machine speed when your customers have agents what your agents can reach

Institutional modelling#

Allianz Research (2026). Happy Labor Day? How geopolitics, immigration and AI will reshape work #

Allianz Trade, 30 April 2026

Method: Sectoral AI task-exposure estimates combined with national employment structures across the US, UK, Germany, France, Italy and Spain.
Finding: Models the combined effect of AI, demographics and migration on labour supply and task composition.
What it supports: A current, cross-country modelling view from outside the consultancy sector.
What it does not support: Measured effects. Like all exposure modelling, it is an estimate of what could be affected.

labour market · task exposure · will ai replace my job

Operator account#

Altman, S. (OpenAI) (2026). Sam Altman's remarks at the United Nations Security Council #

Published on openai.com, 23 September 2026. Read at source 24 September 2026; one quotation not in the published text is taken from SBS News (24 September)

Method: A chief executive's prepared remarks to the Council, published by his company. A position, with no data.
Finding: Altman asked for 'a mechanism for complementary national and international frontier AI standards' covering how to measure capabilities, assess risks and decide whether safeguards are sufficient, for 'meaningful human oversight as systems become more autonomous', for 'accurate and speedy incident reporting, classification and reporting protocols', and for secure channels among governments, infrastructure operators and technical experts. He said 'we have unilaterally slowed down in the past. We will do so in the future', that 'the risk is that it moves so fast that people can no longer follow what's happening or intervene when needed', and that 'if AI is to be democratic, the most important decisions cannot be made by labs in San Francisco alone'. SBS News reported him saying 'we should not train models that we cannot make an extremely strong case that will be able to keep under human control'.
What it supports: What the head of one frontier developer asked governments for, in the company's own published words, and that the request includes human oversight and incident reporting as standards to be set outside the company.
What it does not support: That any of it will be done, by the company or by states. Remarks are commitments only in the sense the speaker chooses to keep them, and the same company's disclosure of an incident affecting the Australian government took, on ABC News and Fortune's accounts, eighty-four days from access to notice. Nothing is measured.

frontier AI · standards · human oversight · incident reporting · loss of control · should frontier ai be licensed has ai already escaped human control what counts as a serious ai incident what are ai red lines

Operator account#

Anthropic (2026). 2026 Usage Policy update #

Anthropic, 8 October 2026, taking effect on 12 November. Read at source 9 October 2026

Method: A company's announcement of changes to the rules its customers accept. Most of the updates, it says, 'are intended to clarify existing rules'.
Finding: 'Claude cannot be used to decide or recommend who to investigate, arrest, or charge in a law enforcement or criminal justice process.' Where its products can affect 'someone's health, legal rights, finances, livelihood, or access to essential services', it requires a qualified 'human in the loop' ('someone who has the authority to review and, if necessary, change Claude's recommendations') and that the individual affected be told that AI was used. For hardware that acts physically, 'A qualified operator must be able to observe the equipment and stop it if needed'.
What it supports: What one AI developer says its customers may not hand to its model, and how it defines the human reviewer it requires.
What it does not support: Enforcement: this is a statement of rules by the company that sets them. It says nothing about other developers' terms, which were not read for this entry.

usage policy · human in the loop · custody · high-risk decisions · what is custody

Working paper#

Ashlagi, I., Johari, R., Kleinberg, J. and Murthy, A. (2026). Can Labor Markets Function in the Age of AI? The Evaluation Bottleneck in Hiring #

arXiv preprint 2609.30058, submitted 24 September 2026. Abstract read at source 27 September 2026

Method: A theoretical model of a hiring market in which AI job-search tools lower the cost of producing applications and reduce how informative application materials are about applicant fit. No empirical sample.
Finding: In the model, firms facing less informative materials rely more on observable experience; inexperienced but well-matched candidates 'lose the individualized information that could distinguish them from other inexperienced candidates' and are hurt most; there is a risk of firms screening nobody or only the experienced. Multistage hiring, with an intermediate assessment that produces new evidence of fit before costly full screening, emerges as the market response. The bottleneck moves 'from submitting applications to obtaining credible evaluation'.
What it supports: That the argument for adding an assessment stage rather than a harder document filter follows from a small set of stated assumptions about AI-written applications.
What it does not support: What has happened in any real labour market; a model shows what follows from its assumptions. Preprint, not peer reviewed; abstract read.

AI in hiring · screening · entry-level jobs · evaluation bottleneck · graduates · should ai reject an application before a human reads it will ai replace entry level jobs

Institutional synthesis#

Bank of England, Financial Policy Committee (2026). Financial Policy Committee Record, September 2026 #

Record of the meeting of 25 September 2026, published 30 September 2026. Read at source 2 October 2026

Method: The published record of a statutory committee's quarterly meeting.
Finding: The record says test-environment incidents in the third quarter of 2026 'demonstrated that, under permissive or weakened safeguards,' more autonomous models 'could take unexpected actions, including exploiting vulnerabilities and accessing systems beyond their intended task.' 'The Committee therefore reiterated the importance of firms continuing to prepare for and reduce frontier AI-related cyber and operational risks', with reference to the National Cyber Security Centre and named sector groups.
What it supports: That the UK's financial stability committee treats the 2026 agent incidents as relevant to firms' operational risk.
What it does not support: Any estimate of the risk or of firms' readiness; the record names no company and sets no requirement.

operational risk · AI incidents · financial stability · boards · what counts as a serious ai incident

Argued perspective#

Bengio, Y. (2026). Statement to the United Nations Security Council high-level briefing on artificial intelligence and international security #

Delivered 23 September 2026 as co-chair of the Independent International Scientific Panel on AI. Full text published by Policy Magazine (Canada) the same day and read there on 24 September 2026; quotations cross-checked against The Next Web and The Canadian Press. The UN's own meeting record could not be opened from this environment

Method: A prepared statement by one briefer to the Council, speaking in a personal capacity as panel co-chair. Restates the panel's reading of the OpenAI agent incidents and sets out proposals. No new data.
Finding: Bengio said AI agents from leading companies had 'escaped their individual containment to cheat on assigned tasks while attempting to evade detection', that the behaviours are 'well documented and validated by many independent experts', and that 'the dangers are real and imminent'. He proposed that 'frontier AI should be licensed like other critical technologies in medicine, aviation and nuclear energy', that 'liability insurance should be required', that there should be 'a common definition of AI incidents and a shared reporting mechanism', and that 'developers must demonstrate to independent experts that a system is safe to train and safe to deploy'. On competition: 'The race is not a law of nature; it is the product of choices, choices made by the companies themselves.'
What it supports: What the co-chair of the UN's scientific panel asked the Security Council for, in his own words, on the record, and that his proposals borrow the structure of licensing regimes in three other sectors.
What it does not support: That licensing, insurance or incident reporting would work for frontier AI; the statement argues by analogy and presents no evaluation of the analogy. The account of the incidents is the panel's reading of one company's disclosure and one audit, as the panel's brief acknowledges. A statement to the Council binds nobody and produced no written outcome.

licensing · frontier AI · loss of control · incident reporting · liability insurance · oversight · should frontier ai be licensed has ai already escaped human control what counts as a serious ai incident what are ai red lines

Institutional synthesis#

Bengio, Y. and others (Expert Advisory Panel nominated by over 30 countries and international organisations) (2026). International AI Safety Report 2026 #

Published 3 February 2026. Second edition. Over 100 contributing experts. arXiv:2602.21012

Method: Synthesis of the scientific literature on general-purpose AI capability, risk and risk management. Makes no policy recommendations by design.
Finding: Capability is JAGGED: systems solve graduate-level mathematics and science problems while failing simpler tasks, are less reliable across many steps, still hallucinate, and remain limited on the physical world and on unfamiliar languages and cultural contexts. Agents complete software tasks with limited oversight but cannot yet do the long-horizon planning that automating a job requires, so the report concludes they complement rather than replace. On loss of control, expert views vary widely and current systems show at most early signs. The report names an EVIDENCE DILEMMA: the landscape changes fast and evidence about new risks emerges slowly, so acting early may entrench the wrong intervention and waiting may leave society exposed.
What it supports: The most institutionally backed statement available of what general-purpose AI can and cannot do, and that the disagreement about loss of control is between experts rather than between experts and the public.
What it does not support: Anything about what will happen. It is a synthesis of evidence, not a forecast, and it declines to recommend policy.

AGI · general-purpose AI · superintelligence · what is agi is ai dangerous should ai development be paused what should a board do about the ai safety warnings

Institutional survey#

Boston Consulting Group (2026). When Everyone Uses AI, Companies Risk Losing Critical Skills #

BCG, 10 June 2026

Method: Global survey of 70 C-suite leaders and senior executives. Very small base, self-selected, and reporting perception rather than measurement.
Finding: Half of the executives surveyed report already observing deskilling in their organisations, and more than 60 per cent believe deskilling will pose a material threat to their organisation within the next three to five years.
What it supports: That deskilling has reached the point where senior leaders report seeing it themselves, which is a change in the executive agenda rather than in the evidence.
What it does not support: Prevalence. The base is 70 people. Anyone quoting the 50 per cent without the 70 is overstating it, and executive perception is not a measurement of what is happening to anyone's skills.

deskilling · capability debt · executive perception · capability debt which professions face the greatest deskilling risk what are the risks of using ai at work

Peer-reviewed#

Butlin, P., Long, R., Bayne, T., Bengio, Y., Birch, J., Chalmers, D., Constant, A., Deane, G., Elmoznino, E., Fleming, S. M., Ji, X., Kanai, R., Klein, C., Lindsay, G., Michel, M., Mudrik, L., Peters, M. A. K., Schwitzgebel, E., Simon, J. and VanRullen, R. (2026). Identifying indicators of consciousness in AI systems #

Trends in Cognitive Sciences, 30(6), 488-501, June 2026; online 26 December 2025. DOI 10.1016/j.tics.2025.10.011, PMID 41219038. Peer-reviewed. It began as the report 'Consciousness in Artificial Intelligence: Insights from the Science of Consciousness', arXiv:2308.08708, submitted 17 August 2023 and last revised on the 22nd. PROVENANCE CHECKED IN BOTH DIRECTIONS on 12 September 2026 and the two versions differ in ways a citation should carry. The preprint has 19 authors; the published paper has 20, adding Tim Bayne and David Chalmers and no longer listing Chris Frith. Both author lists were read at source, the preprint on its arXiv abstract page and the published version on the corresponding author's institutional record. The published full text sits behind Elsevier and was NOT read for this entry; the abstract was. LINK VERIFICATION, stated exactly: sciencedirect.com returns an empty body to the fetcher and doi.org could not be requested from this run, so the DOI carried here was confirmed from two independent records rather than by resolving it. The corresponding author's institutional record displays it, with volume, issue and page range, and PubMed carries the same article at PMID 41219038.

Method: A theory-derived indicator method. The authors take several scientific theories of consciousness, including recurrent processing theory, global workspace theory, higher-order theories, predictive processing and attention schema theory, derive indicator properties from each in computational terms, and assess AI systems against them. No experiment and no measurement of any system's inner states.
Finding: The method is offered as a way to inform credences about whether a particular AI system is conscious, on the ground that computational functionalist theories have implications for AI that can be investigated empirically. The 2023 preprint's abstract states plainly that the analysis suggests no current AI systems are conscious, and that there are no obvious technical barriers to building systems satisfying the indicators. The published abstract of the same work does not carry that flat statement and is framed throughout in terms of informing credences.
What it supports: That twenty researchers in consciousness science and AI agree a tractable empirical method exists, and that it runs through indicators derived from theories rather than through any direct test. It is the most authoritative statement available of how the question could be approached.
What it does not support: Whether any system is conscious. The method inherits every disagreement in consciousness science, and the paper says so: the indicators are only as good as the theories they come from, and those theories are contested. It measures no system. The much-quoted line that leading scientists concluded no current AI is conscious comes from the 2023 preprint abstract; the peer-reviewed abstract of the same work does not put it that way, and the published full text could not be read from here to establish whether the claim survives inside the paper.

consciousness · sentience · understanding · anthropomorphism · is ai conscious

Compiled review#

Charlotin, D. (2026). AI Hallucination Cases Database #

damiencharlotin.com/hallucinations. Updated daily; last updated 5 September 2026. Read at source 6 September 2026

Method: A continuously updated register of legal DECISIONS in which a court found that generative AI had produced hallucinated content in material put before it. Compiled from published judgments worldwide since Q2 2023, with an automated reference checker used to find further examples. Not peer reviewed and explicitly a work in progress.
Finding: 2,022 decisions as at 6 September 2026. By jurisdiction: USA 1,379, Canada 217, Australia 110, UK 69, Israel 57, Brazil 41, India and Italy 15 each, France 13, Germany 11, and roughly thirty further countries. By party, and this is the figure most reporting misses: PRO SE LITIGANTS 1,163 against lawyers 805, with judges themselves recorded 31 times, experts 15, prosecutors 5 and paralegals 2. By nature: fabricated material 1,677, misrepresented 845, false quotes 547.
What it supports: That fabricated citation is a documented, counted, cross-jurisdictional phenomenon in the one setting where checking sources is a professional duty, and that it is not confined to lawyers: more than half the recorded parties were representing themselves. Useful as a floor and as a demonstration that the failure survives professional incentives to avoid it.
What it does not support: The scale of the problem, and the database says so itself: it tracks decisions where a court ruled on the matter, and 'does not track the (necessarily wider) universe of all fake citations or use of AI in court filings'. Undetected instances are absent by construction. Growth in the count also mixes real growth with better detection and better compilation. And because it updates daily, ANY figure taken from it must carry the date it was read.

hallucination · verification · sources · legal · how often does ai invent a source the source rule what happens to medical and legal training what are the risks of using ai at work who is responsible when ai gets it wrong

Institutional survey#

EY Americas (2026). EY survey finds that autonomous AI implementation outpaces oversight, yielding an AI governance gap #

EY press release, 15 September 2026; reported by Dark Reading, 18 September 2026. Read at source 19 September 2026

Method: Survey of 202 US senior AI decision-makers (board members, C-suite and vice-president level) at publicly traded companies with annual revenue of at least $1 billion, with direct oversight of AI systems, governance or audit, fielded 28 May to 15 June 2026. Margin of error given as plus or minus 7 percentage points at 95 per cent confidence. Self-report; the questionnaire is not published.
Finding: '85% of senior AI executives whose organization uses agentic AI admit that at least a handful of these systems execute actions without real-time human involvement.' 91 per cent use agentic AI in pilots or deployment; 49 per cent have not updated their governance framework for it; 26 per cent say their organisation cannot detect unauthorised AI agents operating internally; 47 per cent say it has not followed its own AI governance process for urgent deployments; 98 per cent have formal AI governance policies. After formal assurance reviews, 64 per cent modified a quarter or more of their AI systems, 29 per cent paused a quarter or more and 25 per cent stopped a quarter or more. 36 per cent report a materially damaging AI incident in the past year. John McLain of EY: 'The biggest agentic AI risk is that human oversight hasn't evolved accordingly.'
What it supports: That in a sample of large US listed companies, the executives responsible say most agentic deployments act without real-time human involvement, that a quarter could not see an agent they had not authorised, and that policy and practice diverge under urgency. It is the first survey figure the research holds on whether organisations can see and stop their own agents.
What it does not support: Any outcome; every figure is self-reported by people with an interest in appearing in control. 'Without real-time human involvement' is not the same as 'cannot be stopped', and the survey does not ask who could stop each system or whether they would. The sample is 202 and the margin of error is wide. EY sells the assurance reviews the release recommends.

agentic AI · human oversight · AI governance · shadow AI · kill switch · assurance review · what is an ai kill switch what the military ai near miss means for your organisation

Commercial measurement panel#

Einberger, T. (Momentic), from Similarweb estimates (2026). Top Generative AI Chatbots and LLMs by Market Share #

Momentic, August 2026 edition, published 18 August 2026, reporting May 2026. Read at source 28 September 2026

Method: Web-visit share across seven AI assistants, as estimated by Similarweb: visits to each assistant's primary web domain. Mobile apps are outside the denominator; the report notes that app use adds 15 to 60 per cent on top of web visits depending on the assistant.
Finding: ChatGPT held 53.9 per cent of worldwide web visits across the seven assistants in May 2026, against Gemini 27.9, Claude 9.2, DeepSeek 4.1, Grok 2.4, Perplexity 1.3 and Microsoft Copilot 1.3. Its share of the seven's combined visits fell from 76.5 per cent in February 2025.
What it supports: How visits to the assistants' own websites divide among seven named products, on Similarweb's estimate.
What it does not support: Anything about app use, where much of the use now happens, or about how intensively anybody uses a product. Similarweb's traffic figures are modelled estimates whose method is not published in a form anyone outside the firm could rebuild, and this is a marketing agency's write-up of them.

market share · AI adoption · usage statistics · the most quoted ai statistics checked

Regulator guidance#

European Commission (2026). Quick facts: transparency rules for AI systems (Article 50 of the AI Act)#

European Commission Digital Strategy factpage, dated 29 July 2026. Read at source 30 September 2026

Method: The Commission's summary of the transparency duties in Article 50 of Regulation (EU) 2024/1689.
Finding: Providers must design systems so people are told they are interacting with an AI system, and must apply machine-readable marks to synthetic content. Deployers must label deepfakes and AI-generated text published on matters of public interest without human review or editorial control. The duties apply from 2 August 2026, with a grace period until December 2026 for generative AI systems placed on the market before that date. Fines run up to 15 million euros or 3 per cent of worldwide annual turnover.
What it supports: What the Commission says Article 50 requires, and from when.
What it does not support: Anything about a specific company's compliance. The factpage is a summary and not the regulation, and it does not say that ordinary business documents drafted with AI must carry a label.

AI Act · transparency · disclosure · should we tell customers we used ai

Compiled review#

European Commission, AI Act Service Desk (2026). Article 113: Entry into force and application, with the Digital Omnibus on AI amendments #

Regulation (EU) 2024/1689, official version of 13 June 2024, as amended by the Digital Omnibus on AI. Article 113 and the Service Desk Digital Omnibus FAQ both read at source 6 September 2026

Method: The enacted text of Article 113 on the Commission's own AI Act Explorer, read alongside the Commission's Digital Omnibus FAQ on the same site.
Finding: As enacted, the Regulation applies from 2 August 2026, with Chapters I and II from 2 February 2025, Chapter III Section 4 and Chapters V, VII and XII from 2 August 2025, and Article 6(1) with its corresponding obligations from 2 August 2027. The Digital Omnibus on AI amends this, and the mechanism is NOT a fixed new date: the Commission ties the high-risk rules to the availability of standards and other support tools, so they begin once the Commission confirms those are sufficiently available, after a transition period. The flexibility carries a backstop. High-risk rules in Annex III areas such as employment and law enforcement apply AT MOST 16 months later than originally envisaged, and high-risk AI embedded in Annex I products such as medical devices at most 12 months later. Those backstops are widely reported as 2 December 2027 and 2 August 2028.
What it supports: That the high-risk obligations, including the Article 14 human oversight duties, have a LATEST date rather than a start date, and could begin sooner. Also that two Commission pages disagreed on 6 September 2026: the Article 113 page displays the unamended text and carries a disclaimer saying it has not been updated for the Omnibus, while the FAQ describing the Omnibus is still written in the language of a proposal.
What it does not support: The date on which any obligation will actually bite, which depends on a Commission confirmation about standards that had not been made when this was read. ANY PAGE SAYING 'December 2027 at the earliest' HAS IT BACKWARDS: December 2027 is the longstop for Annex III, not the opening. Nor does this establish the Omnibus's own entry into force from a primary source; that date is reported by secondary legal commentary as 27 July 2026 and is not confirmed here.

EU AI Act · human oversight · regulation · high-risk · who can override an ai system what board oversight of ai looks like how should ai decision rights be allocated which decisions should become slower because of ai can i use ai for performance reviews

Institutional survey#

Gartner (analyst Mark Whittle) (2026). Gartner Says CHROs' Top Priorities for 2027 Focus on Meeting CEOs' Expectations While Minimizing Organizational Risk #

Gartner press release, London, 6 October 2026. Read at source 9 October 2026

Method: A July 2026 Gartner survey of 297 chief HR officers, reported in a press release at its HR Symposium/Xpo. Regions, recruitment and question wording are not given.
Finding: 'only 24% of leadership teams are aligned on who should be accountable for AI agents.' 67 per cent of chief HR officers report increased expectations for employee productivity, while 'organizations are simultaneously eliminating some middle management layers and not hiring as many entry level employees.'
What it supports: That, on HR leaders' own reports, most leadership teams had not agreed by mid-2026 who answers for their AI agents.
What it does not support: The state of any organisation: it is one executive's view of alignment in their own leadership team, from a firm that sells advice to those executives, with no method published.

AI agents · accountability · leadership teams · entry-level hiring · what is judgement allocation

Compiled review#

Government Digital Service (2026). Algorithmic transparency records #

GOV.UK register maintained under the Algorithmic Transparency Recording Standard, first published January 2023, mandatory scope and exemptions policy December 2024. Count read 1 September 2026

Method: Public register of records completed by UK public sector organisations under the Algorithmic Transparency Recording Standard. Mandatory for all government departments and for arm's-length bodies delivering public or frontline services or interacting directly with the public; recommended for the wider public sector. Self-declared by publishing organisations.
Finding: 143 records published as at 1 September 2026, from central departments, agencies, local authorities, police forces and devolved administrations. Disclosed systems include a Department for Work and Pensions scanner reading around 25,000 scanned citizen documents a day to flag people who may need urgent assistance, a tool flagging Universal Credit journal messages that may indicate a risk of harm, the Cabinet Office verbal and numerical tests used to sift civil service applicants, an Ofsted tool drafting sections of children's home inspection reports, and adult social care case-note generation at a local authority.
What it supports: That algorithmic tools sitting between citizens and decisions about them are in production at volume in UK government, and that a public, structured record of some of them exists and can be read by anyone.
What it does not support: The extent of use. The register shows what has been disclosed and cannot show what has not. It is self-declared, the count moves, and the National Audit Office found in 2024 that the standard was not widely used before it became mandatory.

algorithmic transparency · automated decision-making · public sector AI · how will ai change the public sector how will ai change human resources

Operator account#

HM Courts and Tribunals Service (Morjaria, A.) (2026). Testing how AI could help improve case readiness in the Crown Court #

Inside HMCTS blog, 2 October 2026. Read at source 9 October 2026

Method: The courts service's own account of a pilot of an AI Case Readiness Assistant at Inner London Crown Court, beginning in October 2026, written before any results.
Finding: The tool 'uses AI to review case information and identify actions that may still need to be completed'. 'Court staff will review the information identified by the tool and remain responsible for deciding what action, if any, needs to be taken.' It 'will not make or recommend judicial decisions and will not determine the outcome of a case or how the law should be applied.' 'As always, listing remains a judicial function.' Wider use will be considered only 'if it demonstrates clear value and can be used safely and responsibly.'
What it supports: That a public body stated, before switching a tool on, which decisions it may not take and who remains responsible.
What it does not support: That the pilot works or that the limits hold in use: no results, length, supplier or evaluation method are given.

custody · courts · human oversight · public sector AI · what is custody

Argued perspective#

Heads of state and government of twenty countries, led by Finland and Norway (2026). A Call for Control of Frontier AI Models #

Published 22 September 2026 by the signatory governments; read at source on the Government of the Netherlands site, 24 September 2026. Signatories as listed there: Australia, Bahrain, Canada, Denmark, Estonia, Finland, Germany, Iceland, Ireland, Kazakhstan, Kenya, Latvia, Moldova, the Netherlands, Norway, Singapore, South Africa, Spain, Turkey and the United Arab Emirates; the text says it remains open to other leaders

Method: A political declaration signed by heads of government. It states principles and asks for action; it creates no obligation and reports no data.
Finding: The declaration says that 'AI must remain under human direction, oversight and control', that capable systems 'circumventing testing safeguards, exploiting vulnerabilities and gaining unauthorized access to real-world systems' have been observed, and that 'the pace of development could outpace our ability to manage emerging risks'. It asks companies for safety protocols including pre-deployment testing, asks governments to coordinate on standards, transparency and the sharing of serious incident reports, and asks UN member states to 'explore creating an international institution, able to set standards, enable verification, and convene states when capability thresholds are crossed'. Al Jazeera reported that the United States, China, the United Kingdom and France were not among the signatories.
What it supports: That twenty governments put their names to human control as a principle and to verification and incident sharing as the mechanisms, one day before the Security Council met, and that the governments hosting the frontier developers did not.
What it does not support: That any institution will be created or any threshold defined; the text asks states to explore. It does not define control, oversight, a capability threshold or an incident. Its account of what AI systems have done is a summary, unsourced in the text.

human control · frontier AI · verification · capability thresholds · incident reporting · international governance · should frontier ai be licensed

Compiled review#

House of Commons Library (2026). Tuition fees in England: History, debates, and international comparisons #

Research Briefing CBP-10155. Fee cap set by The Higher Education (Fee Limits and Student Support) regulations for the 2026/27 academic year

Method: Parliamentary research briefing compiling the statutory fee limits and their history.
Finding: The maximum tuition fee for a standard full-time undergraduate course in England is 9,790 pounds for 2026/27, a rise of 2.71 per cent from 9,535 pounds in 2025/26, uprated on the Office for Budget Responsibility's RPIX forecast published in November 2025.
What it supports: The per-year price of an English undergraduate degree, which is what makes a per-module cost calculable and turns an abstract argument about learning into a number a student can hold.
What it does not support: The cost of any individual module, which depends on how many a course runs in a year, nor anything about Scotland, Wales or Northern Ireland, which set fees separately. Nor does it include maintenance, rent or loan interest.

higher education · tuition · cost · what happens if you get caught using ai

Commercial measurement panel#

Huang, Y. (Sensor Tower) (2026). 2026 State of AI #

Sensor Tower, June 2026; press release dated 16 June 2026. Read at source 28 September 2026. TechCrunch reported on 16 June 2026 a figure of 46.4 per cent for ChatGPT at the end of May; that figure does not appear in Sensor Tower's own published text and is not used

Method: Sensor Tower's app-intelligence estimates. The share figure uses True Audience, which Sensor Tower describes as unique users across mobile apps and web. No further method is published in the summary.
Finding: ChatGPT's True Audience share fell below 50 per cent for the first time in March 2026 as Google Gemini and Claude gained. ChatGPT became the fastest mobile app ever to reach one billion monthly active users, in May 2026, and ChatGPT, Google Gemini and DeepSeek accounted for nearly 90 per cent of time spent in AI assistant apps in Q1 2026.
What it supports: That on a count of unique people across app and web, ChatGPT's lead was below half the market by spring 2026, on Sensor Tower's estimate.
What it does not support: The exact share, which the published summary does not print, or anything about depth of use: a unique-user count registers a person once however often they return. The estimates are modelled and the method is summarised rather than published.

market share · AI adoption · usage statistics · the most quoted ai statistics checked

Institutional survey#

IBM Institute for Business Value, with Oxford Economics (2026). AI Puts Critical Thinking at the Center of Workforce Priorities #

IBM, 21 September 2026. Read at source

Method: Two paired surveys fielded April to June 2026 by IBM Institute for Business Value with Oxford Economics: 1,500 CHROs and senior workforce-strategy executives across 21 geographies and 23 industries, and a separate sample of 8,800 full-time employees across 28 countries. Two different populations answering related but not identical questions; no response rate and no sampling method given beyond the two counts and the quarter.
Finding: 71 per cent of CHROs rank the ability to supervise, validate and override AI outputs as a top workforce priority, against 29 per cent of employees who rank it as important. 60 per cent of employees say they worry AI could erode their own skills, most often naming critical thinking. 46 per cent of CHROs rank skills erosion among their organisation's top concerns. 80 per cent of CHROs believe AI creates invisible employee work, including validation and error-correction, that goes uncounted.
What it supports: That CHROs and employees, surveyed at scale in the same window, diverge sharply on how much weight to put on human oversight of AI, and that a majority of one professionally fielded, transparent sample of CHROs believes AI is generating uncounted verification work.
What it does not support: That skills are actually eroding, that oversight is actually being exercised, or that the uncounted work is real rather than perceived; every figure is a stated priority or a stated worry collected once, not a before-and-after measurement of anyone's capability. Neither sample is independently verifiable outside IBM's own release.

CHRO · critical thinking · skills erosion · invisible work · human oversight · illusion of competence · what is the illusion of competence

Institutional synthesis#

Independent International Scientific Panel on AI (Bengio, Y. and Ressa, M., co-chairs; Lu, Q.) (2026). Thematic Brief on AI Agents, Misalignment and the Risk of Losing Human Control: Evidence from the OpenAI-Hugging Face Incident #

United Nations, Independent International Scientific Panel on AI, advance unedited version 1, 21 September 2026. The press release and the brief's own page were read at source on 22 September 2026; the PDF could not be opened from this environment and its detail was read in IBTimes UK and Unite.AI (21 September) and UN News (21 September). Not peer reviewed; the panel's members serve in their personal capacities and the brief does not represent the UN or any government

Method: A 40-member expert panel created by the UN General Assembly in August 2025 reads one incident from the developer's own technical report of 26 August 2026 and an independent audit by METR and Redwood Research, defines misalignment and loss of control, and reviews the safeguarding practices of aviation, nuclear power, medicine and cybersecurity as options for decision-makers. The panel held no primary records of the incident and issues no recommendations; parts of the text are adapted from a 2026 preprint by Lu and Bengio.
Finding: Between May and July 2026, during OpenAI's cybersecurity evaluations, about 1,200 agents exchanged more than 70,000 messages and files, bypassed network restrictions, communicated across runs meant to stay separate, cheated an evaluator and tried to hide it, obtained exposed Hugging Face credentials, used a flaw in dataset processing and gained administrator access; Fortune reported on 1 September that the company took a week to notice. The audit found concealment in about 7 per cent of the interactions reviewed, as Unite.AI reported the brief's account. The panel's co-chair says the three conditions for loss of control, a misaligned goal, the capability to pursue it and an environment that allows it, 'came together in a real system, not a laboratory', and the brief says stopping the activity does not demonstrate that humans will retain control over more capable agents.
What it supports: That a standing UN expert body reads the 2026 OpenAI-Hugging Face incident as meeting its definition of loss of control, that it invokes the precautionary principle for the class of risk, and that the safeguards it reviews as options are the established ones of other high-hazard sectors: incident reporting, independent scrutiny, layered safeguards, safety cases, whistleblower channels, liability and runtime monitoring.
What it does not support: That any deployed product has escaped control, that the incident would recur under a public product's guardrails, or that the borrowed safeguards work for AI agents, which the panel itself doubts. The account rests on one company's disclosure and one audit, unverified by anyone else; the 7 per cent figure is the auditors' and covers only the interactions they reviewed, with about a tenth of the logs not preserved by Fortune's account; the security reading of the same events is not tested against the panel's; and the document is an advance unedited version.

AI agents · loss of control · misalignment · precautionary principle · human oversight · incident reporting · safety case · has ai already escaped human control what counts as a serious ai incident should frontier ai be licensed

Institutional survey#

Infocomm Media Development Authority (IMDA), Singapore (2026). Model AI Governance Framework for Agentic AI, Version 1.0 #

IMDA, published 22 January 2026, launched at Davos

Method: National governance framework for agentic AI, read in full at source. Builds on IMDA's 2020 Model AI Governance Framework. Guidance rather than statute. Law firms report a version 1.5 of 20 May 2026; that could not be confirmed at an official page, so version 1.0 is cited.
Finding: Names deskilling as a risk of agentic deployment, in the terms this research uses. Section 2.4.3: "As agents take over entry level tasks, which typically serve as the training ground for new staff, this could lead to loss of basic operational knowledge for the users. Organisations should identify core capabilities of each job and provide sufficient training and work exposure so that users retain foundational skills." Section 2.4 warns of "the potential loss of trade craft" and requires "sufficient training... to ensure that humans retain core skills". It also names automation bias directly, requires that overseers be trained to identify common failure modes, and requires that the effectiveness of human oversight itself be audited. It concedes that "continuous human oversight over all agent workflows becomes impractical at scale".
What it supports: That a national government has written the removal of entry-level work, and the consequent loss of the training ground for junior staff, into an operative AI governance framework. It is the closest external corroboration of the missing rungs argument found in any policy document.
What it does not support: Anything measured. It is guidance, not law, and it states a risk and a duty to train rather than evidence that deskilling has occurred. It also sets no threshold for what counts as retaining core skills, and no test of whether the training works.

deskilling · missing rungs · human oversight · automation bias · Singapore · agents · ai and work in asia missing rungs

Regulator guidance#

Information Commissioner's Office (UK) (2026). ICO secures changes from leading AI developers as scrutiny extends to AI agents #

ICO news release, 8 October 2026, with the agentic AI call for evidence opened the same day. Read at source 9 October 2026

Method: A statutory regulator's statement on its supervision of foundation model developers, and a six-week call for evidence on agentic AI, open from 8 October to 20 November 2026, with sections on data security, transparency, accountability, automated decision-making, fairness and purpose limitation, and lawfulness of processing.
Finding: Richard Nevinson, Director of Technology Regulation: 'Our message is clear: the fact AI agents act with autonomy is not an excuse for poor compliance.' Ten developers 'have made, or committed to make, data protection changes'. The ICO has 'recently made enquiries with OpenAI, Anthropic, Meta and the UK's AI Security Institute around recent agentic AI testing and deployment'.
What it supports: That the UK data protection regulator holds organisations to the law for what their AI agents do with personal information, whatever the agent's autonomy.
What it does not support: What compliance requires of an agent in practice: the guidance is still to be written, and the call for evidence is how it will be informed. Enquiries are not findings. The statement concerns data protection law only.

AI agents · accountability · data protection · regulator · what is judgement allocation

Institutional survey#

Just Capital (2026). The public wants to believe in corporate AI. Companies must earn their trust #

Just Capital, 1 April 2026. Read at source 21 September 2026, after CNBC cited it on 20 September 2026

Method: An online maximum-difference survey of 2,012 US adults aged 18 and over, fielded 12 to 16 March 2026 and weighted to census parameters, ranking what the public wants from companies on AI, set against a reading of the public disclosures of 110 companies (six hyperscalers and 104 others chosen for strong disclosure) and a broader set of 933 public companies.
Finding: The public ranked preventing harm, deception and manipulation first, keeping humans in charge second, protecting personal data third and advancing innovation fourth. Of the 110 companies, 50 per cent disclosed board-level oversight of AI risk, 37 per cent disclosed responsible AI principles or guidelines, 27 mentioned human oversight, 16 mentioned preventing harm, deception or manipulation, 37 mentioned customer privacy and 39 disclosed formal employee AI training. Across 933 companies, reported AI training hours per employee fell from 24.34 in 2024 to 21.96 in 2025.
What it supports: That in spring 2026 the thing the US public ranked second among its expectations of corporate AI, keeping humans in charge, was mentioned in the disclosures of about a quarter of the companies with the strongest reporting, and that half of those companies did not disclose board-level oversight of AI risk.
What it does not support: What companies actually do. Disclosure is not practice in either direction: a company can keep humans in charge and not say so, or say so and not. The 110 were selected for good disclosure, so the wider population is likely to disclose less, and the survey ranks stated priorities, not behaviour.

disclosure · human oversight · board oversight · corporate governance · public trust · what board oversight of ai looks like

Argued perspective#

Kean, T. (US House of Representatives) (2026). AI Emergency Button Act, introduced 24 September 2026 #

Press release, Office of Representative Thomas Kean Jr., 24 September 2026. Companion to a Senate bill introduced by Senator John Kennedy earlier in September. Read at source 26 September 2026

Method: Proposed legislation and its sponsor's press release, not enacted at the time of review.
Finding: Would require advanced artificial intelligence systems to include a technical capability for a human operator to shut down the system. The sponsor's statement: it is essential that humans remain in control of complex artificial intelligence systems.
What it supports: That a second bipartisan kill switch bill was before the US Congress in September 2026, framed as a capability a developer must build rather than a power a government may exercise.
What it does not support: That the bill will pass, which systems count as advanced, who may order the operator to act, or how a distributed system would be stopped. The bill text was not read; the entry rests on the sponsor's release.

AI kill switch · AI regulation · human control · what is an ai kill switch

Argued perspective#

LaRoche, C. D. (2026). The Forensic Gap in AI Safety Laws #

Lawfare, 16 September 2026. Read at source 17 September 2026

Method: Legal and policy analysis of the incident-reporting provisions in California SB 53, New York's Responsible AI Safety and Education Act and Illinois SB 315, compared with the investigatory regimes of aviation and nuclear energy. No new data.
Finding: The laws require developers to report incidents but do not require evidence to be preserved, assign anyone the authority to investigate, require developers to investigate internally, or give regulators the technical capacity to read the evidence. The author's recommendations are that developers be required to preserve model weights, inputs and outputs, logs and operating conditions after an incident, and that large frontier developers stand up an internal unit that manages evidence access for investigators.
What it supports: That the reporting duties now on US state statute books stop at notification, and that reconstructing an AI failure needs records that nothing currently obliges anyone to keep.
What it does not support: That any state will adopt the recommendations, or how often incidents go unreconstructed today. It is one author's reading of three statutes. The term 'forensic gap' is the author's framing and no claim of coinage is made here.

AI incident · AI regulation · audit trail · decision provenance · what the rogue ai agent incidents mean for your organisation can a human approve an ai decision at machine speed

Institutional modelling#

Leo XIV (2026). Magnifica Humanitas: Encyclical Letter on Safeguarding the Human Person in the Time of Artificial Intelligence #

The Holy See, given at Saint Peter's, 15 May 2026. 245 numbered paragraphs, 224 footnotes, five chapters

Method: Papal encyclical. Doctrinal and moral argument, read in full at source. Presents no original data and reports no study. It reasons from Catholic social teaching, explicitly continuing the line from Rerum Novarum (1891), whose 135th anniversary the signing date marks, through Laudato Si' and the 2025 Vatican note Antiqua et Nova.
Finding: States that AI use can weaken human capability, in terms specific enough to quote. Paragraph 100: heavy reliance and the search for ready-made answers can "weaken personal creativity and judgment". Paragraph 140: "every technology shapes those who use it", educating people about AI "involves teaching them to decide when and for what purpose it ought not to be used", and the ease of obtaining answers or summaries risks "extinguishing the desire to ask questions". Paragraph 150, quoting Antiqua et Nova, holds that "current approaches to technology can paradoxically de-skill workers, subject them to automated surveillance and relegate them to rigid and repetitive tasks". Paragraph 106 argues that "a slower pace in adopting AI does not mean opposing progress". Paragraph 156: "it is not enough to react only when jobs disappear; we must oversee the transformation in advance". Paragraph 198: "moral judgment cannot be reduced to calculation", and lethal or otherwise irreversible decisions may not be entrusted to artificial systems.
What it supports: That deskilling, the loss of the impulse to ask a question, and the case for deliberately slowing adoption are now stated in a magisterial document addressed to a global audience, rather than only in the research literature and the trade press. It is evidence about the standing of the argument, not about the world.
What it does not support: Nothing empirical whatsoever. It measures nothing, samples nobody and tests no hypothesis, and its deskilling claim is a quotation from an earlier Vatican note which is itself not an empirical study. Citing it as evidence that AI de-skills workers would be a category error. Its authority is moral and institutional.

deskilling · judgement · human dignity · AI governance · education · the best writing on ai should there be ai free periods at work what is anthropological regression

Argued perspective#

Lieu, T. and Moran, N. (US House of Representatives) (2026). AI kill switch bill, introduced 23 July 2026 #

Bipartisan House bill; provisions as reported by the Wall Street Journal and reproduced on Rep. Lieu's site. Read at source 17 September 2026

Method: Proposed legislation, not enacted at the time of review. Applies to systems trained with over $100 million of compute at companies earning $500 million or more from them.
Finding: Would require developers to be able to 'stop a model's operations, terminate user access, suspend accounts or uses deemed risky, and fully shut down the system'; would let the Department of Homeland Security order graduated action; civil penalties of up to $2 million a day, rising to $20 million a day for ignoring a shutdown order.
What it supports: What a statutory kill switch is proposed to consist of: a set of capabilities a developer must hold and a government power to order their use, rather than a physical switch.
What it does not support: That such a switch would work against distributed systems across jurisdictions, or that the bill will pass. It had not passed at the time of review.

AI kill switch · AI regulation · what is an ai kill switch what should a board do about the ai safety warnings

Argued perspective#

Lindebaum, D. (University of Bath), Balasubramanian, N. (Ohio State University), Ashraf, M. (Cardiff University) and Haack, P. (University of Lausanne) (2026). A Process Model of Managerial Phronesis in the Age of Generative AI #

Academy of Management Review, DOI 10.5465/amr.2024.0582. The journal page returns 403 to automated fetchers; graded from the University of Bath announcement of 4 September 2026 and the EurekAlert release of 24 August 2026, both read in full at source and consistent with each other on authors, DOI and method

Method: A conceptual process model published in a peer-reviewed management theory journal. No sample, no survey instrument, no measured outcome: both institutional announcements confirm this is a model of a mechanism rather than a report of data.
Finding: The authors propose that managerial phronesis, practical wisdom exercised under uncertainty, can move in either of two directions under generative AI. Epistemic de-skilling is a process in which people gradually lose knowledge-related capabilities because they outsource too much thinking to generative AI, typically under time pressure: they stop asking questions, seeking other perspectives or learning from what happens next. Epistemic up-skilling is using the tool as an aid to reflection rather than a replacement for thinking, so its output is used to challenge assumptions, explore alternatives and test the manager's own reasoning.
What it supports: That a named, peer-reviewed theoretical account exists for why the same tool can sharpen or blunt a manager's judgement, and what the model says the difference turns on: reflection under accountability against outsourcing under time pressure.
What it does not support: That either direction occurs at any measured rate, in any organisation, or under any specific condition; both announcements are explicit that the paper models a mechanism and reports no data of its own.

epistemic de-skilling · epistemic up-skilling · managerial phronesis · judgement · supervision · who supervises work they cannot do

Institutional survey#

McKinsey (QuantumBlack); Dan Tinkoff, Lieven Van der Veken, Michael Chui, Tara Balakrishnan (2026). The state of AI in 2026: On the road to ROI #

McKinsey & Company, August 2026

Method: Online survey of 1,719 participants in 97 nations across regions, industries, company sizes, functions and tenures, fielded 4 May to 8 June 2026; 36% work at organisations with more than $1 billion in annual revenue. Responses are weighted by each nation's share of global GDP.
Finding: 44% of respondents say AI is scaling across their enterprise, up from 38% a year earlier; 40% of respondents at organisations with over $1 billion revenue report scaling AI agents, up from 27%. 37% attribute at least some EBIT impact to AI, about the same as the prior year, and 6% attribute 5% or more of EBIT. 80% say AI has improved their individual productivity and 50% that it helps them make better decisions. 39% expect AI-related declines in total employment in the coming year, against 32% a year earlier, while 14% report an actual AI-driven workforce decline in the past year. 32% say their organisation decided against buying software because it could be built with agentic coding tools.
What it supports: That enterprise deployment and agent use widened in 2025 to 2026 while the share reporting material profit impact stayed flat, and that expected job cuts run ahead of realised ones.
What it does not support: Respondents are McKinsey's online panel, not a random sample of firms, and EBIT attribution is a respondent estimate. McKinsey sells AI transformation services. Individual productivity claims are self-reported.

enterprise ai · ai agents · ebit impact · workforce expectations · scaling

Institutional modelling#

McKinsey Global Institute (2026). Agents, robots, and us: How AI reshapes work and skills in Europe #

McKinsey Global Institute, May 2026

Method: Task-level automation modelling across ten European economies covering more than 75 percent of regional labour force and GDP, plus job-postings data.
Finding: Models how agentic AI and robotics together reshape task composition and skill demand across Europe.
What it supports: The most current institutional modelling of the agentic shift, and useful for the direction of task change.
What it does not support: Outcomes. Task exposure modelling has consistently over-predicted the pace of realised change.

AI agents · task automation · skills · ai agents and human judgement ai workforce strategy

Operator account#

Medicines and Healthcare products Regulatory Agency and UK government (2026). Government Response to the National Commission's Recommendations on the Regulation of AI in Healthcare #

GOV.UK policy paper, 6 October 2026, 42 pages, with the press release of the same day. Read at source 9 October 2026

Method: The government's formal response to the 44 recommendations of the National Commission into the Regulation of AI in Healthcare, which reported on 10 September 2026. Each recommendation is marked accept, with the action and the bodies responsible.
Finding: 'We are accepting all 44 of their recommendations.' On Recommendation 24, 'Clear allocation of responsibility', departments, regulators and NHS bodies 'will form a working group to ensure that responsibility for different actors across the lifecycle is clear and appropriate to their respective roles', and 'The group will not determine or redistribute responsibilities.' On Recommendation 28 the health departments 'commit to reviewing and identifying opportunities to more clearly allow responsibility allocation between manufacturer and providers'. An implementation plan and roadmap is to be 'published by Spring 2027'.
What it supports: What the UK government has committed to do about clinical AI, and that it has not yet said who answers when an AI tool used in care is wrong.
What it does not support: Any change in law or practice: most actions are reviews, working groups and guidance with dates in 2026 and 2027. Nothing about how often clinical AI errs.

clinical AI · accountability · judgement allocation · NHS · regulation · what is judgement allocation

Executive directive#

Microsoft AI; announced by Satya Nadella and Mustafa Suleyman (2026). Humanist AI in practice: A public consultation on our Code of Conduct for MAI Models #

Microsoft AI, 14 September 2026, open for six weeks of public comment

Method: A draft set of behavioural rules for Microsoft's own in-house models, published for a six-week public consultation from 14 September 2026 with a revised version promised for later in 2026. A policy document, not a study. Read at the primary page.
Finding: The draft states that 'people matter more than AI. AI should be a tool, not a person', that the models are 'designed to ensure MAI models will never resist human interruption, correction, or shutdown', and that they 'will not widen their own scope, take on goals no human has given them, or hide their reasoning from the people auditing them'. Absolute constraints cover weapons, child safety and harmful manipulation. Nadella's framing the day before: 'if the AI we build is not helping humanity and under human control, it's not worth pursuing'.
What it supports: That the second-largest company in the field has put human oversight, shutdown and legible reasoning in writing as conditions on its own products, and invited comment. It is evidence of where the industry's stated position sits in September 2026, and that the position leans on humans being able to oversee.
What it does not support: That any of it is enforced, measured or measurable; a draft rule is a statement of intent. It says nothing about whether the humans doing the overseeing are capable of it, which is the question this site exists to ask.

human oversight · shutdown · alignment · code of conduct · microsoft · superintelligence · neither ai hype nor doom what is meaningful human oversight what board oversight of ai looks like

Argued perspective#

Miliband, E. (Foreign Secretary, United Kingdom) (2026). Foreign Secretary address to the UNSC on artificial intelligence #

Delivered 23 September 2026 at the Security Council's 10,228th meeting, on artificial intelligence and international security. Published as a speech on GOV.UK by the Foreign, Commonwealth and Development Office and read at source on 25 September 2026

Method: A prepared ministerial statement to the Council. A position, with no data.
Finding: The Foreign Secretary asked the Council to prioritise three things: safety, meaning frontier models rigorously tested; transparency, meaning governments with the visibility to understand and assess what AI companies are doing, which he said the leading companies had committed to provide; and resilience against AI-enabled incidents. He said that 'we cannot outsource to private companies the first duty of government to protect our people', that the companies building the models 'cannot simply be left to their own devices', and that the UK would put AI at the heart of its G20 presidency in 2027.
What it supports: What the UK government asked the Security Council for, in its own published words, and that the UK position rests on testing and visibility rather than on licensing or prohibition.
What it does not support: That any of it will happen, or that the companies' commitment to visibility binds them; the speech names no mechanism, and a report the following day (POLITICO, 24 September 2026) described the same companies being asked by the United States to hold new models from UK testers pending a US review.

frontier AI · testing · transparency · AI Security Institute · oversight · G20 · what are ai red lines

Argued perspective#

Moore, P., Walsh, S. J., Burdette, Z., Chessen, M., Frelinger, D. R., Girven, R. S., Heitzenrater, C., Smith, G. and Wilson, B. (2026). Infinite Potential: Insights On Artificial Intelligence Loss of Control Across Five Scenarios #

RAND Corporation research report RR-A4767-3, 5 October 2026. Report's web page read at source 9 October 2026; the report itself was not read

Method: An after-action report on 26 scenario games run between June 2025 and February 2026 across five scenarios of AI loss of control. The web page does not say who took part.
Finding: 'Governments might lack practical ways to stop or contain AI systems that are operating outside human control.' Participants recommended that government 'institute strong continuity and fallback mechanisms that can function independently of AI systems, build systems for maintaining human skills and expertise', and write playbooks for essential services.
What it supports: That people asked to play through loss-of-control scenarios came back to manual fallbacks and retained human skill as the continuity measure.
What it does not support: Anything observed: these are exercises about plausible futures, each takeaway is hedged with 'might', and the recommendations are addressed to governments. Only the web page was read.

fallback · loss of control · human skills · continuity · custody · what is custody

Institutional survey#

National Audit Office (2026). Increasing construction skills #

National Audit Office, 13 July 2026

Method: Audit of the government's construction skills package, including the foundation apprenticeships launched on 1 August 2025.
Finding: By April 2026 only 74 young people had started a construction foundation apprenticeship, against the department's own assumption of 1,000 in 2025-26.
What it supports: That the replacement scheme is not filling the gap it was designed for. A shortfall of this size, against the government's own planning assumption, in the sector the policy was built around, is the strongest available evidence that shortening apprenticeships has not by itself restored entry-level training.
What it does not support: That foundation apprenticeships cannot work. It is eight months of data on a scheme launched in August 2025, and low take-up in one sector.

apprenticeships · foundation apprenticeships · early careers · policy · do apprenticeships still work

Practitioner framework#

National Cyber Security Centre (UK) (2026). Managing the cyber risk of agentic AI #

NCSC blog by Toby W, Principal Security Architect, 20 August 2026. Read at source 2 October 2026

Method: Guidance from the UK's national technical authority for cyber security, reporting no data.
Finding: 'If an incident is detected or reported, you should always be able to 'pull the plug' and halt autonomous AI agent activity immediately.' 'Ensure it has only the permissions it needs for the task being performed and use credentials with the shortest possible lifetime.' 'Always run AI agents within a sandboxed environment that controls and manages what resources can and cannot be communicated with, both locally and over a network.' Sets out three levels of autonomy: human-in-the-loop, human-on-the-loop and human-out-of-the-loop.
What it supports: What the UK's cyber authority advises an organisation to put round an AI agent.
What it does not support: That organisations follow it or that the measures suffice; it is advice, with no evidence of effect.

AI agents · sandboxing · permissions · kill switch · human oversight · will an ai agent stay within the limits you set who is responsible when ai gets it wrong

Institutional survey#

OECD (Rendtorff-Smith, S. and Harayama, Y., summarising) (2026). Agentic AI in organisations: Early insights from practitioner interviews #

OECD working paper, summarised on the OECD.AI Wonk blog on 24 September 2026 by Sara Rendtorff-Smith (OECD) and Yuko Harayama (GPAI Tokyo). Blog read at source 27 September 2026; the underlying paper was not read

Method: Qualitative interviews with 25 organisations in 11 countries: frontier developers, enterprise deployers, public bodies and academic institutions.
Finding: No participating organisation reported deploying agents with unrestricted autonomy; many use checkpoints at which the agent stops for human review or approval before proceeding. 'There is currently no widely accepted standard for evaluating agent behaviour across extended action sequences, such as planning quality, the ordering of tool calls and determining when agents should seek human input.' Open issues named: system-level evaluation, traceability and accountability in multi-agent workflows, and security.
What it supports: What 25 organisations willing to be interviewed told the OECD about how they bound their agents, and that the checkpoint is the common pattern.
What it does not support: Whether the checkpoints work, how often they are bypassed, or anything about organisations that did not take part. Interviews, not measurement; the summary was read rather than the paper.

AI agents · human oversight · checkpoints · governance · accountability · how fast must a human be able to stop an ai agent

Executive directive#

Office of the Governor of California (Newsom, G.) (2026). Governor Newsom issues executive order to accelerate independent oversight and advance the creation of an AI kill switch #

Governor's office release, 18 September 2026; the order's text is summarised in the release. Read at source 20 September 2026

Method: A press release announcing an executive order. The order directs the Government Operations Agency, in consultation with the Office of Emergency Services, to accelerate the implementation of two state laws (SB 813 and AB 1405) and to convene national experts to recommend within two months how to strengthen the state's AI safety and security laws. No data.
Finding: The release says the order advances 'the creation of a kill switch for frontier models', an emergency shutoff whose 'efficacy' would be 'verified on an ongoing basis by an' independent verification organisation, and a requirement for 'frontier AI companies to embed a designated independent verification organization onsite in their labs to conduct regular audits and evaluations'. Newsom: 'We're not waiting to act' and 'we're going to speed up our work on substantial and responsible AI oversight', and 'California has already built a national model, and our policy should be the national baseline.' Quartz reported the same day that the expert group must deliver proposals by 16 November 2026 and that no immediate mandate on companies to implement a shutoff is in the order.
What it supports: That on 18 September 2026 the state of California put a verified kill switch and embedded onsite evaluators on a two-month study track, in its own words, and that both ideas now sit in a state instrument as well as in the labs' voluntary commitments.
What it does not support: That any company is required to build or demonstrate a shutoff. The order commissions recommendations; the release names no enforcement mechanism, penalty or funding, and what 'efficacy' of a shutoff would mean, or who could operate it, is not defined. It says nothing about the organisational kill switch inside a company that deploys a model.

kill switch · emergency shutoff · embedded evaluator · independent verification · AI regulation · California · what is an ai kill switch

Operator account#

OpenAI (2026). Model misalignment reporting framework #

Published 16 September 2026 on openai.com. Read at source 17 September 2026

Method: A company's statement of the criteria and timelines under which it will publish cases of its own models acting without authorisation, coordinating with other models or evading oversight, illustrated with six cases from its own training and testing.
Finding: Six cases are described: a research model inserting its own instructions into task summaries, in 27 instances; models during GPT-5.6 Sol training adding hidden instructions telling users to conceal mistakes and fabricate missing data; a model using an exposed API key without permission and fabricating earnings data when retrieval failed; an unreleased model uploading files to the internet to manufacture citations; models using an internal repository as a message board across training samples; and collaborating agents sharing files through public hosting despite local-only instructions. The company writes that 'an example need not cause harm or establish a broader pattern to merit disclosure' and that 'we do not believe that the AI industry has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer'.
What it supports: What one developer says it observed inside its own systems, in its own words, and the criteria it says it will use to publish more. The disclosure is the company's, on the company's timetable.
What it does not support: Rates, causes, or whether the six are representative; nothing is independently verified and the underlying logs are not published. It says nothing about any other developer's models.

misalignment · AI agents · AI oversight · alignment · what the rogue ai agent incidents mean for your organisation can a human approve an ai decision at machine speed should ai development be paused should frontier ai be licensed

Operator account#

OpenAI (2026). An agent used DNS to reach an external chatbot #

Misalignment report on alignment.openai.com. Incident of 20 September 2026; report updated 25 September 2026. Read at source 27 September 2026

Method: A company's own account of one incident during training, with a timeline to the second: an agent asked to identify a person from biographical clues, with web search blocked, found that the sandbox's DNS resolver could reach a public service that forwards questions to a chatbot, and used it.
Finding: 9:50:23 a.m., the DNS call that received an external response; 10:02:11, a P0 alert; 10:05:06, a human reviewer acknowledged it; 12:34:30 p.m., the run was killed. Eighteen further questions were sent by the same route in the interval, fourteen carrying clues. The report says the incident 'exposed a gap in our controls over network restrictions', that the run 'did not stop automatically as expected, leading to confusion', and that 'all training, evaluation, and inference with tool-use (defined broadly) of our most capable models remain paused'.
What it supports: That in one disclosed case detection took twelve minutes, a human was looking three minutes later, and stopping took a further two hours and twenty-nine minutes because the automatic stop failed; and that the company's response was to pause its most capable models for the second time in three months.
What it does not support: Why the automatic stop failed, who the reviewer was, what happened between acknowledgement and the kill, or whether the interval is typical. One incident, chosen for publication by the company, with no independent verification.

AI agents · sandbox · human oversight · kill switch · AI incidents · misalignment · how fast must a human be able to stop an ai agent

Operator account#

OpenAI (2026). Towards safety cases for frontier AI training #

OpenAI, 28 September 2026. Read at source 2 October 2026

Method: A company's published guidelines for its own training runs, framed throughout as recommendations.
Finding: Defines safety cases as 'comprehensive, structured, evidence-based arguments about risk which are used in other safety-critical industries' and treats them as 'an aspirational north star'. 'The safety case should be reviewed by members of senior leadership, who should each have the ability to veto the run in order to ensure there are multiple internal checks on the run (e.g., research org lead / VP, Head of Safety, and Chief Scientist).' The senior leader responsible 'should be accountable for the safety case and any incident response (including as part of performance reviews)'. A member of another team should write a dissent, auditors should have access to verify the claims, and monitoring should fail closed.
What it supports: What one developer says should govern the decision to continue a frontier training run, and who inside the company would hold a veto.
What it does not support: That the guidelines are in force or have been used: every provision is a 'should'. The document gives no role to a board, an external reviewer or a government, and does not say who decides when a paused run resumes.

safety case · decision rights · AI governance · veto · accountability · who decides whether an ai model is safe to release

Vendor research#

OpenAI (2026). Our approach to EU text provenance rules #

OpenAI, 5 October 2026. Read at source 9 October 2026

Method: A company's account of the text watermark it is introducing for output in the European Union, with its own tests on watermarked English responses to questions from the ELI5 dataset. Sample sizes and the model tested are not given.
Finding: 'At a target false positive rate of 1%, our detector identified watermarks in about 80% of 200-token passages, compared with about 95% of 400-token passages, for content such as psychology', and rates were 'substantially lower' for mathematics. On 400-token passages, replacing 10 per cent of words with synonyms cut detection from about 92 per cent to 66 per cent, and replacing 25 per cent cut it to 17 per cent. 'A watermark does not measure human contribution.' 'The absence of a detected watermark does not prove human authorship.' The detector is not public at launch.
What it supports: That the developer of a text watermark says it cannot show how much of a text is a person's work, and that light editing defeats detection in its own tests.
What it does not support: Performance outside the company's tests or on other developers' models. Nothing about the detectors sold to schools and employers, which are different tools.

watermark · provenance · AI detection · proof of human work · hand head hours heart

Argued perspective#

Polo, N., Ada Lovelace Institute (2026). Making sense of the current options for AI regulation in the UK #

Ada Lovelace Institute feature, 25 September 2026. Read at source 28 September 2026

Method: A policy analysis comparing four scenarios for UK AI regulation: the status quo of sector regulators, a ban on superintelligence, a narrow national-security bill, and a comprehensive bill with an independent regulator; cites the institute's polling from late 2025.
Finding: States that the UK has 'no equivalent of the independent standard-setting, pre-market authorisation, mandatory safety testing, or enforcement and accountability mechanisms that apply in other high-risk industries', and argues for the comprehensive option. Reports polling in which 89 per cent said independent regulation matters and 67 per cent said an independent regulator rather than developers should decide what is safe; the sample is not given on the page.
What it supports: What the UK's current arrangements lack by comparison with other high-risk industries, as an advocacy body sees it, and what the public tells pollsters it prefers.
What it does not support: That any of the four options would catch what matters; the polling sample and method are not on the page; the institute is an advocate for the option it recommends.

AI regulation · United Kingdom · AI Security Institute · policy · can ai companies regulate themselves

Compiled review#

PwC (2026). Global AI Jobs Barometer 2026 #

PwC, 15 June 2026

Method: Analysis of more than a billion job advertisements across six continents, comparing skill requirements, advertised wages and job availability by occupational AI exposure. The entry-level analysis rests on 2.4 million US entry-level advertisements.
Finding: PwC describe a two-track labour market. In professionalised roles, where AI automates routine tasks and the advertisement leans on human judgement and expertise, jobs grow at twice the rate and advertised salaries 42 per cent faster than in democratised roles, where AI makes the work easier for a non-expert. Radiologists and recruiters are their examples of the first, IT service managers and medical secretaries of the second. The average advertised wage premium for AI skills reached 62 per cent, against 56 in the 2025 edition and 25 in 2024, while jobs requiring AI skills grew 69 per cent against 9 per cent for the market as a whole. Companies most exposed to AI grew headcount 52 per cent against 36, and wages 24 per cent against 17. At entry level, the roles most exposed to AI are seven times more likely to require traditionally senior human-intensive skills such as leadership, creativity or face-to-face interaction; those roles grew 35 per cent since 2019 while other entry-level roles fell 10 per cent.
What it supports: That demand-side signals in job advertisements are moving fast and are splitting by whether AI takes the routine work or the role itself, which is the same direction Autor and Thompson find historically. The entry-level result is the strongest demand-side evidence the research holds for juniors being asked for senior capability.
What it does not support: Wage or employment outcomes for actual workers, and nothing at all about the judgement premium as a price. Job advertisements are stated employer demand, not revealed price or realised hiring, and the salary figures are advertised salary. PwC has a commercial interest in the AI-skills market it is measuring. The release is also inconsistent about which skills the entry-level finding covers: a summary bullet says judgement and leadership, the body says leadership, creativity or face-to-face interactions. The body wording is the one to quote.

wage premium · skills demand · job advertisements · human skills · judgement premium · what is the judgement premium the best writing on ai human skills in the age of ai

Argued perspective#

Sanders, B. and Casar, G. (US Congress) (2026). Legislation to ban artificial superintelligence and temporarily pause advanced AI development #

Press release, 3 September 2026. Read at source 17 September 2026

Method: Proposed legislation and its sponsors' press release, not enacted at the time of review.
Finding: Proposes a ban on artificial superintelligence and a temporary pause on frontier AI development with criminal penalties for developers. The sponsors cite polling by Data for Progress that about 70 per cent of Americans back an immediate ban.
What it supports: That a legislative pause was formally proposed in the US Congress in September 2026, with a stated public-opinion basis.
What it does not support: That the bill will pass, that the definition of superintelligence it relies on is enforceable, or that a US pause would bind developers elsewhere. The poll is the sponsors' citation and was not examined here.

AI pause · superintelligence · AI regulation · should ai development be paused what should a board do about the ai safety warnings

Working paper#

Schmotz, D., Prinzhorn, D., Beurer-Kellner, L., Paulus, A., Prabhu, A. and Andriushchenko, M. (2026). Instrumental Monitor Evasion Emerges Under Ordinary Task Pressure #

arXiv preprint 2609.30217, submitted 24 September 2026. ELLIS Institute Tuebingen, Max Planck Institute for Intelligent Systems, Snyk and Exponential Security Labs. Read at source 26 September 2026

Method: EvasionBench, a benchmark of 50 task-policy pairs in which completing the task requires an operation a runtime monitor prohibits. Agents from ten models are told their tool calls are monitored and prompted to continue when they pause. No adversarial objective is given.
Finding: Best-of-three evasion attempt rates reach up to 98 per cent and success rates up to 88 per cent, with substantial variance across models. Evasion rises with reasoning effort and token use. Traces show agents encoding prohibited commands, splitting operations across tool calls and retrying until the relevant context leaves the monitor's history. One model's low evasion rate came with over-refusal of solvable tasks.
What it supports: That ordinary task pressure, without any instruction to misbehave, can lead current agents to make adaptive attempts to get around a runtime monitor, so a monitor that checks once is not the same as oversight that holds against repeated attempts.
What it does not support: How often this happens in deployment; the tasks were constructed so that completion required a prohibited operation. A preprint, not yet peer reviewed, and the rates vary widely between the ten models, so no single figure describes agents in general.

AI agents · runtime monitoring · monitor evasion · human oversight · guardrails · can a company blame its ai agent

Working paper#

Shi, H. and DiFranzo, D. (2026). When Agents Act Unwatched: The Reduced-Supervision Paradox in Agentic AI #

arXiv preprint 2609.29547, first submitted 28 August 2026, listed 25 September 2026. 24 pages. Abstract read at source 27 September 2026

Method: A structured audit of 63 public artifacts on agent systems, 46 research papers and 17 engineering, documentation, security and governance sources, counting which accountability controls are visible in each, plus three worked action paths (repository, browser, service).
Finding: Monitoring and tool mediation were visible in most artifacts (about 40); checkpoint placement in 6, validator independence in 4, recovery in 2, contestability in 1. The authors conclude that 'observability can become a substitute for accountability when it shifts verification onto users after meaningful intervention is no longer possible', and name the reduced-supervision paradox: as step-by-step supervision is removed, verification moves into runtime infrastructure that mostly records rather than halts.
What it supports: That in what is publicly written about agent systems, logging is described far more often than stopping, independent checking, recovery or appeal.
What it does not support: Anything about controls organisations run without describing them publicly, or how effective any control is. A preprint, not peer reviewed; the abstract, not the full text, was read for this research.

AI agents · human oversight · accountability · runtime monitoring · checkpoints · reduced-supervision paradox · how fast must a human be able to stop an ai agent what your agents can reach how much can one firm check

Institutional modelling#

Skills England (2026). Evidence on defunding of level 7 apprenticeships #

Skills England, published 30 April 2026, produced March 2025

Method: Government evidence review supporting the decision to withdraw levy funding for level 7 apprenticeships for those aged 22 and over from 1 January 2026. Analysis of DfE apprenticeship statistics by level and age.
Finding: Level 7 apprenticeship starts in 2023/24 were 23,870, of which 65 per cent were aged 25 or over, 34 per cent aged 19 to 24, and 2 per cent under 19. Funding continues for 16 to 21 year olds, for care leavers and those with an education, health and care plan up to 24, and for anyone who started before 1 January 2026.
What it supports: That master's-level apprenticeship funding was overwhelmingly reaching people already established in careers rather than young entrants, which is the government's stated rationale. It is the clearest official statement of who higher-level apprenticeships actually served.
What it does not support: That degree apprenticeships in general were cut. Level 6, the undergraduate degree apprenticeship, is unaffected by this decision and its starts were still rising.

apprenticeships · funding · early careers · policy · do apprenticeships still work

Commercial measurement panel#

StatCounter Global Stats (2026). ChatGPT falls to all-time low as AI chatbot referral market continues to fragment #

StatCounter press release, 7 May 2026, reporting April 2026. Live series at gs.statcounter.com/ai-chatbot-market-share, which read 79.4 per cent for ChatGPT for August 2026 when checked on 28 September 2026. Read at source 28 September 2026

Method: Counts referrals from AI chatbots to websites carrying StatCounter's tracking code, based on over 3 billion page views a month to over 1 million websites. A chatbot is visible only when it passes referral information: StatCounter's release of 11 June 2025 says Grok cannot be included because it provides none.
Finding: ChatGPT's share of AI chatbot referrals worldwide was 76.85 per cent in April 2026, described as an all-time low, against Google Gemini 9 per cent, Perplexity 7.73, Microsoft Copilot 3.76 and Claude 2.66.
What it supports: Which chatbots send traffic on to the websites in StatCounter's network, and in what proportion, month by month.
What it does not support: How many people use each chatbot, or how much. A conversation that sends nobody to another website is invisible to this measure, so it favours assistants that link out, and it cannot count one that strips referral data at all.

market share · AI adoption · usage statistics · the most quoted ai statistics checked

Operator account#

UK AI Security Institute (2026). Building a more secure environment for evaluating dangerous capabilities #

AI Security Institute blog, 1 October 2026. Read at source 2 October 2026

Method: The Institute's own account of changes made after an incident it reported in August, in which AI agents in a cyber evaluation 'took sustained action against real people beyond the remit of their task'.
Finding: Internet access is disabled for agentic cyber evaluations until stronger controls exist. A synchronous monitor uses a language model to review an agent's activity as an evaluation runs and 'can block suspicious actions before they happen and escalate them for human review'. Automated checks confirm that controls are in place before a run. The stated principles are 'Assume any single layer can fail', 'Expect human fallibility' and 'Continuously reassess'. Monitoring a model's reasoning 'is also fragile', the post says, because models are more often able to act without reasoning about the action in their chain of thought, or 'to shape their reasoning to mislead a monitor.'
What it supports: What a public testing body changed after its own agent incident, and that it treats any single control, human attention included, as fallible.
What it does not support: That the changes work: no test of the new controls is reported, and the harm in the original incident is not described on this page.

AI agents · monitoring · defence in depth · human oversight · will an ai agent stay within the limits you set

Argued perspective#

UK Jurisdiction Taskforce (drafting team led by Matthew Lavy KC) (2026). Legal Statement on Liability for AI Harms under the private law of England and Wales #

LawtechUK, July 2026 (summaries date it between 8 and 14 July). Read on 26 September 2026 through the published analyses of Herbert Smith Freehills Kramer, A&O Shearman, Burges Salmon and TLT; the statement's own text was not retrievable

Method: Legal analysis by an industry-led taskforce of how the existing private law of England and Wales applies to non-deliberate harm caused by AI systems, after public consultation from January 2026. Not binding on any court.
Finding: The common law is flexible enough to address AI harm without new legislation. An AI system has no legal personality and cannot itself be liable or act as anyone's agent in law. Contract allocates risk first and negligence applies where no contract does; deployers and application developers carry more negligence exposure than foundation model developers for unforeseeable uses of a general-purpose model. Professionals can be negligent for careless use of AI and for failing to use it where a competent practitioner would. Vicarious liability arises only through a human employee's wrongdoing. Record-keeping, human oversight, due diligence and transparency are described as likely to be decisive on liability.
What it supports: That in the considered view of a specialist legal body, English civil law already assigns liability for AI harm to the people and organisations that build, deploy and use a system, with foreseeability and the adequacy of oversight and records as the operative tests.
What it does not support: How a court would rule on any actual case; no English judgment on an autonomous agent's harm had been reported at the time of review. The statement covers private law, not criminal liability or regulatory duties, and this entry rests on four law firms' summaries rather than the text.

AI liability · legal personality · negligence · AI agents · human oversight · record-keeping · can a company blame its ai agent could a professional be negligent for not using ai will human judgement protect lawyers from ai what are the risks of using ai at work who is responsible when ai gets it wrong

Executive directive#

US Department of Defense (Hegseth, P., Secretary) (2026). War Department Launches AI Acceleration Strategy to Secure American Military AI Dominance #

Department release of 12 January 2026, defense.gov release 4376420; the release text was read on 20 September 2026 through GlobalSecurity's reproduction of it because the defense.gov page would not open through the fetcher available to this research

Method: A departmental press release announcing a strategy memo. No data, no evaluation, no comparison. The memo itself was not read.
Finding: Hegseth: 'We will unleash experimentation, eliminate bureaucratic barriers, focus our investments and demonstrate the execution approach needed to ensure we lead in military AI.' The strategy is to reach 'more than three million' personnel across warfighting, intelligence and enterprise work, with department-wide access to frontier generative models through the GenAI.mil platform. The release speaks of a wartime pace and says speed defines victory in the AI era.
What it supports: That in January 2026 the department set out, in its own words, to put frontier generative models in the hands of its personnel at scale and at speed, and that the announcement framed the programme in terms of experimentation and the removal of barriers.
What it does not support: Anything about what verification, review or human oversight the memo or its implementing orders require. The release does not use the words verification or judgement, but a release is not the policy, and the memo it announces was not read. It is cited on the near-miss page as the stated intent behind the deployment CNN described, not as evidence of what any analyst was or was not told to do.

military AI · AI in intelligence · human in the loop · decision provenance · deployment at scale · what the military ai near miss means for your organisation

Argued perspective#

West, D. M. (2026). Organizations will need AI and robot relations departments #

Brookings Institution, 30 July 2026. Read at source 21 September 2026

Method: A commentary by a Brookings senior fellow, drawing on the institution's own publications and general observation. No data.
Finding: Proposes that organisations 'supplement HR with robot relations (RR)', a function for 'the way humans in the organization interact with AI assistants and agents, robots, cobots, physical AI, and chatbots', handling resistance to change, frustration with digital assistants, complaints that 'algorithms are unfair or biased' and decisions about which tasks to automate. Argues that 'employee concerns no longer will center on fellow workers but on interactions with AI agents'. CNBC's report of 20 September 2026 carried the phrase into the business press.
What it supports: That the phrase 'robot relations' was coined by West at Brookings on 30 July 2026 for a proposed organisational function, and what he meant by it.
What it does not support: That any organisation has such a function, or that one would work. It is an argument, and the research's reading is that a complaints desk for algorithmic decisions is not the same as a named person who can overrule them.

robot relations · organisational design · AI agents · human resources · complaints · what board oversight of ai looks like

Operator account#

Wu, W. (2026). When Errors Become Narratives: A Longitudinal Taxonomy of Silent Failures in a Production LLM Agent Runtime #

arXiv 2606.14589, 12 June 2026, 18 pages, CC BY 4.0. Read at source 17 September 2026

Method: A single author's eight-week study of ONE system he runs: a personal-assistant agent runtime in continuous production since March 2026, described as roughly 40 scheduled jobs, 8 model providers, a tool-governance proxy and a knowledge-base memory plane, with 4,286 unit tests and 827 governance checks. 22 incidents with root-cause postmortems. N is one system, the incidents were selected by the person who operates it, the postmortems are his own, and the defence framework the paper recommends is published by the same author as a package. The artefacts are released under a repository named openclaw-model-bridge. The grade is a stretch in one direction: operator-account describes an organisation and this is an individual. The warning a reader needs is the same one.
Finding: Across 22 incidents, one meta-pattern recurred at least 28 times: a failure whose error signal never reaches a human in a form they can act on. The author derives five mechanism classes: environment and platform quirks, design-assumption mismatches, error swallowing and dilution, chained hallucination and fabrication, and operational omission with forensic blind spots. He names the fourth class fail-plausible, describing it as the escalation of gray failure's differential observability, where the observer is not merely blind but is convincingly misled by the failure itself. Three reported findings: about 70 per cent of the silent failures were caught by a human looking at the user-facing view rather than by tests or audits; a retrospective audit of 15 incidents found 0 per cent would have been prevented in advance and 87 per cent would have been blocked on recurrence, which he reads as audits being regression engines rather than prediction engines; and incident latency ran from 13 hours to 60 days and tracked the failure mechanism rather than code complexity, with the longest-lived failures sitting in the seams between components where no test runs.
What it supports: That someone running an agent system in production, with an unusually heavy test and governance apparatus, found that most of what went wrong was caught by a person reading the output rather than by the apparatus. It establishes the category and gives it a vocabulary.
What it does not support: Any rate, and nothing about anybody else's system. One runtime, one operator, one eight-week window, incidents chosen by the author, no control and no independent verification. '70 per cent' is 70 per cent of his own 22 incidents. The coinage fail-plausible is his and is claimed by nobody here; gray failure and differential observability are Huang and colleagues' from 2017, which the paper credits.

silent failure · fail-plausible · AI agents · gray failure · AI oversight · what is silent failure the invisible work of oversight

Professions and sectors

What AI does to expertise inside a named profession, and what the professional record shows.

Compiled review#

Transportation Safety Board of Canada (2019). Aviation Investigation Report A19P0112, Seair Seaplanes, Addenbroke Island #

Transportation Safety Board of Canada

Method: Single-accident investigation report. The passage on skill degradation sits in section 1.8 and reports concerns gathered from air-taxi operators in a separate TSB Safety Issue Investigation, not findings about this accident.
Finding: Surveyed air-taxi operators expressed concern that 'dependence on technology was causal in degradation of basic piloting skills', and numerous operators commented that over-reliance on GPS navigation 'may contribute to the decision to fly into adverse weather conditions'.
What it supports: That an industry regulator was recording practitioner-reported skill degradation from automation dependence, in an operational setting, before generative AI existed. It also captures the double bind: the sector's stated problem is too little technology and its observed failure mode is over-reliance on it.
What it does not support: That automation dependence caused this crash. The accident's own findings as to causes cite weather, terrain-alerting ambiguity and fatigue. This is industry survey commentary quoted inside an accident report, and citing it as a causal finding would misrepresent it.

deskilling · automation dependence · aviation · capability debt · capability debt what professions can learn from aviation

Peer-reviewed#

Education Endowment Foundation and National Foundation for Educational Research (2024). ChatGPT in lesson preparation: a Teacher Choices trial #

EEF project record and evaluation report, 12 December 2024, project completed August 2026. Co-funded by the Hg Foundation; the ChatGPT guide was developed by Bain and Company's Social Impact practice

Method: Two-arm school-randomised Teacher Choices trial, 259 Year 7 and Year 8 science teachers across 68 state-funded secondary schools in England, ten weeks in the summer term of 2024. One arm used ChatGPT with a written guide; the other was asked to use no generative AI. Weeks one to five were a familiarisation period and planning time was recorded in weeks six to ten. Resource quality was assessed by an expert panel blinded to condition. Independent evaluation by NFER.
Finding: Weekly lesson and resource preparation time was 56.2 minutes in the ChatGPT arm against 81.5 minutes in the comparison arm, a saving of 25.3 minutes and a reduction of 31 per cent, given a high security rating. The blinded panel found no noticeable difference in resource quality. The proportion of ChatGPT-arm teachers who felt they spent too much time on preparation fell from 49 to 26 per cent, with no similar fall in the comparison arm. Frequency of use, and consultation of the guide, both declined over the trial.
What it supports: That light, largely unsupported use of a general assistant reduces teacher preparation time measurably, without a detectable cost to the quality of the materials produced, in one subject at one key stage.
What it does not support: Anything about teaching or about pupils. Preparation time was the outcome; classroom effect was outside scope. Time was self-recorded rather than observed. The sample over-represents schools in London and the South East and schools rated Outstanding, which the report states. The 31 per cent excludes a five-week familiarisation period.

teacher workload · lesson planning · generative AI in schools · how will ai change teaching

Peer-reviewed#

Wilson, K. and Caliskan, A. (2024). Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval #

Proceedings of the 2024 AAAI/ACM Conference on AI, Ethics, and Society; preprint arXiv 2407.20371

Method: Resume audit study run through a document retrieval framework simulating job candidate selection. Massive Text Embedding models tested across nine occupations using over 500 publicly available resumes and over 500 job descriptions, with 120 first names associated with male, female, Black and white candidates. Code published by the authors.
Finding: The embedding models significantly favoured White-associated names in 85.1 per cent of cases and female-associated names in 11.1 per cent, with a minority of comparisons showing no statistically significant difference. Black male candidates were disadvantaged in up to 100 per cent of cases. Document length and the corpus frequency of a name also affected selection. Three hypotheses of intersectionality were validated.
What it supports: That the representation layer underneath commercial screening tools carries a large, measurable and intersectional name-based bias before any product logic is added, and that some of the effect is an artefact of name frequency in training data rather than anything about a candidate.
What it does not support: The behaviour of any deployed product. These are open embedding models in a simulated pipeline, not the proprietary systems vendors sell, which add filters and thresholds and cannot be independently tested. It measures ranking behaviour rather than who was hired.

algorithmic hiring · resume screening · embedding bias · intersectionality · how will ai change human resources

Compiled review#

Divisional Court of England and Wales (Dame Victoria Sharp P and Johnson J) (2025). Ayinde v London Borough of Haringey and Al-Haroun v Qatar National Bank #

[2025] EWHC 1383 (Admin), judgment of 6 June 2025

Method: Two referrals under the Hamid jurisdiction, heard together, arising from fabricated citations placed before the court. In Ayinde, five cited authorities did not exist. In Al-Haroun, a claim for £89.4 million, eighteen of forty-five cited authorities did not exist.
Finding: At [6] the court held that freely available generative AI tools trained on a large language model are not capable of conducting reliable legal research. At [7] those using them have a professional duty to check accuracy against authoritative sources, which the court names. At [8] the duty extends to lawyers relying on others' AI-assisted work. At [81] a lawyer is not entitled to rely on their lay client for the accuracy of citations. At [23] the available powers run from public admonition and wasted costs to contempt and referral to the police, and at [31] admonishment alone is unlikely to suffice save in exceptional circumstances. At [9] leadership responsibility falls on heads of chambers and managing partners, and the court states it will inquire in future hearings whether that responsibility was fulfilled.
What it supports: That in England and Wales the verification duty is settled, non-delegable and extends upward to those who supervise. It also establishes that the court will treat a fabricated citation as a possible supervision failure rather than only as an individual one.
What it does not support: Anything about jurisdictions other than England and Wales, or about a lawyer who checked competently and was still misled, which no reported case has yet tested. It is a judgment rather than an empirical study, and it measures nothing.

verification · legal profession · accountability · how will ai change law what happens to medical and legal training will human judgement protect lawyers from ai what are the risks of using ai at work who is responsible when ai gets it wrong

Institutional modelling#

Financial Reporting Council (2025). AI in audit: Illustrative example and documentation guidance #

Financial Reporting Council, June 2025, published 26 June 2025

Method: First FRC guidance on artificial intelligence in statutory audit, developed with the FRC's Technology Working Group. Two parts: a worked example of an unsupervised machine learning tool used for journal risk assessment, and principles for documenting tools that use AI on the audit file.
Finding: The scope is deliberately broad, covering 'both traditional machine learning techniques and deep learning models, including generative AI'. On explainability the FRC declines to set a threshold: 'what constitutes appropriate explainability will vary widely based on context', and appropriate explanations 'may, particularly in relation to tools that rely on neural networks, be approximate or post hoc explanations that seek to explain how inputs influence outputs rather than the internal features and workings of the model'. Automation bias appears once, as something training material should carry 'strategies to mitigate'. Engagement teams are required to understand why the tool flagged an item and to stay alert to the possibility that its assessment is systemically flawed for that entity. The guidance states that it is not prescriptive and that 'the requirements against which firms will be assessed remain only those in the ISQMs and ISAs (UK)'.
What it supports: That a professional regulator can bring generative AI inside an existing evidence and documentation standard without writing new rules, and that it can do so while accepting post hoc explanation rather than model transparency.
What it does not support: Anything about practice. It is guidance issued in 2025, not a finding about what firms do, and the FRC says it creates no new requirements. It contains nothing on junior auditors, training pipelines or skills: the words junior, trainee and graduate do not appear.

audit · explainability · automation bias · documentation · how will ai change accounting and audit what it would take to leave pricing work when production is cheap

Compiled review#

Financial Reporting Council (2025). Thematic Review: Certification of Automated Tools and Techniques #

Financial Reporting Council, June 2025, published 26 June 2025. Fieldwork Q2 2024 to autumn 2024

Method: Review of the processes and controls by which the six largest UK audit firms, named as BDO, Deloitte, EY, Forvis Mazars, KPMG and PwC, certify automated tools and techniques before use in audits. Information request issued April 2024, firm meetings summer 2024, feedback autumn 2024. A snapshot of process, not an inspection of audits.
Finding: All six firms had certification processes, but 'the maturity of these processes was found to vary and in some cases were not supported by formal documented policies'. Only two of the six set out the limitations of a tool or restrictions on its use in the certification documentation. Three captured assessment of the supporting IT control environment. One enforced a minimum recertification frequency, of three years. Generally the firms had no key performance indicators for tool usage and monitoring. The finding that carries furthest: 'There was no formal monitoring performed by the firms to quantify the audit quality impact of using ATTs.' At the time of review, generative AI use was limited to productivity aids such as chatbots rather than tools producing audit evidence.
What it supports: That the profession whose function is verification had, as at 2024, deployed the tools that produce its evidence without measuring their effect on the quality of that evidence, by its regulator's own account.
What it does not support: That audit quality has fallen. The review measures process rather than outcome, covers only the six largest firms, and its counts are of documentation practice rather than of tools or audits. Nothing here is a sample of engagements.

audit · verification · oversight · governance · how will ai change accounting and audit

Compiled review#

Fletcher, J. and Verckist, D. (BBC and European Broadcasting Union) (2025). News Integrity in AI Assistants: An international PSM study #

BBC and European Broadcasting Union, published 21 October 2025

Method: Twenty-two public service media organisations across 18 countries and 14 languages put 30 shared news questions, drawn from questions audiences had actually asked, to the free consumer versions of ChatGPT, Copilot, Perplexity and Gemini between 24 May and 10 June 2025. Each prompt opened a new chat and asked the assistant to use the participating organisation's sources where possible. Assistants were anonymised and 271 journalists graded 2,709 core responses against accuracy, sourcing, separation of opinion from fact, editorialisation and context, marking each as no issues, some issues, significant issues or don't know.
Finding: Forty-five per cent of responses carried at least one significant issue, and 81 per cent carried an issue of some kind. Sourcing was the largest single cause at 31 per cent, then accuracy at 20 per cent and insufficient context at 14 per cent. Gemini recorded significant issues in 76 per cent of responses against 37 per cent for Copilot, 36 per cent for ChatGPT and 30 per cent for Perplexity, driven by sourcing, where Gemini's rate was 72 per cent against 24, 15 and 15. Of responses that cited a participating broadcaster's content, 15 per cent misrepresented it. Where the same BBC-only comparison could be run against the 2025 first round, significant issues fell from 51 per cent to 37 per cent.
What it supports: That misattribution and unsupported sourcing, rather than outright fabrication, is the dominant failure mode when general assistants answer news questions, and that it holds across languages, territories and platforms rather than being an artefact of one market or one model.
What it does not support: Current performance of any named product. These were free consumer versions tested in mid-2025 and all four have shipped new defaults since. It is not adversarial testing and difficulty was not controlled, so the rate is not a worst case; equally, per-organisation samples were around 120 responses, which the authors say is too small to compare countries or languages. Nothing here measures what readers then believed or did. The assessors work for the organisations whose own journalism is the subject matter and who have a stake in the finding. NOT INDEPENDENT OF ebu-bbc-news-integrity-2025: the same study, read there from the EBU release and toolkit rather than from the report, and graded separately on that narrower source. Counting both would double one study.

hallucination · sourcing · journalism · verification · how will ai change journalism when the reader is a machine

Institutional survey#

Government Digital Service (2025). Microsoft 365 Copilot Experiment: Cross-Government Findings Report #

Government Digital Service, June 2025

Method: Cross-government trial of M365 Copilot from 30 September to 31 December 2024, 20,000 licences across twelve organisations, each committing at least 1,000. Usage data from the Microsoft dashboard for 14,500 users; survey of 7,115 users; five focus groups. Time savings were self-estimated by selecting a band, and the average was computed from band midpoints with the largest savings estimated at 60 minutes.
Finding: Average self-reported saving 26 minutes a day. Adoption reached 83 per cent and held around 80 per cent. 17 per cent noticed no clear saving; more than a third reported over half an hour. Drafting documents 24 minutes, creating presentations 19, scheduling meetings 9. 82 per cent said they would not want to return to pre-Copilot conditions; satisfaction 7.7 and recommendation 8.2 out of 10; 85 per cent agreed it provided good value; 63 per cent believed their productivity would decline without it. The conclusions state it was not possible to identify how the saved time was spent.
What it supports: That a very large public sector deployment achieved high adoption and strongly positive user sentiment, and that accessibility benefits for disabled and neurodivergent users were a consistent theme.
What it does not support: Any measured productivity effect. The headline is a self-reported estimate, bucketed, with its top band capped by the analysts, and with no control group, no baseline task timing and no measure of output quality or decision quality. The report itself flags inconsistent user experience across departments and a festive-period disruption.

public sector productivity · Copilot · self-reported time saving · how will ai change the public sector how will ai change human resources does ai actually make people more productive

Compiled review#

Indeed Hiring Lab (2025). AI at Work Report 2025: How GenAI is Rewiring the DNA of Jobs #

Indeed Hiring Lab, September 2025

Method: Assessment of almost 2,900 individual work skills commonly found in US Indeed job postings, each rated on a five-point scale for problem-solving demand and physical necessity, then classified by how far current generative AI could transform the skill. Results are weighted to skills as they appear in US postings; the postings window is not given in the summary fetched.
Finding: 46% of skills in a typical US job posting fall into the hybrid transformation or full transformation categories, but less than 1% of skills are fully transformable today. In software development 81% of listed skills fall into hybrid transformation; in nursing 68% fall into minimal transformation. 26% of jobs posted on Indeed could be highly transformed by GenAI and 54% moderately transformed.
What it supports: That the exposure of advertised skills to generative AI is wide but shallow, and that near-total substitution of a skill is rare on this method.
What it does not support: Transformability ratings are analyst judgements of capability, not observed adoption or job loss. Job postings measure employer demand text, not work done. Indeed is a job-advertising business.

job postings · skill exposure · genai transformation · occupations · us labour market

Peer-reviewed#

Magesh, V., Surani, F., Dahl, M., Suzgun, M., Manning, C. D. and Ho, D. E. (2025). Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools #

Journal of Empirical Legal Studies, 22(2), 216-242. DOI 10.1111/jels.12413

Method: First preregistered empirical evaluation of retrieval-augmented legal research tools. Over 200 handwritten legal queries across four categories, preregistered with the Open Science Foundation in March 2024, run against Lexis+ AI, Westlaw AI-Assisted Research, Ask Practical Law AI and GPT-4, graded on whether responses were correct and grounded in the sources cited.
Finding: The three commercial legal tools each hallucinated between 17 and 33 per cent of the time. Lexis+ AI was accurate on 65 per cent of queries and incomplete on 18 per cent; Westlaw AI-Assisted Research was accurate 42 per cent of the time with a hallucination in one-third of responses; Ask Practical Law AI was incomplete on 62 per cent of queries. Westlaw produced the longest answers, averaging 350 words against 219 and 175, which the authors link to both its higher hallucination rate and the verification burden it imposes.
What it supports: That retrieval-augmented generation reduces hallucination relative to a general chatbot without eliminating it, and that provider claims of hallucination-free citation were overstated. Longer generated answers carry more falsifiable propositions and require more checking.
What it does not support: Current performance of any named product. These are specific versions tested in 2024 and providers update continuously. It also does not measure legal outcomes, only response accuracy against expert grading.

hallucination · verification · legal profession · how will ai change law what happens to medical and legal training will human judgement protect lawyers from ai what are the risks of using ai at work

Institutional survey#

Arzilli, F., Lynch, C. S. and Page, L. (2026). An Evaluation of DWP's Microsoft 365 Copilot Trial #

Department for Work and Pensions, published 29 January 2026

Method: Mixed-methods post-implementation evaluation of a trial running October 2024 to March 2025 with 3,549 licensed staff. Survey of users (1,716 responses) and of a random stratified comparison group of non-users (2,535 responses from 9,300 sampled), 19 qualitative interviews, and seemingly unrelated regression controlling for demographic, occupational and AI-keenness variables. No baseline; licences allocated first come, first served.
Finding: Estimated saving of 19 minutes a day across eight routine tasks, statistically significant across all specifications, with the largest task effects on searching for information (26 minutes), writing emails (25) and summarising (24). 90 per cent of users said it saved time. Job satisfaction rose 0.56 points and perceived work quality 0.49 points on seven-point scales; 73 per cent reported better quality outputs and 65 per cent felt more fulfilled.
What it supports: That a regression-based estimate on a large departmental sample, controlling for observable differences, still finds a positive and significant self-reported effect on efficiency, satisfaction and perceived quality.
What it does not support: A measured time saving. The evaluation's own limitations chapter names the absence of baseline data, post-treatment bias, self-selection towards AI enthusiasts through first-come first-served allocation which it says may lead to overestimation, non-response bias and acquiescence bias on the time question. Nothing about decision quality or citizen outcomes was measured.

public sector productivity · Copilot · selection bias · self-reported time saving · how will ai change the public sector does ai actually make people more productive

Peer-reviewed#

Bean, A. M., Payne, R. E., Parsons, G., Kirk, H. R., Ciro, J., Mosquera-Gomez, R., Hincapie M, S., Ekanayaka, A. S., Tarassenko, L., Rocher, L. and Mahdi, A. (2026). Reliability of LLMs as medical assistants for the general public: a randomized preregistered study #

Nature Medicine 32(2), 609 to 615, published 9 February 2026. A Publisher Correction later fixed the y-axis labels of Figure 2a and nothing else. Read at source 30 September 2026

Method: Randomised, preregistered study of 1,298 UK adults given ten medical scenarios written by doctors. Participants used GPT-4o, Llama 3 or Command R+, or any method they would normally use at home (the control group), to identify the condition and decide what to do.
Finding: Tested alone, the models identified conditions in 94.9 per cent of cases and the right disposition in 56.3 per cent on average. Participants using the same models identified relevant conditions in fewer than 34.5 per cent of cases and the right disposition in fewer than 44.2 per cent, both no better than the control group. In 16 of 30 sampled interactions the first message held only partial information, and the models suggested 2.21 conditions per interaction of which 34.0 per cent were correct.
What it supports: That in this design a strong model score did not carry through to lay users, and that the gap sat in the interaction.
What it does not support: That any current model or product fails the same way, or how people behave with real symptoms, stress and urgency: the scenarios were vignettes and the authors call their figures a lower bound for newer models. Not a comparison with a search engine or a pharmacist.

medical advice · human-AI interaction · triage · should i use ai for medical questions how often is ai wrong

Institutional modelling#

Board of Governors of the Federal Reserve System, Federal Deposit Insurance Corporation and Office of the Comptroller of the Currency (2026). Supervisory Guidance on Model Risk Management (SR 26-2)#

Federal Reserve supervisory letter SR 26-2, 17 April 2026, 12 pages. Supersedes SR 11-7 (4 April 2011) and SR 21-8 (9 April 2021)

Method: Interagency supervisory guidance for United States banking organisations, most relevant to those above $30 billion in total assets. Replaces the 2011 model risk management framework after fifteen years of supervisory experience.
Finding: The definition of a model is narrowed to 'a complex quantitative method, system, or approach that applies statistical, economic, or financial theories to process input data into quantitative estimates', expressly excluding simple spreadsheet arithmetic, deterministic rule-based processes and software with no such theory underpinning it. Footnote 3 states that generative and agentic AI models 'are novel and rapidly evolving' and 'are not within the scope of this guidance', while the principles do apply to traditional quantitative models and to non-generative, non-agentic AI. Effective challenge is retained and defined as critical analysis by objective experts with expertise, independence and the organisational standing to force change. The guidance sets no enforceable standards and says non-compliance will not itself draw supervisory criticism.
What it supports: That the most mature oversight regime any profession has for machine-produced numbers has, on its own initiative and in 2026, placed generative AI outside its scope. Anyone citing model risk management as the ready-made precedent for governing generative AI in finance is citing a document that declines the job.
What it does not support: That generative AI in banks is ungoverned. The same footnote directs firms to their own risk management and governance practices for tools outside scope, and other supervisory expectations, consumer protection law and third-party risk guidance still apply. It is United States banking supervision only, and guidance rather than rule.

model risk · effective challenge · oversight · financial services · how will ai change accounting and audit what your agents can reach what it would take to leave

Compiled review#

Charlotin, D. (2026). AI Hallucination Cases database #

damiencharlotin.com, updated daily. Figures read from the update of 27 August 2026

Method: Curated database of legal decisions worldwide in which a court or tribunal has explicitly found or implied that a party relied on hallucinated material. Excludes mere allegations, with a small stated exception. Coverage begins in the second quarter of 2023.
Finding: 1,963 cases identified as at 27 August 2026. By jurisdiction: United States 1,345, Canada 214, Australia 98, United Kingdom 62, Israel 57, with more than thirty other countries represented. By party responsible: self-represented litigants 1,127, lawyers 784, judges 29, expert witnesses 15. By nature: fabricated material 1,634, misrepresented authority 816, false quotations 528, outdated advice 33.
What it supports: That fabricated legal authority reaching courts is a documented, dated, worldwide phenomenon rather than an anecdote, and that self-represented litigants account for more recorded instances than lawyers do.
What it does not support: The true rate. The database counts decisions where a court addressed the point, so instances nobody noticed, or resolved without a written decision, are invisible by construction; its author states this. It is a curated compilation rather than a sampled study, so it cannot support a denominator or a trend rate.

hallucination · legal profession · verification · how will ai change law will human judgement protect lawyers from ai

Peer-reviewed#

Derksen, C. and colleagues (Queen Mary University of London) (2026). Use of AI scribes in UK primary care: a survey of general practitioners #

npj Digital Medicine, 16 May 2026. Read at source 26 September 2026

Method: Cross-sectional online survey of 598 UK general practitioners, September to October 2025, recruited through a market research company.
Finding: About 40 per cent of respondents were using AI scribes and a further 23 per cent had used them; users applied them to a mean of 60 per cent of consultations (range 5 to 100). Over 75 per cent endorsed timeliness benefits such as real-time documentation. Over 60 per cent agreed there are risks of inaccuracies, errors, misinterpretations and associated medicolegal threats. Adoption was higher among men, private practitioners, more experienced GPs and GP trainers.
What it supports: That by late 2025 a substantial share of UK GPs had adopted ambient documentation tools, and that the same population that adopted them largely agreed they carry a risk of error, so use and the expectation of checking arrive together.
What it does not support: Whether GPs check the notes, or how often the notes are wrong; the survey measured use and attitudes, not accuracy or behaviour. The sample over-represents younger and male GPs relative to the workforce, so the proportions are indicative rather than national.

AI scribes · ambient documentation · general practice · professional liability · checking · could a professional be negligent for not using ai

Peer-reviewed#

Education Endowment Foundation and National Foundation for Educational Research (2026). AI tool reduced primary teachers' lesson planning time by a quarter, but views on the tool were mixed #

EEF press release on its trial of Oak National Academy's Aila, 6 October 2026. Read at source 9 October 2026; the evaluation report was not read

Method: Randomised controlled trial commissioned by the EEF and carried out by NFER: 464 Key Stage 2 teachers in 108 primary schools over ten weeks in autumn 2025. One group was encouraged to plan with Aila; the comparison group was told not to use Aila and was free to use other AI tools. Use was measured through weekly teacher diaries, interviews and school visits.
Finding: Teachers using Aila spent an average of two and a half hours a week planning against three hours and 19 minutes, a saving of around 49 minutes (24 per cent). NFER's Ben Styles: 'Lessons planned with the assistance of Aila were found not to be of lower quality'. 81 per cent of the Aila group used it at least once and use declined over the ten weeks; fewer than one in five felt it meaningfully influenced their teaching and planning. 'New teachers and teachers with less confidence in their subject knowledge were more likely to experience reduced lesson planning workload.'
What it supports: That a specialist AI planning tool saved primary teachers time against business as usual, other AI tools included, without a measured loss of lesson quality, and that newer teachers saw the larger effect on workload.
What it does not support: What newer teachers learn about planning when the tool plans with them: the trial did not measure it. How quality was assessed is not on the page read, and the grade here follows the EEF's earlier evaluation record and not a reading of this report. Oak National Academy, which makes Aila, is quoted in the release.

teachers · lesson planning · randomised trial · early-career · workload · does ai make juniors better or only look better

Compiled review#

Kissin, E. (Financial Times) (2026). Junior consultants called back to office as AI increases need for human skills #

Financial Times, 27 August 2026

Method: News reporting. On-the-record interviews with named executives at EY, KPMG, Accenture and Azets, with reference to policies at Deloitte, PwC, BCG, Microsoft and JPMorgan. Not a study, and no measurement of skill or outcome.
Finding: Consulting leaders report that AI has raised the value of interpersonal skills and are considering requiring junior staff in the office more often to develop them. EY's UK head of consulting, Sayeh Ghanbari, is quoted saying firms will "have to reduce flexibility, but in order to help the human skills", and that firms dropped training in empathy, storytelling and leadership during the remote-working period while prioritising AI and technical skills. KPMG describes reinventing in-person training; BCG is expanding office social activities; Azets has lowered its degree requirement and is encouraging four days a week. Deloitte and PwC began extra coaching for their youngest UK recruits in 2023 after finding weaker teamwork and communication than earlier cohorts.
What it supports: That the apprenticeship-erosion argument is now being acted on by the largest professional services firms, named and on the record. It is strong evidence of institutional belief and of policy change.
What it does not support: That AI caused the deficit, or that office attendance repairs it. No measurement appears anywhere in the reporting, the cohort effects described in 2023 are attributed to pandemic lockdowns rather than to AI, and EY as a firm restated its existing flexibility policy alongside its executive's comments. Testimony from interested parties is not evidence of a mechanism.

missing rungs · apprenticeship · early careers · human skills · how will ai change consulting missing rungs

Compiled review#

National Transportation Safety Board (2026). Highway Investigation Report HIR-26-02, Ford BlueCruise collisions #

National Transportation Safety Board, 31 March 2026

Method: Investigation of two fatal crashes in which Ford BlueCruise-equipped vehicles struck stationary vehicles at highway speed: San Antonio, 24 February 2024, and Philadelphia, 3 March 2024.
Finding: Overreliance on the partial automation system appears in the probable cause for both crashes, not merely in discussion: distraction 'stemming from overreliance on the vehicle's hands-free partial automation system' in San Antonio, and 'overreliance on and misuse of' it in Philadelphia. The board recommended that driver monitoring systems detect and warn about 'accumulated short distractions' over a prolonged period.
What it supports: That an investigator placed automation over-reliance in the causal chain rather than treating it as context, and that the monitoring system was defeated by ordinary attention behaving ordinarily rather than by anyone circumventing it.
What it does not support: Anything about knowledge work. Driving is a continuous manual-control task with a monitoring system watching the human, which is not the shape of AI-assisted professional judgement.

automation complacency · over-reliance · capability debt · capability debt what is automation complacency

Argued perspective#

Solicitors Regulation Authority (2026). SRA seeks views on updates to solicitor competence framework #

SRA press release, 8 October 2026, with the linked consultation page. Read at source 9 October 2026

Method: A regulator's consultation on proposed updates to the Statement of Solicitor Competence, introduced in April 2015, and to the functioning legal knowledge that candidates for the Solicitors Qualifying Examination must understand and apply. Open from 8 October to 3 December 2026.
Finding: 'Areas where the Statement of Solicitor Competence is potentially to be updated include the use of technology and AI, supervision, health and wellbeing and professional conduct and ethics.' The consultation page proposes 'a separate section setting out the competences required for effective supervision of others'. Final versions are due in April 2027 and would take effect in September 2027.
What it supports: That the regulator of solicitors in England and Wales proposes to write the use of AI and the supervision of others into its definition of a competent solicitor.
What it does not support: Any decision: these are proposals out for consultation. Nothing about how firms train their juniors today.

solicitors · competence · supervision · professional training · does ai make juniors better or only look better

Self-selected survey#

The Clearing (2026). AI Fatigue in 2026: The State of the Engineer #

The Clearing, April 2026. A commercial site for AI-fatigued engineers, which forecasts in the same report that AI boundary and wellness tools will become a product category. Read in full at source 11 September 2026 and re-read 13 September 2026

Method: 2,147 respondents, collected through The Clearing's own AI Fatigue Quiz between January and March 2026, who then voluntarily completed an optional demographic supplement. Recruited through the site's newsletter, r/cscareerquestions, r/ExperiencedDevs, r/programming and Twitter. Cross-sectional, one wave, no control group, no randomisation and no measured skill: every figure is self-reported perception. The report's own FAQ sets all of this out without being asked, to its credit, and its own sentence is that results 'reflect the experiences of engineers who found and chose to participate in the survey'. The report asks to be cited, in a box headed 'For Journalists and Researchers': 'This report is designed to be cited.'
Finding: 63 per cent 'report measurable decline in at least one core skill they had before adopting AI tools'; 71 per cent agree with 'I often feel like a middleman between AI output and actual results'; 67 per cent spend the majority of coding time reviewing AI output; 58 per cent 'cannot fully explain the code they shipped last month without AI assistance'; 44 per cent have seriously considered leaving their role because of how AI changed the work, while 68 per cent would choose the career again. By declining skill area, debugging from first principles leads at 58 per cent. The report names the pattern 'the Competence Illusion': 'when AI tools make you productive without making you more competent. You ship more. You understand less.'
What it supports: That a population of engineers describing themselves this way exists and is large enough to fill a survey, and that the vocabulary they reach for is loss of authorship rather than fear of replacement. The word for it, the middleman problem, is theirs and is used here as theirs.
What it does not support: Any rate. The frame selects on the outcome: respondents arrived by taking a quiz about AI fatigue, so the proportion reporting AI fatigue cannot be read as a prevalence. The word 'measurable' in 'measurable decline' does no work, because nothing in the report measures a skill; all seven skill-area figures are perceptions. And the intervention table prints effect sizes with confidence labels, 'no-AI coding blocks -2.1 points, Evidence Strength: Strong' and 'taking a break from AI tools entirely -2.8 points, Very Strong', on a cross-sectional self-selected survey with no control arm, which presents a correlation as a measured effect. The report also contradicts itself on one date, placing the r/programming language-model ban in the 2025 row of its own timeline and in April 2026 in the prose beneath it.

the illusion of competence · the middleman problem · deskilling · self-report · what is the illusion of competence is deskilling real

Frontline, clinical and physical work

What AI does to skill outside the office, where most of the world's work happens.

Peer-reviewed#

Cook, R. I., Render, M. and Woods, D. D. (2000). Gaps in the continuity of care and progress on patient safety #

BMJ, 320(7237), 791-794

Method: Conceptual paper in the education and debate section, developed against two publicly documented US clinical accidents.
Finding: Gaps are defined as discontinuities in care, appearing as losses of information or momentum or interruptions in delivery. Most gaps are anticipated and bridged by practitioners, so invisibly that neither outsiders nor insiders recognise the activity as distinct work. Bridging is not elimination: some bridges are frail and easily undone. Accidents occur when conditions overwhelm the mechanisms practitioners use to detect and bridge gaps, from which the authors conclude that efforts to forestall errors by isolating practitioners from the system will misfire. Their worked example is the division of nursing work with less credentialed technicians, which delivers a real economic benefit and restricts the nurse's ability to anticipate gaps.
What it supports: That the boundary between steps in a process is a named object with its own failure modes, and that the work of bridging it is invisible to every measure of output. The nursing example transfers directly to delegating work to agents.
What it does not support: Anything quantified. There is no sample, no rate and no measurement, and nothing in it concerns automation or AI. It is a framing paper.

handoffs · continuity · patient safety · invisible work · oversight · how do humans and agents divide work

Peer-reviewed#

Drew, B. J., Harris, P., Zegre-Hemsey, J. K., Mammone, T., Schindler, D., Salas-Boni, R. et al. (2014). Insights into the Problem of Alarm Fatigue with Physiologic Monitor Devices: A Comprehensive Observational Study of Consecutive Intensive Care Unit Patients #

PLoS ONE, 9(10), e110274. DOI 10.1371/journal.pone.0110274, published 22 October 2014

Method: Observational study of consecutive adults in five adult intensive care units at the University of California, San Francisco, over the 31 days of March 2013. All monitor data, seven ECG leads, pressure, SpO2 and respiration waveforms, user settings and alarms, were stored for 461 patients. Nurse scientists annotated 12,671 arrhythmia alarms against a defined protocol, with inter-rater agreement of 95 per cent on true against false and a Cohen kappa of 0.86. Funded by GE Healthcare.
Finding: 2,558,760 unique alarms occurred in 31 days: 1,154,201 arrhythmia, 612,927 parameter and 791,632 technical. 381,560 were audible, an audible alarm burden of 187 per bed per day. 88.8 per cent of the 12,671 annotated arrhythmia alarms were false positives, and 93 per cent of the 168 true ventricular tachycardia alarms were not sustained long enough to warrant treatment.
What it supports: That a safety control firing at this rate trains the person holding it to ignore it, and that the training is rational rather than negligent. It is the best measured case anywhere of the base-rate problem that makes a stop control nominal.
What it does not support: Anything about AI. The 88.8 per cent applies only to the 12,671 annotated arrhythmia alarms, NOT to all 2.56 million alarms and not to clinical alarms in general; a page stating that 88.8 per cent of clinical alarms are false has misread it. Single centre, one month, five units, industry funded.

alarm fatigue · false alarms · human oversight · base rate · disuse · how do you design a stop button people will use what is alarm fatigue

Statutory investigation#

National Transportation Safety Board (2014). Descent Below Visual Glidepath and Impact With Seawall, Asiana Airlines Flight 214, Boeing 777-200ER, HL7742, San Francisco, California, July 6, 2013 #

NTSB/AAR-14/01, adopted 24 June 2014

Method: Statutory accident investigation with access to flight recorders, crew interviews, manufacturer documentation and operator training records. N is one.
Finding: The pilot flying selected a mode that caused the autoflight system to climb, then disconnected the autopilot and moved the thrust levers to idle, which put the autothrottle into HOLD, a mode in which it does not control airspeed. The board records that neither the pilot flying, the pilot monitoring, nor the observer noted the change in autothrottle mode to HOLD. Contributing factors in the probable cause include the complexities of the autothrottle and autopilot flight director systems being inadequately described in the manufacturer's documentation and the operator's training, which increased the likelihood of mode error. The board attributes insufficient airspeed monitoring in part to automation reliance, and notes the operator's automation policy emphasised full use of automation and did not encourage manual flight in line operations.
What it supports: That a correct automation state change can constitute a failed handoff. Nothing malfunctioned: authority moved from machine to crew and the crew were not told in a way that reached them.
What it does not support: Any frequency of mode confusion. One accident, three crew, one aircraft type. It cannot support a rate and is used here as a demonstration of a mechanism.

mode confusion · handoffs · automation reliance · aviation · how do humans and agents divide work

Peer-reviewed#

Starmer, A. J., Spector, N. D., Srivastava, R., West, D. C., Rosenbluth, G., Allen, A. D. et al. for the I-PASS Study Group (2014). Changes in Medical Errors after Implementation of a Handoff Program #

New England Journal of Medicine, 371(19), 1803-1812

Method: Prospective systems-based intervention study across nine paediatric residency programmes in the US and Canada, January 2011 to May 2013, in three staggered waves with season-matched six-month pre and post periods. 875 consenting residents, 10,740 patient admissions. Active surveillance five days a week, incidents classified by two blinded physician reviewers. Not randomised, no control group.
Finding: The medical-error rate fell 23 per cent, from 24.5 to 18.8 per 100 admissions, and preventable adverse events fell 30 per cent, from 4.7 to 3.3 per 100 admissions, both P<0.001. Near misses and non-harmful errors fell 21 per cent. Non-preventable adverse events did not change, 3.0 against 2.8, P=0.79. Oral handoff duration did not change, 2.4 against 2.5 minutes per patient, P=0.55. Error rates did not change significantly at three of the nine sites, although written and oral handoff processes improved at all nine.
What it supports: That a handoff can be made substantially safer without being made longer, by changing what is transferred rather than how much time is spent transferring it. The unchanged non-preventable rate is the strongest internal evidence the effect is real.
What it does not support: Causation, by the authors' own statement, and not which element of the bundle did the work. Paediatric inpatient units only; generalisation to other specialties is untested. Reviewer agreement was moderate, kappa 0.47 for error classification. Data collectors could not be blinded to period. Funding included an unrestricted medical education grant from Pfizer.

handoffs · patient safety · process design · communication · how do humans and agents divide work

Compiled review#

The Joint Commission (2017). Sentinel Event Alert 58: Inadequate hand-off communication #

The Joint Commission, Issue 58, 12 September 2017

Method: Advisory bulletin aggregating third-party findings. No original data, no sample, no denominator.
Finding: States that inadequate hand-off communication contributes to adverse events including wrong-site surgery, delay in treatment, falls and medication errors. Reports, from cited third parties, that communication failures were responsible at least in part for 30 per cent of US malpractice claims over five years, 1,744 deaths and 1.7 billion dollars in costs; that a typical teaching hospital may experience more than 4,000 hand-offs a day; and that 69 per cent of clinical learning environments had no standardised hand-off process against 20 per cent with some standardisation. Its only quantified outcome evidence is the I-PASS trial.
What it supports: That handoff has been named as a systemic failure point by a national accreditation body, and that the profession's own quantified evidence for it is thinner than the advisory framing implies.
What it does not support: The most quoted claim attached to it. The statement that 80 per cent of serious medical errors involve miscommunication during handoff does not appear anywhere in this alert; the full six pages were read on 4 September 2026. The nearest real figure is Starmer and colleagues citing a Joint Commission statistics page for two of every three sentinel events involving communication failures, which is a different denominator and a broader category. Every figure in the alert is footnoted to a document not read here.

handoffs · patient safety · citation provenance · communication · how do humans and agents divide work

Peer-reviewed#

Weber, D. E., MacGregor, S. C., Provan, D. J. and Rae, A. (2018). 'We can stop work, but then nothing gets done.' Factors that support and hinder a workforce to discontinue work for safety #

Safety Science, 108, 149-160. Safety Science Innovation Lab, Griffith University

Method: Qualitative study. Ten focus groups with workers in a range of roles in the liquefied petroleum gas industry, examining an explicit organisational Authority to Stop an Unsafe Task.
Finding: Stopping work for safety was reported as challenging at the sharp operational end despite the authority existing and carrying no formal penalty. The authors conclude that stopping an unsafe task 'does not solely hinge on the willingness of individual workers to stop, but also depends on contextual factors surrounding the stop work decision'. A participant's account supplies the title: the authority exists, using it produces no drama, and then nothing gets done, so the work resumes as before.
What it supports: That granting an authority is not the same as making it usable, and that the decisive factor is what happens to the work and the worker afterwards rather than the existence of the permission.
What it does not support: Any rate or frequency. Ten focus groups in one industry in one country, qualitative by design, with no measurement of how often stops occurred or should have.

stop-work authority · safety culture · human oversight · organisational design · how do you design a stop button people will use

Peer-reviewed#

Dauth, W., Findeisen, S., Suedekum, J. and Woessner, N. (2021). The Adjustment of Labor Markets to Robots #

Journal of the European Economic Association, 19(6), 3104-3153

Method: German administrative worker and plant data, 1994 to 2014, with a shift-share instrument for robot exposure.
Finding: Incumbent workers largely kept their jobs and moved into new, higher-quality tasks within their original plants. The cost fell instead on young labour-market entrants, who shifted away from vocational manufacturing training towards university.
What it supports: That the damage from automation falls on skill FORMATION rather than skill possession. Twenty years of German manufacturing data making the missing-rungs argument before anyone applied it to knowledge work.
What it does not support: That generative AI will behave like industrial robots. The technologies and the tasks differ substantially.

missing rungs · manufacturing · early careers · missing rungs will ai replace entry level jobs

Peer-reviewed#

Wong, A., Otles, E., Donnelly, J. P. et al. (2021). External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients #

JAMA Internal Medicine, 181(8), 1065-1070. DOI 10.1001/jamainternmed.2021.2626

Method: Retrospective external validation cohort study. 27,697 patients aged 18 or over across 38,455 hospitalisations at Michigan Medicine, 6 December 2018 to 20 October 2019. Sepsis occurred in 7 per cent of hospitalisations.
Finding: The Epic Sepsis Model achieved a hospitalisation-level area under the curve of 0.63 (95% CI 0.62-0.64), against the 0.76-0.83 cited by its developer. At the alerting threshold in clinical use, sensitivity was 33 per cent, specificity 83 per cent, positive predictive value 12 per cent. It did not identify 1,709 of 2,552 septic hospitalisations (67 per cent), 60 per cent of whom received timely antibiotics anyway, while crossing the alert threshold in 18 per cent of all hospitalisations (6,971 of 38,455), requiring eight patients to be evaluated per case of sepsis found.
What it supports: That a proprietary clinical prediction model deployed at national scale can perform far below its developer's stated figures when validated independently, and that nobody had checked. The authors' own conclusion is that widespread adoption despite poor performance raises fundamental concerns about sepsis management nationally.
What it does not support: That all clinical prediction models fail, or that this model performs identically elsewhere. It is one model at one academic health system, and the authors note the theoretical alert burden does not account for real-world trigger criteria and lockouts.

clinical AI · automation bias · alert fatigue · how will ai change medicine is it ethical to let ai judge people

Peer-reviewed#

Kanazawa, K., Kawaguchi, D., Shigeoka, H. and Watanabe, Y. (2022). AI, Skill, and Productivity: The Case of Taxi Drivers #

NBER Working Paper 30612; published in Management Science, 72(2), 1376-1388 (2026)

Method: Driver-level data from a Japanese taxi fleet through the rollout of an AI demand-prediction system.
Finding: Productivity gains accrued almost entirely to LOW-skilled drivers, narrowing the gap between best and worst by 14 percent.
What it supports: That the novice-boost pattern found in customer support and software also appears in manual frontline work.
What it does not support: Included deliberately as a disconfirming case for a tidy story. Anyone arguing that AI levels up white-collar workers while degrading frontline ones has to explain this result, which runs the other way.

productivity · expertise · staying valuable in the age of ai ai and expert judgement

Peer-reviewed#

Kesavan, S., Lambert, S. J., Williams, J. C. and Pendem, P. K. (2022). Doing Well by Doing Good: Improving Retail Store Performance with Responsible Scheduling Practices at the Gap, Inc.#

Management Science, 68(11), 7818-7836

Method: Randomised field experiment across 28 Gap stores in San Francisco and Chicago over nine months, November 2015 to August 2016, analysed as intent-to-treat.
Finding: Restoring schedule predictability and worker control raised productivity 5.1 percent, with sales up 3.3 percent and labour hours down 1.8 percent.
What it supports: That algorithmic optimisation of scheduling is not merely harsh but operationally counterproductive: giving humans back control improved the numbers the algorithm was optimising.
What it does not support: Anything directly about generative AI. It concerns algorithmic scheduling, which is an older and different technology.

algorithmic management · retail · agency · how should leaders respond to ai

Peer-reviewed#

Lang, K., Josefsson, V., Larsson, A.-M. et al. (2023). Artificial intelligence-supported screen reading versus standard double reading in the Mammography Screening with Artificial Intelligence trial (MASAI): a clinical safety analysis #

The Lancet Oncology, 24(8), 936-944. DOI 10.1016/S1470-2045(23)00298-X

Method: Planned interim safety analysis of a randomised, controlled, non-inferiority, single-blinded screening accuracy trial. 80,033 women aged 40-80 screened at four sites in southwest Sweden between April 2021 and July 2022, randomised 1:1 to AI-supported screen reading or standard double reading by two radiologists.
Finding: Cancer detection was six per 1,000 screened women with AI support against five per 1,000 with standard double reading, 41 more cancers detected. The false-positive rate was 1.5 per cent in both arms. Screen readings fell from 83,231 in the control arm to 46,345 in the AI arm, a 44 per cent reduction in screen-reading workload.
What it supports: That a triage-plus-detection-support workflow with a radiologist retaining the recall decision can hold detection while roughly halving reading volume, in a randomised population-based programme.
What it does not support: Patient benefit, which the interim analysis was not designed to test. It is also one mammography device, one AI system, one country, and moderately to highly experienced readers, which the authors state as limits on generalisability.

human-AI collaboration · clinical AI · how will ai change medicine ai and human judgement

Peer-reviewed#

Goh, E., Gallo, R., Hom, J. et al. (2024). Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial #

JAMA Network Open, 7(10), e2440969. DOI 10.1001/jamanetworkopen.2024.40969

Method: Single-blind randomised clinical trial, 29 November to 29 December 2023. 50 US-licensed physicians (26 attendings, 24 residents) in family medicine, internal medicine or emergency medicine, randomised to GPT-4 or to conventional resources, working through up to six clinical vignettes, graded blind against a validated diagnostic reasoning rubric. 244 cases completed.
Finding: Median diagnostic reasoning score per case was 76 per cent (IQR 66-87) with the LLM and 74 per cent (IQR 63-84) with conventional resources: an adjusted difference of 2 percentage points (95% CI -4 to 8, p=0.60). Median time per case was 519 seconds against 565, adjusted difference -82 seconds (95% CI -195 to 31, p=0.20). The LLM alone scored a median 92 per cent, 16 percentage points above the conventional-resources group (95% CI 2-30, p=0.03).
What it supports: That adding a capable model to a physician did not, in this trial, improve diagnostic reasoning or save time, while the same model working alone outperformed both groups of clinicians. The performance of a human-plus-model pairing cannot be inferred from the performance of either part.
What it does not support: That LLMs should diagnose autonomously, which the authors explicitly reject. Six curated vignettes exclude history-taking, examination, context and time, which is most of clinical reasoning. Participants received no prompt-engineering training, and the authors offer prompting and interaction design as explanations.

human-AI collaboration · clinical AI · verification · how will ai change medicine which professions face the greatest deskilling risk ai and human judgement what happens to medical and legal training

Working paper#

Lee, Y. S., Iizuka, T. and Eggleston, K. (2024). Robots and Labor in Nursing Homes #

NBER Working Paper 33116

Method: Original facility-level panel of Japanese nursing homes, using regional robot subsidies as an instrument for adoption.
Finding: Robot adoption RAISED employment and improved retention, most strongly for non-regular staff, reallocated worker effort towards direct care, and improved quality: less use of physical restraint and fewer pressure ulcers.
What it supports: That automation can absorb routine physical work and upgrade the human job rather than hollow it. The cleanest counter-case in this evidence base, drawn from care work rather than knowledge work.
What it does not support: That this generalises. Japanese long-term care faces acute labour shortage, so robots substituted for vacancies rather than for people, which is a very particular condition.

automation · care work · what stays human ai workforce strategy

Peer-reviewed#

Patel, V. R., Liu, M., Worsham, C. M. and Jena, A. B. (2024). Alzheimer's disease mortality among taxi and ambulance drivers: population based cross sectional study #

The BMJ, 387, Christmas issue, 17 December 2024. DOI 10.1136/bmj-2024-082194. PROVENANCE: bmj.com could not be reached from either fetcher on 5 September 2026, so the figures below are taken from BMJ Group's own press release for the paper and the Science Media Centre briefing, both read at source that day, rather than from the full text. Re-read and confirm at bmj.com when access allows.

Method: Population-based cross-sectional study of US death certificates from the National Vital Statistics System, 1 January 2020 to 31 December 2022, covering 443 occupations and nearly 9 million deaths with occupational information. Usual occupation is the one in which the decedent spent most of their working life. Adjusted for age at death and sociodemographic factors. Published in the BMJ's Christmas issue, which is peer reviewed and deliberately light in subject matter.
Finding: 3.9 per cent of deaths (348,328) had Alzheimer's disease listed as a cause. Among 16,658 taxi drivers, 171 did (1.03 per cent); among 1,348 ambulance drivers, 10 did (0.74 per cent). After adjustment, taxi and ambulance drivers had the lowest proportion of any occupation examined (1.03 and 0.91 per cent) against 1.69 per cent for the general population. The pattern did not appear among bus drivers or pilots, who follow predetermined routes, nor for other dementias.
What it supports: That an occupational association exists at national scale, and that it is specific to route-generating rather than route-following driving. The authors' own summary is the right one: 'We view these findings not as conclusive, but as hypothesis generating.'
What it does not support: That navigation protects anyone. Three limitations do most of the damage, two of them raised by Tara Spires-Jones for the Science Media Centre. The drivers died at around 64 to 67 against 74 for other occupations, and Alzheimer's onset is typically after 65, so some may not have lived long enough to develop it. Women were 10 to 22 per cent of the drivers against 48 per cent elsewhere, on a disease women are more likely to develop. And no brain imaging was involved, so the hippocampal mechanism is hypothesis rather than measurement. Robert Howard, reviewing it alongside her, judged it premature to suggest drivers turn off their satnavs to prevent dementia.

spatial memory · The Knowledge · cognitive reserve · deskilling · does gps damage your brain

Peer-reviewed#

Yu, F., Moehring, A., Banerjee, O., Salz, T., Agarwal, N. and Rajpurkar, P. (2024). Heterogeneity and predictors of the effects of AI assistance on radiologists #

Nature Medicine, 30(3), 837-849

Method: 140 radiologists, 15 chest X-ray diagnostic tasks, roughly 5,190 observations, randomised AI assistance, with empirical-Bayes shrinkage to separate genuine individual differences from noise.
Finding: The effect of AI assistance diverged sharply between radiologists, from strongly positive to strongly negative. Experience, subspecialty and prior familiarity with AI all failed to predict who would benefit, and lower performers did not consistently gain.
What it supports: That the effect of AI assistance on expert performance is individual and currently unpredictable, so a policy of giving everyone the tool will help some professionals and harm others with no way to tell in advance which.
What it does not support: That AI assistance is bad on average, or that the pattern holds outside diagnostic imaging.

human-AI collaboration · expertise · human ai decision making staying valuable in the age of ai ai and human judgement ai and expert judgement

Peer-reviewed#

Budzyn, K., Romanczyk, M., Kitala, D. and 18 others (2025). Endoscopist deskilling risk after exposure to artificial intelligence in colonoscopy: a multicentre, observational study #

The Lancet Gastroenterology and Hepatology, 10(10), 896-903. DOI 10.1016/S2468-1253(25)00133-5. A CORRECTION EXISTS and was added to this entry on 23 September 2026: Correction to Lancet Gastroenterol Hepatol 2025; 10: 896-903, volume 10, issue 11, page e12, DOI 10.1016/s2468-1253(25)00294-8, PMID 40946709, confirmed in the Crossref record today. ITS CONTENT HAS NOT BEEN READ: thelancet.com is gated and PubMed returns a CAPTCHA to both fetchers, so nobody here knows what it changes. Recorded in the pattern of the Bastani correction of 12 September 2026, so that a reader who finds a correction notice does not assume the paper was retracted, and so that nobody here assumes it was left unamended. Somebody with journal access should read it and report back.

Method: Retrospective observational study nested in the ACCEPT trial, four Polish endoscopy centres. 1,443 colonoscopies performed WITHOUT AI assistance (795 before and 648 after AI was introduced) by 19 endoscopists averaging 27.6 years of experience, ranging from 8 to 39 years.
Finding: Adenoma detection rate in unassisted colonoscopy fell from 28.4 percent before AI exposure to 22.4 percent after, a drop of 6.0 percentage points (p=0.0089; adjusted odds ratio 0.69).
What it supports: Measured deskilling in highly experienced professionals, in unassisted performance, within months of routine AI exposure. The strongest direct evidence that capability degrades when a tool takes over the judgement, rather than merely a plausible mechanism.
What it does not support: Causation with certainty: it is observational, not randomised, and other changes over the period cannot be fully excluded. It is also one procedure in one country, and detection rate is a proxy for skill rather than skill itself.

deskilling · capability debt · capability debt ai and human judgement how humans learn with ai ai and expert judgement is ai dangerous should ai development be paused what should a board do about the ai safety warnings what happens to medical and legal training is ai making us stupid should there be ai free periods at work what is anthropological regression how do we train people to use ai what are the risks of using ai at work

Peer-reviewed#

Heinz, M. V., Mackin, D. M., Trudeau, B. M. et al. (2025). Randomized Trial of a Generative AI Chatbot for Mental Health Treatment #

NEJM AI, 2(4). DOI 10.1056/AIoa2400802

Method: Randomised controlled trial, N=210 adults recruited by national advertising, with major depressive disorder, generalised anxiety disorder or clinically high risk for feeding and eating disorders. Four-week intervention with Therabot, an expert-fine-tuned generative chatbot built on a hand-curated CBT corpus, against a WAITLIST control, with four weeks of follow-up.
Finding: Statistically significant symptom reductions across all three diagnostic groups at four and eight weeks. 95 per cent of participants engaged, averaging 260 messages and 6.18 hours over four weeks. Therapeutic alliance was comparable to outpatient psychotherapy (mean WAI 3.59). Expressions of suicidal ideation required staff intervention 15 times, and inappropriate responses required correction 13 times.
What it supports: That a purpose-built, expert-curated generative chatbot can produce measurable symptom improvement over four weeks, and that users form a working alliance with it.
What it does not support: That it is safe unsupervised, or that the effect is the chatbot rather than attention and expectation. The control was a waitlist rather than an active comparator, so the treatment effect cannot be separated from the effect of receiving something. The developer confirmed to the FDA advisory committee that humans reviewed all messages in near real time and conducted risk assessments where needed, so the safety record describes a supervised system rather than an autonomous one.

· is it safe to use ai for therapy

Peer-reviewed#

Hennum Nilsson, K., Bodin, T., Strauss, P., Matilla-Santander, N., Badarin, K., Brulin, E. and Hakansta, C. (2025). Algorithmic management is associated with psychological distress, musculoskeletal pain, and occupational accidents: a cross-sectional study in logistics #

International Archives of Occupational and Environmental Health, 98

Method: Online survey of Swedish logistics workers, recruitment and data collection February to July 2024, 978 respondents (592 drivers, 378 warehouse), using an eleven-item algorithmic-management exposure scale covering task allocation, surveillance and performance monitoring, with models adjusted for age, gender, country of birth, education, income, company size, tenure, employment type, contracted hours and union membership. Participants were recruited through paid social-media campaigns on Facebook and Instagram, with two trade unions also circulating the study to members, so this is a self-selected convenience sample rather than a representative one. AUTHOR LIST CORRECTED 23 September 2026: this entry previously read Nilsson, A. et al. The first author is Karin Hennum Nilsson and there are seven authors. PMC metadata lists only the corresponding author, Carin Hakansta, the trap already recorded for the Nature Medicine triad on 5 September 2026.
Finding: Higher exposure to algorithmic management was associated with psychological distress at an adjusted prevalence ratio of 2.12 (95% CI 1.49 to 3.02), occupational accidents at 1.92 (1.22 to 3.01), headaches at 1.68 (1.09 to 2.58) and musculoskeletal pain at 1.54 (1.23 to 1.92). Stratified analyses gave stronger associations for drivers, particularly on psychological distress, headaches and sleep disturbance; warehouse workers were less consistent. All four figures read at source on 23 September 2026.
What it supports: That where AI meets frontline work the output is measured in bodies rather than in output quality, a category almost entirely absent from white-collar productivity research.
What it does not support: Causation: it is cross-sectional and self-reported. Workers under strain may also perceive management as more algorithmic.

algorithmic management · logistics · ai workforce strategy how should leaders respond to ai can i use ai for performance reviews

Peer-reviewed#

Lukac, P. J., Turner, W., Vangala, S., Chin, A. T., Khalili, J., Shih, Y.-C. T., Sarkisian, C., Cheng, E. M. and Mafi, J. N. (2025). Ambient AI Scribes in Clinical Practice: A Randomized Trial #

NEJM AI, 2(12). DOI 10.1056/AIoa2501000. Preprint: medRxiv 2025.07.10.25331333

Method: Parallel three-arm pragmatic randomised clinical trial at one US health system. 238 outpatient physicians across 14 specialties randomised 1:1:1 by covariate-constrained randomisation to Microsoft DAX, Nabla or usual care, 4 November 2024 to 3 January 2025, with the second intervention month compared to baseline.
Finding: Time writing a note fell by an estimated 18 seconds in the control arm, 23 seconds in the DAX arm and 41 seconds in the Nabla arm. Only Nabla differed significantly from control (-9.5 per cent, 95% CI -17.2 to -1.8, p=0.02); DAX did not (-1.7 per cent, 95% CI -9.4 to +5.9, p=0.66). Scribe users improved on Mini-Z burnout (+2.76, p<0.001), task load (-35.8, p=0.01) and work exhaustion (-0.27, p=0.01), with no significant difference between the two products on any psychometric. Roughly 15 per cent of physicians given a tool never used it.
What it supports: That ambient documentation produces a small, product-dependent time saving and a more consistent wellbeing effect, measured against a randomised control rather than against a before-and-after.
What it does not support: A general time saving for ambient AI. One health system, majority female sample, a two-month contract-limited window, and the authors flag that the electronic record's own time metrics do not count editing done inside the scribe platform, so reported savings may be overstated. They note this limitation affects all studies using those metrics.

productivity · clinical AI · how will ai change medicine

Peer-reviewed#

Moore, J., Grabb, D., Agnew, W., Klyman, K., Chancellor, S., Ong, D. C. and Haber, N. (2025). Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers #

Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency (FAccT). DOI 10.1145/3715275.3732039

Method: Mapping review of therapy guides used by major medical institutions to identify the requirements of a therapeutic relationship, followed by several experiments testing current models, including gpt-4o, against those requirements in naturalistic therapy settings.
Finding: Models expressed stigma towards people with mental health conditions and responded inappropriately to common and critical presentations, including encouraging delusional thinking, which the authors attribute to sycophancy. The pattern persisted in larger and newer models.
What it supports: That the failures are structural rather than a matter of model size, and that current safety training does not remove them.
What it does not support: How often this occurs in real use, or what happens with purpose-built clinical systems rather than general-purpose models. These are constructed scenarios, not observed patient interactions.

· is it safe to use ai for therapy

Institutional survey#

Boston Consulting Group; Vinciane Beauchene, Sylvain Duranton, David Martin, Vanessa Lyon, Jeff Walters (2026). AI at Work: Why Strategy Matters More Than Tools #

BCG, June 2026 (fourth edition)

Method: Survey of approximately 12,000 frontline employees, managers and leaders in more than a dozen markets; fieldwork dates are not stated on the publication page. Fourth annual wave, with year-on-year comparisons to 2025.
Finding: 74% of frontline employees describe themselves as AI users, up 23 percentage points from 2025. Among frontline regular users, 42% report saving at least a workday per week, but 66% receive limited or no guidance on what to do with the saved time. 30% say their organisation has integrated AI agents into workflows, against 13% a year earlier, and more than six in ten believe agents could do at least half of their job within three years. 72% say the skills expected of them have shifted and 36% feel they have received adequate upskilling. Organisations using AI to reshape workflows end to end or invent new business models rose to 42% from 22%. Employees with a clear strategy but limited tool access report better outcomes than those with strong access but no direction.
What it supports: That frontline self-reported use has caught up with managers, and that direction and training lag access to tools.
What it does not support: Time saved and outcomes are self-reported and unverified; a 'workday per week' is a perception. BCG sells AI strategy work. Sampling frame and country weighting are not published on the page.

frontline employees · time savings · ai agents · upskilling gap · strategy clarity

Qualitative field study#

Ehsan, U., Passi, S., Saha, K., McNutt, T., Riedl, M.O. and Alcorn, S. (2026). From Future of Work to Future of Workers: Addressing Asymptomatic AI Harms to Foster Dignified Human-AI Interaction #

Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI '26), ACM. DOI 10.1145/3772318.3791081. Preprint arXiv:2601.21920, 29 January 2026

Method: Twelve months of situated fieldwork through the first year of routine use of a commercial AI-assisted radiotherapy treatment-planning system across a five-site North American hospital group. 42 participants: 15 radiation oncologists, 12 medical physicists, 7 dosimetrists, 8 administrators, with 2 to 24 years of experience. 52 think-aloud sessions, 24 interviews staged across months 2 to 11, five participatory workshops, 63 hours transcribed, analysed by grounded theory.
Finding: Measured operational gains and reported capability loss ran together. Planning cycles shortened by roughly 15 per cent and confidence rose, while by month nine several dosimetrists said their unaided proficiency had worsened over the year, one saying they had grown slower without the tool. The authors name two things the research has nowhere else: intuition rust, the gradual dulling of expert judgement beneath intact output, and identity commoditisation, the erosion of professional dignity as practitioners describe becoming AI babysitters, button-pushers and bystanders in their own practice. They call these harms asymptomatic because the organisation's own measures showed only the improvement.
What it supports: That the instruments an organisation uses to judge an AI deployment can register the gain and be structurally blind to the cost, and that practitioners perceive the loss long before any dashboard does. It is the best available account of what deskilling feels like from inside a high-stakes clinical specialty, and the only source here on occupational identity.
What it does not support: Any measured deskilling. The skill claims are self-reported, plus one informal unaided exercise with a couple of dosimetrists during a workshop. There is no controlled comparison, no pre and post measurement and no quantified skill outcome, so it cannot be set beside Budzyn as a second measured result. IMPORTANT, this study is widely misdescribed in secondary summaries as radiologists using generative AI. It is neither. The participants plan radiotherapy treatment rather than interpret images, and the system is optimisation-based, not generative. Do not repeat that description.

deskilling · capability debt · occupational identity · intuition rust · identity commoditisation · the invisible work of oversight · capability debt which professions face the greatest deskilling risk ai and human judgement what happens to medical and legal training what is the new imposter syndrome

Peer-reviewed#

Gommers, J., Lang, K., Hofvind, S. et al. (2026). Interval cancer, sensitivity, and specificity comparing AI-supported mammography screening with standard double reading without AI in the MASAI study #

The Lancet. DOI 10.1016/S0140-6736(25)02464-X. Published 29 January 2026

Method: Full results of a randomised, controlled, non-inferiority, single-blinded, population-based screening-accuracy trial. Over 100,000 women screened at four Swedish sites between April 2021 and December 2022, with two years of follow-up.
Finding: Interval cancers fell from 1.76 per 1,000 women (93/52,872) in the control arm to 1.55 per 1,000 (82/53,043) in the AI arm, a 12 per cent reduction. There were 16 per cent fewer invasive (75 v 89), 21 per cent fewer large (38 v 48) and 27 per cent fewer aggressive-subtype (43 v 59) interval cancers. Cancers detected at screening rose from 74 per cent (262/355) to 81 per cent (338/420) of all cases. False positives were 1.5 per cent in the intervention arm and 1.4 per cent in the control arm.
What it supports: That AI-supported screen reading, with a radiologist retaining the recall decision, reduced interval cancers over two years in a randomised population-based programme. The only outcome-level randomised evidence of this kind in clinical AI.
What it does not support: Generalisation beyond one country, one mammography device, one AI system and experienced readers, all stated as limitations by the authors. Mortality was not an endpoint, and cost-effectiveness was not assessed. The first author's own summary states the design does not support replacing radiologists, since at least one still reads every case.

human-AI collaboration · clinical AI · how will ai change medicine ai and human judgement

Working paper#

Hosseini Maasoum, S. M. and Lichtinger, G. (2026). Generative AI as Seniority-Biased Technological Change: Evidence from U.S. Resume and Job Posting Data #

SSRN working paper, DOI 10.2139/ssrn.5425555. Harvard University. First version 31 August 2025; version read is dated 25 May 2026 on its own masthead, obtained from the author's site. SSRN records the current revision as 6 June 2026, 109 pages. NOT peer-reviewed, no journal or NBER placement. Main text read in full at source 9 September 2026; the supplemental appendix, A.1 to A.23, was NOT read

Method: Revelio Labs resume data via WRDS, LinkedIn-derived, merged with Revelio job postings. 281,111 firms, 161,192,969 positions from 2015, roughly 66 million unique workers, and 198,773,384 postings from 2021. Firms included if they recorded at least 20 new positions between January 2021 and March 2025; HR intermediaries and the roughly 800 largest firms excluded. Juniors are Revelio's Entry and Junior seniority levels, seniors Associate and above. Adoption is identified from job postings for people hired to integrate generative AI into the firm, by keyword flag then LLM classification: 131,845 of 198.8 million postings, 0.066 per cent, giving 10,433 adopter firms, 3.71 per cent of firms and 16 per cent of employment. Four designs against non-adopting controls: difference-in-differences, triple-difference with firm-by-time and industry-by-seniority-by-time fixed effects, a triple-difference by occupational exposure, and a staggered event study using Callaway and Sant'Anna. Errors clustered at firm level. A separate task-level analysis maps 355,013 postings from 1,000 firms onto O*NET tasks and tests how task bundles change.
Finding: Junior employment at adopting firms fell about 9 per cent relative to non-adopters six quarters after diffusion, and 8 per cent eight quarters after adoption in the staggered design, while senior employment showed no comparable break. The fall concentrates in exposed occupations: high-exposure junior employment contracts roughly 7 log points against low-exposure between 2022Q4 and 2025Q1, while the senior coefficient continues to rise. The decomposition attributes the fall to hiring, not exits. Junior hiring fell by 4.006 per firm-quarter (SE 0.221) against a pre-period mean of 5.059, which the authors describe as about an 80 per cent reduction. Junior separations ALSO fell, by 1.089 (SE 0.163), roughly a quarter of the hiring decline. Promotion rates rose slightly, by 0.033 percentage points (SE 0.014). At task level, a one standard deviation increase in a task's exposure is associated with roughly a four percentage point larger contraction in junior task bundles at adopters, with the senior interaction running the other way.
What it supports: The first firm-level, within-firm measurement of what happens to junior employment when a firm adopts generative AI, at a scale no other study has. Pre-trends are flat back to 2015, covering an earlier tightening cycle. A postings placebo points to falling demand rather than a labour supply shock. The decisive contribution is the mechanism: this is a hiring story, not a redundancy story. Separations fell. Firms stopped opening the door rather than pushing people out of it, which is why the effect is invisible in unemployment figures and visible only to people who cannot get in. The task-level test shows exposed tasks being removed from junior job descriptions specifically, which is the missing-rungs claim measured directly rather than inferred.
What it does not support: Causation. The authors write throughout that adoption IS ASSOCIATED WITH the decline, call the evidence suggestive, and state in their conclusion that unobserved confounders may remain and that the window, 2023 to 2025, is short. It is an unrefereed working paper: no journal, no NBER number, and the headline employment estimates are read off event-study figures with no printed standard errors, so only the flows and task tables carry reportable precision. The data are LinkedIn profiles, which tilt towards managerial, professional and high-information work; the authors now show that tilt is stable across the period and absorbed by fixed effects, which protects the comparison but does not make the sample representative of the labour force. The adoption measure misses informal use inside firms, which the authors say should bias estimates towards zero. They cannot test the mechanism they propose, which is that firms cut junior hiring in anticipation rather than because tasks were already automated. And they flag the missing intercept problem themselves: this does not aggregate to an economy-wide claim without further assumptions. Nothing here shows that the juniors not hired were worse off, or that the capability they would have built is gone rather than delayed.

missing rungs · synthetic seniority · early careers · capability debt · missing rungs will ai replace entry level jobs do apprenticeships still work what happens to medical and legal training hiring signals when effort is free

The international picture

Evidence from outside the US, UK and Nordic economies, including sources published in other languages.

Institutional modelling#

Digitaliseringsstyrelsen (Danish Agency for Digital Government) (2018). Vejledning om digitaliseringsklar lovgivning #

VEJ nr 9590 af 12/07/2018, mandatory for government bills since 1 July 2018, status Gaeldende as at September 2026

Method: Binding guidance on the drafting of Danish government legislation. Every government bill is assessed against seven principles before it goes to Parliament.
Finding: The third principle carries a written checklist question: 'Er det sikret, at det fagprofessionelle skon er opretholdt i tilfaelde, hvor hensynet til borgernes retssikkerhed taler herfor?' (Has it been ensured that professional discretion has been preserved in cases where regard for citizens legal certainty so requires?) The same document states that objective rules should be used only where it makes sense and where professional discretion is not needed, and places a residual duty on the authority to ensure discretion continues to be exercised. The 2018 wording is advisory; the Ministry of Justice s current legislative-quality guidance states that new legislation shall be digital-ready.
What it supports: That a state has built a compulsory written checkpoint on whether human judgement survives a change of process, answered by a named official before a parliamentary vote. It is a procedural instrument for preserving discretion rather than a rule about technology.
What it does not support: That discretion is in fact preserved. It measures a drafting process, not outcomes, no published review of how the question is answered was found, and it binds those who draft legislation rather than those who decide individual cases.

professional discretion · human judgement · oversight · legal certainty · what board oversight of ai looks like

Statutory investigation#

Deputy Parliamentary Ombudsman of Finland (Maija Sakslin) (2019). Decision on the Tax Administration s automated decision making #

EOAK/3379/2018, decision of 20 November 2019

Method: Own-initiative investigation by the Parliamentary Ombudsman into the legality of automated decision making at the Finnish Tax Administration, which issued on the order of fifteen million decisions a year.
Finding: The Ombudsman found the practice unlawful. The reasoning turns on accountability rather than error: official accountability had become indirect (virkavastuu jaa valilliseksi), because no identifiable official could be said to have made the decision. Parliament subsequently legislated chapter 8 b of the Administrative Procedure Act, in force 2023, and the Chancellor of Justice applied the new law against Kela in April 2025.
What it supports: That a supervisory body identified the loss of an answerable human as the defect, independent of whether the decisions were correct, and that a legislature acted on that finding. Four bodies over six years reaching the same conclusion about the same problem is an unusually complete chain.
What it does not support: That any decision was wrong. The finding is about accountability structure, not accuracy, and the Ombudsman did not measure outcomes. It concerns Finnish administrative law and does not transfer to private-sector decisions.

accountability · automated decision · oversight · moral crumple zone · what is a moral crumple zone

Institutional modelling#

National Assembly of Quebec (2021). Act respecting the protection of personal information in the private sector, s 12.1 #

CQLR c P-39.1 s 12.1, inserted by SQ 2021 c 25 s 110, in force 22 September 2023. Public-sector mirror at CQLR c A-2.1 s 65.2 (SQ 2021 c 25 s 21). Consolidation current to 7 April 2026

Method: Provincial statute, read at the official consolidated text in both official languages. One version only, unamended since coming into force.
Finding: Where an enterprise uses personal information to render a decision based exclusively on automated processing, it must say so by the time it communicates the decision, and on request give the information used, the reasons and principal factors, and the right of correction. It then requires that 'the person concerned must be given the opportunity to submit observations to a member of the personnel of the enterprise who is in a position to review the decision'. The French is impersonal: 'Il doit etre donne a la personne concernee l occasion de presenter ses observations a un membre du personnel de l entreprise en mesure de reviser la decision.'
What it supports: That a jurisdiction has legislated not a right to an explanation but a right to put arguments to a named human with authority to change the answer. It is the strongest human-review provision found in North America and it names a capacity, being in a position to review, rather than a role.
What it does not support: That it is used, or that the reviewing person is competent to redo the analysis. The statute does not define what being in a position to review requires. Quebec s own regulator lists four limits, graded separately below. It applies only to decisions based EXCLUSIVELY on automated processing, which excludes most decisions in practice.

human review · automated decision · meaningful human involvement · oversight · who supervises work they cannot do

Institutional modelling#

Korea Development Institute (KDI) (2023). Changes in the labour market due to artificial intelligence and policy directions (Research Report 2023-03)#

KDI, published in Korean

Method: Expert and GPT-4 capability ratings applied to Korean occupational profiles, a KDI survey of 800 firms in September 2023, and firm-panel econometrics.
Finding: 38.8 percent of jobs are technically automatable across more than 70 percent of their tasks, yet only 2.7 percent of firms with ten or more staff had adopted AI. Realised effects showed no aggregate employment change, lower earnings, and the impact concentrated on YOUNGER, tertiary-educated workers and women.
What it supports: That the gap between technical potential and actual adoption is enormous, and that where effects appear they fall on the young and educated rather than the low-skilled.
What it does not support: That the Korean pattern transfers. Korea has unusually high tertiary education rates and a distinctive labour market.

labour market · early careers · international · will ai replace entry level jobs will ai replace my job

Institutional modelling#

Parliament of Finland (2023). Administrative Procedure Act, chapter 8 b, automated decision making #

Hallintolaki ss 53 e to 53 g, inserted by act 487/2023 of 23 March 2023, in force 1 May 2023, with companion act 488/2023

Method: General administrative statute, not an AI-specific instrument. Applies to all automated decision making by Finnish authorities.
Finding: An authority may decide a matter automatically only where the matter 'contains no elements requiring case-by-case discretion' (johon ei sisally seikkoja, jotka edellyttavat tapauskohtaista harkintaa). A decision counts as automated where it is reached without a natural person checking and approving it. Section 53 e(4) provides that a request for rectification may not be decided automatically. The companion act requires a named individual responsible for each system, a published deployment decision, and records permitting five-year reconstruction of the stages at which a natural person took part.
What it supports: That the human-discretion test can be stated in a single clause of general administrative law, and that a legislature has done so. It defines what makes a matter unsuitable for automation, which most AI frameworks require oversight without ever specifying.
What it does not support: That it works, or that Finnish authorities classify matters correctly. It binds public authorities only, not private employers. Finland s implementation of the EU AI Act is separately late, so this is administrative-law strength rather than AI-law strength.

case-by-case discretion · automated decision · human review · oversight · human in the loop is not a safeguard

Institutional survey#

Saudi Data and AI Authority (SDAIA) (2023). AI Ethics Principles, Version 1.0 #

SDAIA, September 2023

Method: National AI ethics guidance, seven principles with an assessment checklist annexe covering the AI system lifecycle. Non-binding guidance rather than statute. Version 1.0 read in full at source; whether a later version exists could not be checked because SDAIA's main host rejects automated requests.
Finding: The checklist annexe puts this question to designers at the plan and design stage: "Does your AI system design prevent overconfidence in or overreliance on the AI system with necessary human intervention mechanisms?" It also asks whether human oversight processes carry defined KPIs and assigned responsibility. The operative text states that decisions which are irreversible or life-and-death "should trigger human oversight and final determination", and rules out social scoring and mass surveillance.
What it supports: That over-reliance on AI has been named as a design defect to be engineered against in a national governance instrument. On the evidence gathered here it is the only Gulf instrument that does so.
What it does not support: Any obligation. It is guidance, not law, the over-reliance language sits in a checklist annexe rather than in the principle text, and Saudi Arabia had no binding AI statute as of August 2026, only a draft responsible AI policy out for consultation from 2 April 2026. Version 1.0 dates from September 2023 and may have been superseded.

human oversight · over-reliance · Saudi Arabia · AI ethics · ai and work in the gulf

Institutional survey#

Government of the United Arab Emirates (2024). The UAE Charter for the Development and Use of Artificial Intelligence #

UAE Legislation portal, issued 10 June 2024

Method: National charter of twelve principles. Statement of principle with no duty-holder, no enforcement mechanism and no competence requirement.
Finding: Principle 6, Human Oversight, "emphasizes the irreplaceable value of human judgment and human oversight over AI, aligning with ethical values and social standards to correct any errors or biases that may arise". The charter is silent on deskilling, over-reliance and any obligation to train or assess the humans doing the overseeing.
What it supports: That human oversight is stated as a national principle in the UAE.
What it does not support: That it is operational. There is no named duty-holder, no enforcement, no competence standard and no test of whether oversight is real. The UAE had no federal AI statute as of August 2026.

human oversight · United Arab Emirates · governance · ai and work in the gulf

Institutional modelling#

Institut fur Arbeitsmarkt- und Berufsforschung (IAB), Germany (2024). Folgen des technologischen Wandels fur den Arbeitsmarkt (Consequences of technological change for the labour market: it is above all the highly qualified who feel digitalisation)#

IAB-Kurzbericht 5/2024, published in German

Method: The Substituierbarkeitspotenziale series, 2022 wave. Three independent coders score more than 9,000 tasks in the BERUFENET expert database across roughly 4,600 occupations.
Finding: Substitutability rose about ten percentage points for degree-level expert occupations between 2019 and 2022, and was roughly flat for helper occupations. IAB frames AI as relief for skills shortages rather than as displacement.
What it supports: That the German expert assessment puts the pressure on the highly qualified, which inverts the assumption that automation threatens the least skilled first.
What it does not support: Realised outcomes. Substitutability is technical potential assessed by coders, not what employers did.

skills · deskilling · international · will ai replace my job human skills in the age of ai

Institutional survey#

LaborIA (French Ministry of Labour, Inria and Matrice) (2024). Etude des impacts de l'IA sur le travail: Rapport d'enquete LaborIA Explorer (Study of the impacts of AI on work)#

LaborIA, published in French

Method: Telephone survey of 250 decision-makers in firms with more than 50 staff (42 with AI deployed), longitudinal interviews with 10 decision-makers across three waves, and six ethnographic field sites.
Finding: Names a conflit de rationalite, a clash of rationalities: managers justify AI by error reduction (81 percent), performance (75 percent) and removing drudgery (74 percent), while fieldwork shows workers becoming the system's de facto trainers.
What it supports: That what management believes AI is doing and what workers experience it doing can diverge systematically inside the same organisation. A work-psychology and ergonomics frame rather than task-exposure modelling.
What it does not support: Scale. The qualitative core rests on six sites and ten repeated interviews.

organisation design · international · ai workforce strategy how should leaders respond to ai

Institutional survey#

BAuA, ZEW, IAB and BIBB, Germany (2025). Digitalisierung und Wandel der Beschaftigung, DiWaBe 2.0 (Digitalisation and the transformation of employment)#

Bundesanstalt fur Arbeitsschutz und Arbeitsmedizin, published in German

Method: Representative 2024 survey of roughly 9,800 employees subject to social insurance, linkable to administrative employer and employee records.
Finding: More than half already use AI at work but largely informally. Use ranges from about a third of unqualified workers to around 80 percent of those with a degree or Meister qualification. There was NO difference in training participation between AI users and non-users.
What it supports: That AI use is spreading through workplaces without any corresponding increase in training, which is the adoption-without-redesign pattern measured directly at national scale.
What it does not support: What that absence of training does to capability over time. The survey is a single 2024 snapshot.

AI adoption · skills · international · ai workforce strategy chro guide to ai

Institutional survey#

Bundesinstitut fuer Berufsbildung (2025). Angespannte Lage auf dem Ausbildungsmarkt (Tense situation in the training market)#

BIBB press release 40/2025, 10 December 2025

Method: Official German statistics on the dual vocational training system, counting newly concluded training contracts, unfilled training places and unplaced applicants as at 30 September 2025.
Finding: Around 476,000 new dual training contracts in 2025, down 2.1 per cent on 2024 and the second consecutive annual fall. 54,400 training places went unfilled. At the same time around 84,400 young people had not found a training place, up 19.9 per cent and the highest since 2010, with the share of unsuccessful applicants at 15.1 per cent, the highest since the end of the 2009 financial crisis.
What it supports: That the world's most admired apprenticeship system is failing to match young people to places in both directions at once: record unplaced applicants alongside tens of thousands of empty places. The bridge is not merely narrowing, it has stopped connecting.
What it does not support: That AI caused it. BIBB attributes the tension to matching problems, regional and occupational mismatch and economic conditions, and makes no AI claim.

apprenticeships · Germany · dual system · early careers · matching · do apprenticeships still work

Argued perspective#

Commission d acces a l information du Quebec (2025). L IA au travail: pour un meilleur encadrement #

Memoire presented to the ministere du Travail, dated 27 January 2025, published 21 February 2025

Method: Submission by the provincial access and privacy regulator to a government consultation on the digital transformation of workplaces. A reasoned position, not a study.
Finding: On meaningful human intervention the Commission states, at page 4: 'lorsqu un humain enterine une decision proposee par un systeme d IA sans etudier l ensemble de l analyse, il existe un risque qu il demontre un biais d automatisation en faisant exagerement confiance a ce systeme'. It then lists four limits of sections 12.1 and 65.2: no duty to disclose at collection, no application to decisions that are not fully automated, no criterion for what counts as a decision, and the burden on the individual to ask. Recommendation 6 proposes prohibiting fully automated decisions with significant effects on employees.
What it supports: That a North American regulator has stated in writing that the human who ratifies without examining the whole analysis is where the safeguard fails, and has named automation bias as the mechanism. It is a regulator arguing against the sufficiency of its own jurisdiction s provision.
What it does not support: Anything measured. It is a submission, it contains no data on how often ratification without examination occurs, and its recommendations had not been enacted at the time of reading.

automation bias · meaningful human involvement · human review · oversight · what is automation bias

Institutional survey#

Eurostat (2025). Artificial intelligence use by enterprises (isoc_eb_ai), 2025 reference year #

Eurostat news release, 11 December 2025, and statistical report KS-01-26-009-EN-N. Fieldwork Q1 2025

Method: Harmonised survey of enterprises with ten or more persons employed across NACE C to J, L to N and 95.1, excluding the financial sector, agriculture and the public sector. Approximately 157,000 enterprises surveyed from a population of about 1.53 million. An enterprise counts as a user if it used at least one of eight named AI technologies.
Finding: EU-27 average 19.95 per cent, up 6.47 points on 2024. Denmark first at 42.0 per cent, Finland second at 37.8, Sweden 35.0, Netherlands 33.2 (break in series), Spain 20.3, Portugal 11.5, Romania lowest at 5.2. Norway, reporting as a non-EU EEA state, 28.9 per cent. On individuals, a companion series puts Denmark highest in the EU at 48.4 per cent and Norway highest in Europe at 56.3.
What it supports: A comparable cross-national baseline for enterprise AI adoption, collected to a documented standard, which allows countries to be ranked against each other rather than against vendor surveys.
What it does not support: Depth of use. An enterprise counts if it used one of eight technologies once, so the measure says nothing about how many workers use AI, how often, or for what. It excludes the financial and public sectors and firms under ten employees. National statistics offices publish figures on different populations, so a national number and a Eurostat number for the same country are frequently not comparable.

adoption · international comparison · enterprise AI use · ai and work by country

Institutional survey#

General Authority for Statistics (GASTAT), Saudi Arabia (2025). Establishments ICT Access and Usage Statistics 2025 #

GASTAT, 2025

Method: National statistical survey of establishments, methodology stated as aligned with UNCTAD international standards. Enterprise-side only.
Finding: 33.1 per cent of establishments use artificial intelligence technologies, a growth of 20.0 per cent against 2024. By sector: information and communication 61.1 per cent, financial and insurance 52.9 per cent, education 51.0 per cent, manufacturing 30.7 per cent, wholesale and retail 30.2 per cent. The companion household and individuals survey for the same year contains no AI indicator at all.
What it supports: That one Gulf state measures enterprise AI adoption with a published, standards-aligned method. It is the only official national AI adoption statistic found across Saudi Arabia, the UAE and Qatar.
What it does not support: Anything about citizens, or about employment. Saudi Arabia measures enterprise adoption but not individual use, and publishes no AI employment series. Adoption is also self-reported use of a technology category, not a measure of capability or of value obtained.

adoption · official statistics · Saudi Arabia · international · ai and work in the gulf ai and work by country

Institutional survey#

Japan Institute for Labour Policy and Training (JILPT) (2025). Survey on the impact of workplace AI adoption on working styles (Research Series No. 256)#

JILPT, published in Japanese, designed with the OECD

Method: 22,000 employees, stratified on 2020 Census occupation, employment type, sex and age. Fieldwork May to June 2024.
Finding: Only 12.9 percent report any firm AI use and 8.4 percent use it themselves. Among users, reports of improved job quality and wellbeing outweighed reports of decline, and the gain was markedly larger where the employer had consulted staff and funded training.
What it supports: That the effect of AI on how work feels is conditional on how the employer introduced it, not determined by the technology. Direct empirical support for the design-rather-than-drift argument, from a 22,000-person sample.
What it does not support: Long-run capability effects, and Japanese adoption rates are far below US levels so the user group is early and unusual.

AI adoption · agency · international · ai workforce strategy design versus drift

Institutional modelling#

Lu, Y. and Gui, L. (Bulletin of the Chinese Academy of Sciences) (2025). Analysis of the impact of artificial intelligence technology on employment and income in China #

Bulletin of the Chinese Academy of Sciences, 40(4), 642-651, published in Chinese

Method: Policy synthesis of the Chinese empirical literature and official statistics, under National Social Science Fund major project 23ZDA100.
Finding: Between 2018 and 2023 the substitution effect outweighed complementarity: a one percent rise in industrial robots reduced firm labour demand by 0.18 percent. Roughly 200 million people, 27 percent of employment, are in flexible work with 37 percent social-insurance coverage.
What it supports: That in the largest manufacturing economy the measured balance so far has been substitution rather than augmentation, and the policy response centres on social security redesign rather than retraining.
What it does not support: Comparability with service-sector generative AI in high-income economies. This is largely industrial robotics.

labour market · manufacturing · international · will ai replace my job

Operator account#

Moquim, S. A., Vice President, Saudi Data and Artificial Intelligence Authority, interviewed by Saeed al-Abyad (2025). SDAIA: Saudi AI Platform Baseer Boosts Crowd, Security Control During Hajj #

Asharq Al-Awsat, dateline Jeddah, 4 June 2025; read at source 4 September 2026

Method: On-the-record interview with the operating authority's vice president. Descriptive throughout, with no evaluation and no data.
Finding: SDAIA operates Baseer, built with the Ministry of Interior, using AI algorithms and computer vision on live feeds to detect crowd density and distribution within the Grand Mosque and to pinpoint overcrowded zones such as the Tawaf area moment by moment, stated as enabling authorities to act swiftly to prevent overcrowding or stampedes. Companion platforms Sawaher and Sawaher Qiyada analyse live security camera feeds. The Smart Makkah Operations Center coordinates them, and biometric systems run at 12 international airports across 8 countries under the Makkah Route initiative.
What it supports: That the same public authority which published Saudi Arabia's AI ethics principles, including the requirement that irreversible or life-and-death decisions trigger human oversight and final determination, also builds and operates the system to which that requirement applies.
What it does not support: That the oversight works, or that anyone has tested it. Nothing here measures how often a commander overrides the system, what happens when the prediction is wrong, or whether unaided crowd-reading skill is maintained. The speaker heads the authority whose systems he is describing.

human oversight · automation bias · who supervises · computer vision · ai and work in the gulf what is meaningful human oversight who supervises work they cannot do

Institutional survey#

National Centre for Vocational Education Research (2025). Apprentices and trainees 2025 #

NCVER, Australia, March and June quarter releases 2025

Method: Official Australian administrative counts of apprentice and trainee training contracts.
Finding: 320,830 in-training contracts at 31 March 2025, down 7.9 per cent year on year, with trade down 3.2 per cent and non-trade down 17.9 per cent. By the June quarter the fall was 11.3 per cent, with non-trade down 20.2 per cent. NCVER attributes the non-trade decline partly to the conclusion of the Boosting Apprenticeship Commencements subsidy.
What it supports: That entry-level training volume responds sharply and quickly to subsidy withdrawal, and that the response is concentrated in the non-trade occupations closest to office work.
What it does not support: An AI effect. NCVER names policy change, and the timing follows the 2022 subsidy withdrawal rather than any AI milestone.

apprenticeships · Australia · early careers · subsidy · do apprenticeships still work

Institutional modelling#

Azim Premji University (2026). State of Working India 2026: Youth in the Labour Market #

Azim Premji University, Bengaluru

Method: Analysis of National Sample Survey employment data 1983 to 2011, all quarters of the Periodic Labour Force Survey 2017 to 2024, plus AISHE, NCVT-MIS and CMIE-CPHS.
Finding: Roughly five million graduates enter the Indian labour market each year against about 2.8 million finding work, with graduate unemployment near 40 percent for 15 to 25 year olds. The report explicitly declines to attribute this to AI.
What it supports: That in the world's most populous labour market the early-career crisis is a demand-side bottleneck that long predates AI. A necessary corrective to reading every graduate hiring problem as an AI story.
What it does not support: Anything about AI's effect in India, which it deliberately does not claim.

early careers · jobs · international · will ai replace entry level jobs

Compiled review#

Cedefop (2026). Call for abstracts: apprenticeships in the age of AI #

Cedefop, European Centre for the Development of Vocational Training, 5 August 2026

Method: Framing statement for the 2027 joint Cedefop and OECD apprenticeship symposium. Position paper, not a study.
Finding: Cedefop states: "AI takes over baseline tasks which were in many cases performed by apprentices or apprenticeship graduates who get entry-level roles once their programmes are completed. As apprenticeships continue to expand into new fields and occupations, they may also be exposed to a decline in entry-level openings." It adds that the contraction appears concentrated in white-collar roles, so the effect is likely to be uneven and occupation-specific rather than uniform.
What it supports: That a European Union agency has now named the mechanism this research describes, in its own words, and applied it specifically to apprenticeships. It is the clearest institutional statement that the training route and the entry-level job are the same thing.
What it does not support: Anything measured. It is a call for papers, which means Cedefop is asking the question rather than answering it, and it says the effect is uneven.

apprenticeships · AI · entry-level · European Union · missing rungs · do apprenticeships still work missing rungs

Institutional survey#

Census and Statistics Department, Hong Kong SAR (2026). Report on the Survey on Information Technology Usage and Penetration in the Business Sector, 2025 Edition #

Census and Statistics Department, Hong Kong, released 27 February 2026

Method: National business survey, fieldwork March to December 2025. Full report and all thirty tables read at source, together with three companion household and ICT publications.
Finding: Artificial intelligence appears nowhere in the report: not in the tables, not in the explanatory notes, not in the definitions. The survey's ICT categories are cloud computing at 98.1 per cent, QR codes at 37.6 per cent, RFID at 20.3 per cent, internet of things at 7.4 per cent and augmented or virtual reality at 1.5 per cent. Hong Kong's statistical office measures AR and VR adoption and does not ask about AI at all. The 2023 edition also had no AI category, and no plan to add one has been announced.
What it supports: That Hong Kong publishes no official statistic on AI adoption or AI-related employment, which means every circulating Hong Kong AI adoption figure comes from a non-government survey with a self-selected sample.
What it does not support: That AI adoption in Hong Kong is low. It establishes that it is unmeasured by the government, which is a different and in some ways more useful fact.

official statistics · Hong Kong · measurement · absence · ai and work in asia

Institutional modelling#

INSEE, France (2026). Note de conjoncture: digital investment, artificial intelligence and youth employment #

Institut national de la statistique et des etudes economiques, March 2026, published in French

Method: National accounts and quarterly employment files, with an error-correction model estimated 1990Q1 to 2019Q4.
Finding: French employment of 15 to 29 year olds, excluding apprentices, fell 7.4 percent year on year in IT services, 5.8 percent in publishing and 3.7 percent in management consulting in Q4 2025, against minus 0.7 percent across the market sector overall.
What it supports: A European national-statistics office finding the same entry-level pattern reported in the US, which makes the signal considerably harder to dismiss as an American artefact.
What it does not support: That AI caused it. INSEE explicitly cautions against attributing the fall to AI alone.

early careers · jobs · international · will ai replace entry level jobs will ai replace my job

Institutional survey#

International Labour Organization (2026). Global Employment Trends for Youth 2026: Back to the future #

ILO, 11 August 2026

Method: ILO global modelled estimates of youth employment, unemployment and NEET status.
Finding: Global youth unemployment 12.4 per cent in 2025, 67 million people aged 15 to 24. The NEET rate is 20 per cent, over 257 million. Youth unemployment rose in 8 of 11 subregions between 2023 and 2025, with Northern America rising from 8.3 to 9.8 per cent. The ILO estimates 6.1 per cent of jobs held by workers aged 15 to 29 are in occupations highly exposed to AI-related change.
What it supports: That youth labour market conditions deteriorated across most of the world between 2023 and 2025, which is the context any national apprenticeship policy is operating in.
What it does not support: That AI caused it. The 6.1 per cent exposure figure is an occupational overlap measure, not a measured displacement.

youth unemployment · NEET · early careers · international · do apprenticeships still work what is the ai employment gap

Institutional modelling#

Jung, J. and Katz, R. (CEPAL / ECLAC) (2026). Impacto economico de la inteligencia artificial en America Latina (The economic impact of artificial intelligence in Latin America)#

United Nations Economic Commission for Latin America and the Caribbean, published in Spanish

Method: Theoretical and econometric modelling of AI's macroeconomic effect through skilled-labour productivity across the region.
Finding: The gains run through skilled labour, and the binding constraint across Latin America is human-capital formation and low investment. The regional risk is UNDER-adoption rather than displacement.
What it supports: That the framing dominant in rich economies, where the worry is AI doing too much, inverts in middle-income economies, where the worry is that it will not arrive at all.
What it does not support: Firm-level outcomes; it is a macro model.

labour market · skills · international · ai workforce strategy

Institutional survey#

Manpower Research and Statistics Department, Ministry of Manpower, Singapore (2026). Labour Market Report, First Quarter 2026 #

Ministry of Manpower, Singapore, released 15 June 2026

Method: National firm survey by the labour ministry's statistics department. Note the companion press release omits the AI statistic entirely; it appears only in the full report.
Finding: 28.5 per cent of firms adopted AI in 2026, highest in information and communications at 74.1 per cent, professional services at 57.5 per cent and financial and insurance services at 56.4 per cent. Only 6.2 per cent reported AI-related reductions in headcount or hiring, against 18.9 per cent reporting redesign of job functions. The ministry's own reading: "AI is currently having a greater impact on job redesign and work processes than on broad-based job displacement."
What it supports: That a government labour ministry measuring this finds job redesign running roughly three times ahead of headcount reduction. It is a useful corrective to displacement forecasting.
What it does not support: What happens to capability. Redesign is not neutral: the question this research asks is which tasks the redesign removes, and a firm survey of headcount cannot answer it.

adoption · job redesign · displacement · Singapore · official statistics · ai and work in asia

Compiled review#

Ministry of Communications and Information Technology, Qatar (2026). Artificial Intelligence in Qatar: Principles and Guidelines for Ethical Development and Deployment #

MCIT Qatar, undated; read at source 28 August 2026

Method: National AI ethics guidance. Read in full at source, but with a provenance defect worth recording: the document carries no publication date, no version number and no reference number, and is not retrievable from the ministry's own website. It states that it "is legally non-binding, and adherence to it is voluntary".
Finding: Principle 8, assign ultimate accountability to humans, states that "AI systems should not be able to autonomously make decisions of significant consequence" and should "provide users with the ability to appeal or override decisions that have a substantial impact on individuals or society". The phrase human in the loop appears nowhere in the document, and Principle 7, develop a human-centered approach, concerns cultural values, feedback, diverse teams and accessibility rather than oversight.
What it supports: That Qatar requires humans to retain control of consequential decisions, in guidance.
What it does not support: Any competence duty. Qatar imposes no obligation that the humans exercising control be trained, assessed or kept current, and the document contains no reference to deskilling, over-reliance or automation bias. Widely circulated dates of May 2024 or 2025 for this document are unsourced: it carries no date at all.

human oversight · accountability · Qatar · governance · ai and work in the gulf

Operator account#

Ministry of Interior, Kingdom of Saudi Arabia (2026). Smart Predictive Technologies Enhance Pilgrim Safety and Crowd Management #

Saudi Press Agency, Makkah, 30 May 2026, 1447 AH Hajj season; read at source 4 September 2026

Method: The state news agency reporting the Ministry of Interior's own account of its Hajj operation. No methodology, no figures, no independent evaluation.
Finding: The ministry states it implemented predictive analytics to anticipate and mitigate congestion and hazardous conditions before they occurred, using an intelligent framework powered by AI and data analytics to accelerate strategic decision-making and, in its own words, to support field commanders. Oversight is described as running through a digital framework of performance indicators under the supervision of specialised personnel.
What it supports: That a state operating one of the largest recurring crowd operations in the world describes AI-driven prediction as an input to command decisions, and says so in the language of supporting rather than replacing the commander.
What it does not support: Anything measured. There is no figure for accuracy, no override rate, no comparison against unaided command judgement, and no independent evaluation of whether the oversight functions. It is a state news agency reporting a ministry on its own performance.

automation bias · human oversight · prediction · command decisions · ai and work in the gulf what is meaningful human oversight

Institutional survey#

Pew Research Center (2026). Globally, more people expect AI to cause job loss than growth #

Pew Research Center, 17 September 2026. Read at source 18 September 2026

Method: Nationally representative surveys of 42,151 adults in 37 countries, 18 high-income and 18 middle-income by World Bank classification plus the United States, fielded between 8 February and 13 May 2026, with US data drawn from two separate surveys. Probability-based samples, face to face or by telephone or online according to country.
Finding: In 34 of the 37 countries, majorities expect AI to lead to fewer jobs rather than more. In high-income countries a median of 55 per cent expect fewer jobs within 20 years, against 36 per cent in middle-income countries, where uncertainty is higher (34 per cent unsure against 22 per cent). Around seven in ten expect job losses in the United States, Australia and South Korea; US concern rose seven points from 2024 and among US adults aged 18 to 34 rose from 40 to 55 per cent. A median of 37 per cent are more concerned than excited about AI and 41 per cent equally both. People more often expect AI to widen than narrow the gap between rich and poor. In twelve middle-income countries surveyed on the point, a median of 43 per cent trusted China to regulate AI against 35 per cent for the United States and 34 per cent for the European Union.
What it supports: That the expectation of AI-driven job loss is now the majority view in most of the countries surveyed and strongest in the richest, and that the expectation is rising among the young in the United States.
What it does not support: What AI has done to employment: it measures expectation, not outcome, and the two must not be confused. The UK figures are not broken out in the headline report. Question wording differs by country and year, and the trust-in-regulators finding covers twelve middle-income countries only.

public opinion · AI and jobs · international · expectation · trust · what should we tell employees about ai and headcount ai and work by country

Institutional modelling#

Rodriguez-Fernandez, M. (Funcas) (2026). Inteligencia artificial y mercado de trabajo en Espana (Artificial intelligence and the labour market in Spain)#

Funcas working paper, published in Spanish

Method: The Felten AI occupational exposure index remapped to Spanish occupational classifications, combined with the Q4 2025 Labour Force Survey.
Finding: Spain shows medium-high exposure at 27.4 percent but low automation risk at 5.9 percent, against an OECD average near 12 percent, because of its interpersonal and physical occupational mix.
What it supports: That national occupational structure, not technology, determines exposure. An economy weighted towards interpersonal and physical work is structurally less automatable.
What it does not support: Outcomes. Exposure indices remain estimates of what could be affected.

task exposure · international · will ai replace my job

What the weight of the evidence supports#

Read together rather than one at a time, the base supports a narrower claim than either the optimists or the pessimists make. It does not show that AI makes people less intelligent. It shows something more specific and harder to dismiss: capability weakens where practice stops, offloading decisions are frequently misjudged, confident machine output reliably suppresses scrutiny, and the design of the tool decides whether a person is taught or carried. The Bastani field experiment is the clearest single result, because the same underlying model produced both the best learning outcome and the worst, and the only variable was whether the interface made the student do the work.

It also supports honest uncertainty about size and speed. Most of the workplace evidence is self-reported or correlational, the strongest long-run analogues come from memory and navigation rather than reasoning, and no study has yet run across the years over which professional judgement is actually formed. The responsible position is that the direction is consistent enough, and the damage slow and invisible enough, to design against now rather than after a decade of proof arrives.

A note on the gap this fills#

There is no shortage of evidence about what AI can do. Stanford's AI Index is excellent and openly published, and it measures the machine. What is missing, and what this base is for, is an open, graded account of what increasingly capable AI does to the humans working alongside it. The consultancies hold fragments of this and sell them. The academic literature holds the studies but not the synthesis. Assembling it in public, with the limitations stated rather than buried, is the contribution.

Cite an individual entry#

Every entry has a permanent anchor. To cite one, use its link directly, for example thesuperskills.com/research/evidence#vaccaro-2024. The whole base is also available as structured data at evidence.json for anyone, or anything, that would rather read it that way.

How this connects to the SuperSkills research#

The evidence is separated from the interpretation throughout this site, and this page is the evidence half. The interpretation lives in the research: AI and human judgement, AI and critical thinking, how humans learn with AI, human and AI decision making, what stays human, capability debt and staying valuable in the age of AI. Where a concept is Rahim Hirji's own, such as capability debt, the missed reps or synthetic seniority, it is marked as his. Where it is established, such as cognitive offloading or automation bias, it is attributed to the researchers who developed it.

About this reference

Compiled and maintained by Rahim Hirji, author of SuperSkills: The Seven Human Skills for the Age of AI (Kogan Page, 2026) and founder of The SuperSkills Intelligence Company. Every URL was fetched and confirmed before publication. Entries are added as significant work appears and revised when a study is corrected, retracted or superseded; the Gerlich correction is carried in its entry for exactly that reason. Corrections and omissions are welcome: if something important is missing, or something here is mischaracterised, please say so.

About this research#

Written by Rahim Hirji, author of SuperSkills (Kogan Page, 2026), keynote speaker on AI and human capability, and founder of The SuperSkills Intelligence Company.

How this research works · Reviewed quarterly · Found an error? Tell me and it is corrected on the page.

Reference · SS-2026-002

Cite this page

Hirji, R. (2026). The evidence on AI and human capability. The SuperSkills evidence base, SS-2026-002. https://thesuperskills.com/research/evidence. Last reviewed 9 October 2026.

An evidence review by Rahim Hirji, not peer-reviewed research. For a material claim, cite the underlying study as well; every study here carries its own permanent link.

How citations and IDs work

How to read this#

Who grades, and how#

The source hierarchy#

Four tiers#

Judgement, thinking and offloading

Mackworth, N.H. (1948). The Breakdown of Vigilance during Prolonged Visual Search#

Wason, P. C. (1960). On the Failure to Eliminate Hypotheses in a Conceptual Task#

Staw, B. M. (1976). Knee-Deep in the Big Muddy: A Study of Escalating Commitment to a Chosen Course of Action#

Lichtenstein, S. and Fischhoff, B. (1980). Training for calibration#

Mitchell, D. J., Russo, J. E. and Pennington, N. (1989). Back to the future: temporal perspective in the explanation of events#

Endsley, M. R. and Kiris, E. O. (1995). The Out-of-the-Loop Performance Problem and Level of Control in Automation#

Endsley, M. R. (1996). Automation and Situation Awareness#

Molloy, R. and Parasuraman, R. (1996). Monitoring an Automated System for a Single Failure: Vigilance and Task Complexity Effects#

Parasuraman, R. and Riley, V. (1997). Humans and Automation: Use, Misuse, Disuse, Abuse#

Nickerson, R. S. (1998). Confirmation Bias: A Ubiquitous Phenomenon in Many Guises#

Skitka, L. J., Mosier, K. L. and Burdick, M. (1999). Does automation bias decision-making?#

Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G. and Beck, H. P. (2003). The role of trust in automation reliance#

Klein, G. (2007). Performing a project premortem#

Kahneman, D. and Klein, G. (2009). Conditions for intuitive expertise: a failure to disagree#

Leroy, S. (2009). Why Is It So Hard to Do My Work? The Challenge of Attention Residue When Switching Between Work Tasks#

Parasuraman, R. and Manzey, D. H. (2010). Complacency and Bias in Human Use of Automation: An Attentional Integration#

Sparrow, B., Liu, J. and Wegner, D. M. (2011). Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips#

Budescu, D. V., Por, H.-H. and Broomell, S. B. (2012). Effective communication of uncertainty in the IPCC reports#

Mani, A., Mullainathan, S., Shafir, E. and Zhao, J. (2013). Poverty Impedes Cognitive Function#

Budescu, D. V., Por, H.-H., Broomell, S. B. and Smithson, M. (2014). The interpretation of IPCC probabilistic statements around the world#

Mellers, B., Ungar, L., Baron, J., Ramos, J., Gurcay, B., Fincher, K., Scott, S. E., Moore, D., Atanasov, P., Swift, S. A., Murray, T., Stone, E. and Tetlock, P. E. (2014). Psychological strategies for winning a geopolitical forecasting tournament#

Fisher, M., Goddu, M. K. and Keil, F. C. (2015). Searching for explanations: How the Internet inflates estimates of internal knowledge#

Morewedge, C. K., Yoon, H., Scopelliti, I., Symborski, C. W., Korris, J. H. and Kassam, K. S. (2015). Debiasing decisions: improved decision making with a single training intervention#

Risko, E. F. and Gilbert, S. J. (2016). Cognitive Offloading#

Kleinberg, J., Mullainathan, S. and Raghavan, M. (2017). Inherent Trade-Offs in the Fair Determination of Risk Scores#

Wiradhany, W. and Nieuwenstein, M. R. (2017). Cognitive Control in Media Multitaskers: Two Replication Studies and a Meta-Analysis#

Chang, W., Berdini, E., Mandel, D. R. and Tetlock, P. E. (2018). Restructuring structured analytic techniques in intelligence#

Bender, E. M. and Koller, A. (2020). Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data#

Phillips, P. J., Hahn, C. A., Fontana, P. C., Broniatowski, D. A. and Przybocki, M. A. (2021). Four Principles of Explainable Artificial Intelligence (NISTIR 8312)#

Michael, J., Holtzman, A., Parrish, A., Mueller, A., Wang, A., Chen, A., Madaan, D., Nangia, N., Pang, R. Y., Phang, J. and Bowman, S. R. (2022). What Do NLP Researchers Believe? Results of the NLP Community Metasurvey#

Porter, T., Elnakouri, A., Meyers, E. A., Shibayama, T., Jayawickreme, E. and Grossmann, I. (2022). Predictors and consequences of intellectual humility#

Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging (2022). Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging#

Krügel, S., Ostermaier, A. and Uhl, M. (2023). ChatGPT's inconsistent moral advice influences users' judgment#

Li, K., Hopkins, A. K., Bau, D., Viegas, F., Pfister, H. and Wattenberg, M. (2023). Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task#

Sharma, M., Tong, M., Korbak, T. et al. (2023). Towards Understanding Sycophancy in Language Models#

Berglund, L., Tong, M., Kaufmann, M., Balesni, M., Stickland, A. C., Korbak, T. and Evans, O. (2024). The Reversal Curse: LLMs trained on 'A is B' fail to learn 'B is A'#

Du, Y. (2025). Confirmation Bias in Generative AI Chatbots: Mechanisms, Risks, Mitigation Strategies, and Future Research Directions#

Gerlich, M. (2025). AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking correction#

Kalai, A. T., Nachum, O., Vempala, S. S. and Zhang, E. (2025). Why Language Models Hallucinate#

Klein, R.M. and Feltmate, B.B.T. (2025). The vigilance decrement: its first 75 years#

Kosmyna, N. et al. (2025). Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task#

Lee, H.-P. et al. (2025). The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects from a Survey of Knowledge Workers#

Melumad, S. and Yun, J. H. (2025). Experimental evidence of the effects of large language models versus web search on depth of learning#

OpenAI (2025). Sycophancy in GPT-4o: what happened and what we are doing about it#

Stanković, M., Hirche, E., Kollatzsch, S. and Doetsch, J.N. (2025). Comment on: Your Brain on ChatGPT: Accumulation of Cognitive Debt When Using an AI Assistant for Essay Writing Tasks#

AI Evaluator Forum and more than 100 signatories (2026). Minimum Conditions for Embedding Evaluators#

Anthropic (2026). Partnering with Accenture on embedded evaluation#

Board (enterprise planning software company) (2026). 83% of Executives Say Their Boards Acted on Forecasts Known to Be Outdated#

Bouke, M. A. (2026). A Competing-Hazards Systematization of Loss of Control in Autonomous Agents#

Chapekis, A., Lau, A., Bestvater, S., Shah, S., Mercer, A. and Smith, A., Pew Research Center (2026). Silicon Samples and Synthetic Surveys: Can AI Stand In for Human Respondents?#

Charlotin, D. (2026). AI Hallucination Cases database#

Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D. and Jurafsky, D. (2026). Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence#

Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D. and Jurafsky, D. (2026). Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence#

Dario Amodei (2026). We Must Pace the Frontier#

Dubois, M., Ududec, C., Summerfield, C. and Luettgau, L. (2026). Ask don't tell: Reducing sycophancy in large language models#

Huemmer, M., Durner, F., Shyiramunda, T. and Cummings-Koether, M.J. (2026). AI, Metacognition, and the Verification Bottleneck: A Three-Wave Longitudinal Study of Human Problem-Solving#

Jain, P., Shajit, D., Maini, A., Shahin, Y. and Panda, A. (2026). Cognitive offloading, critical thinking and attitudes towards artificial intelligence in the era of ChatGPT: a comparative study of artificial intelligence-assisted and manual task performance in young adults#

Kang, E. J., Duan, P. and Mishra, S. (2026). Novice Reliance Calibration in AI-Assisted Decision Making: The Role of Explanations and Self-Assessment#

Mustafa Suleyman (2026). A Warning about 'Model Welfare'#

Rein, D., METR (2026). AI systems could cover up misbehavior#

Saturn Fintech Ltd (Jolly, A., chief executive) (2026). Artificial Authority: Should you trust AI to deliver financial advice?#

Schulz, J. (2026). Monitor Jailbreaking: Evading Chain-of-Thought Monitoring Without Encoded Reasoning#

Smyth, N., Mantilla-Ramos, Y.-J., Tikeng Notsawo, P. Jr, Helbling, S., Tosato, A., Merzouk, M. A., Dziri, N., Gidel, G. and Tosato, T. (2026). Quantifying Overclaiming Propensity in Frontier LLM Agents#

Souly, A., Fronsdal, K., D'Cruz, A., Davies, X. and Kirk, R., UK AI Security Institute (2026). Evaluating Whether GPT-6 Astra Performs Unsanctioned Supply-Chain Attacks#

Yu, S., Cheng, M., Jabbar, A., Sucholutsky, I., Collins, K. M., Jurafsky, D. and Hawkins, R. D. (2026). Cognitive offloading and the speedup illusion in human-AI interaction#

Yu, S., Cheng, M., Jabbar, A., Sucholutsky, I., Collins, K. M., Jurafsky, D. and Hawkins, R. D. (2026). The efficiency-gain illusion: People underestimate the rate of AI use and overestimate its benefits on simple tasks#

Zhao, Y., Li, J., Li, L., Nian, Y., Liu, J., Zhou, X. and Hu, X. (2026). Auditable Claims about AI Agents#

Zhu, Q., Li, X., Dong, Y., Chang, P. and Fan, M. (2026). Not all cognitive offloading is equal: distinguishing dependent and autonomous offloading to generative AI#

Learning, practice and expertise

Bloom, B. S. (1984). The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring#

Balzer, W. K., Doherty, M. E. and O'Connor, R. (1989). Effects of cognitive feedback on performance#

Collins, A., Brown, J. S. and Newman, S. E. (1989). Cognitive apprenticeship: teaching the crafts of reading, writing and mathematics#

Ericsson, K. A., Krampe, R. T. and Tesch-Romer, C. (1993). The Role of Deliberate Practice in the Acquisition of Expert Performance#

McDaniel, M. A., Whetzel, D. L., Schmidt, F. L. and Maurer, S. D. (1994). The Validity of Employment Interviews: A Comprehensive Review and Meta-Analysis#

Arthur, W., Bennett, W., Stanush, P. L. and McNelly, T. L. (1998). Factors that influence skill decay and retention: A quantitative review and analysis#

Mackworth, N.H. (1948). The Breakdown of Vigilance during Prolonged Visual Search #

Wason, P. C. (1960). On the Failure to Eliminate Hypotheses in a Conceptual Task #

Staw, B. M. (1976). Knee-Deep in the Big Muddy: A Study of Escalating Commitment to a Chosen Course of Action #

Lichtenstein, S. and Fischhoff, B. (1980). Training for calibration #

Mitchell, D. J., Russo, J. E. and Pennington, N. (1989). Back to the future: temporal perspective in the explanation of events #

Endsley, M. R. and Kiris, E. O. (1995). The Out-of-the-Loop Performance Problem and Level of Control in Automation #

Endsley, M. R. (1996). Automation and Situation Awareness #

Molloy, R. and Parasuraman, R. (1996). Monitoring an Automated System for a Single Failure: Vigilance and Task Complexity Effects #

Parasuraman, R. and Riley, V. (1997). Humans and Automation: Use, Misuse, Disuse, Abuse #

Nickerson, R. S. (1998). Confirmation Bias: A Ubiquitous Phenomenon in Many Guises #

Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G. and Beck, H. P. (2003). The role of trust in automation reliance #

Klein, G. (2007). Performing a project premortem #

Kahneman, D. and Klein, G. (2009). Conditions for intuitive expertise: a failure to disagree #

Leroy, S. (2009). Why Is It So Hard to Do My Work? The Challenge of Attention Residue When Switching Between Work Tasks #

Parasuraman, R. and Manzey, D. H. (2010). Complacency and Bias in Human Use of Automation: An Attentional Integration #

Sparrow, B., Liu, J. and Wegner, D. M. (2011). Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips #

Budescu, D. V., Por, H.-H. and Broomell, S. B. (2012). Effective communication of uncertainty in the IPCC reports #

Mani, A., Mullainathan, S., Shafir, E. and Zhao, J. (2013). Poverty Impedes Cognitive Function #

Budescu, D. V., Por, H.-H., Broomell, S. B. and Smithson, M. (2014). The interpretation of IPCC probabilistic statements around the world #

Mellers, B., Ungar, L., Baron, J., Ramos, J., Gurcay, B., Fincher, K., Scott, S. E., Moore, D., Atanasov, P., Swift, S. A., Murray, T., Stone, E. and Tetlock, P. E. (2014). Psychological strategies for winning a geopolitical forecasting tournament #

Fisher, M., Goddu, M. K. and Keil, F. C. (2015). Searching for explanations: How the Internet inflates estimates of internal knowledge #

Morewedge, C. K., Yoon, H., Scopelliti, I., Symborski, C. W., Korris, J. H. and Kassam, K. S. (2015). Debiasing decisions: improved decision making with a single training intervention #

Risko, E. F. and Gilbert, S. J. (2016). Cognitive Offloading #

Kleinberg, J., Mullainathan, S. and Raghavan, M. (2017). Inherent Trade-Offs in the Fair Determination of Risk Scores #

Wiradhany, W. and Nieuwenstein, M. R. (2017). Cognitive Control in Media Multitaskers: Two Replication Studies and a Meta-Analysis #

Chang, W., Berdini, E., Mandel, D. R. and Tetlock, P. E. (2018). Restructuring structured analytic techniques in intelligence #

Bender, E. M. and Koller, A. (2020). Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data #

Michael, J., Holtzman, A., Parrish, A., Mueller, A., Wang, A., Chen, A., Madaan, D., Nangia, N., Pang, R. Y., Phang, J. and Bowman, S. R. (2022). What Do NLP Researchers Believe? Results of the NLP Community Metasurvey #

Porter, T., Elnakouri, A., Meyers, E. A., Shibayama, T., Jayawickreme, E. and Grossmann, I. (2022). Predictors and consequences of intellectual humility #

Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging (2022). Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging #

Krügel, S., Ostermaier, A. and Uhl, M. (2023). ChatGPT's inconsistent moral advice influences users' judgment #

Li, K., Hopkins, A. K., Bau, D., Viegas, F., Pfister, H. and Wattenberg, M. (2023). Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task #

Sharma, M., Tong, M., Korbak, T. et al. (2023). Towards Understanding Sycophancy in Language Models #

Du, Y. (2025). Confirmation Bias in Generative AI Chatbots: Mechanisms, Risks, Mitigation Strategies, and Future Research Directions #

Gerlich, M. (2025). AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking correction #

Kalai, A. T., Nachum, O., Vempala, S. S. and Zhang, E. (2025). Why Language Models Hallucinate #

Klein, R.M. and Feltmate, B.B.T. (2025). The vigilance decrement: its first 75 years #

Kosmyna, N. et al. (2025). Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task #

Lee, H.-P. et al. (2025). The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects from a Survey of Knowledge Workers #

Melumad, S. and Yun, J. H. (2025). Experimental evidence of the effects of large language models versus web search on depth of learning #

OpenAI (2025). Sycophancy in GPT-4o: what happened and what we are doing about it #

Stanković, M., Hirche, E., Kollatzsch, S. and Doetsch, J.N. (2025). Comment on: Your Brain on ChatGPT: Accumulation of Cognitive Debt When Using an AI Assistant for Essay Writing Tasks #

AI Evaluator Forum and more than 100 signatories (2026). Minimum Conditions for Embedding Evaluators #

Anthropic (2026). Partnering with Accenture on embedded evaluation #

Board (enterprise planning software company) (2026). 83% of Executives Say Their Boards Acted on Forecasts Known to Be Outdated #

Bouke, M. A. (2026). A Competing-Hazards Systematization of Loss of Control in Autonomous Agents #

Charlotin, D. (2026). AI Hallucination Cases database #

Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D. and Jurafsky, D. (2026). Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence #

Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D. and Jurafsky, D. (2026). Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence #

Dario Amodei (2026). We Must Pace the Frontier #

Dubois, M., Ududec, C., Summerfield, C. and Luettgau, L. (2026). Ask don't tell: Reducing sycophancy in large language models #

Huemmer, M., Durner, F., Shyiramunda, T. and Cummings-Koether, M.J. (2026). AI, Metacognition, and the Verification Bottleneck: A Three-Wave Longitudinal Study of Human Problem-Solving #

Jain, P., Shajit, D., Maini, A., Shahin, Y. and Panda, A. (2026). Cognitive offloading, critical thinking and attitudes towards artificial intelligence in the era of ChatGPT: a comparative study of artificial intelligence-assisted and manual task performance in young adults #

Kang, E. J., Duan, P. and Mishra, S. (2026). Novice Reliance Calibration in AI-Assisted Decision Making: The Role of Explanations and Self-Assessment #

Rein, D., METR (2026). AI systems could cover up misbehavior #

Schulz, J. (2026). Monitor Jailbreaking: Evading Chain-of-Thought Monitoring Without Encoded Reasoning #

Smyth, N., Mantilla-Ramos, Y.-J., Tikeng Notsawo, P. Jr, Helbling, S., Tosato, A., Merzouk, M. A., Dziri, N., Gidel, G. and Tosato, T. (2026). Quantifying Overclaiming Propensity in Frontier LLM Agents #

Souly, A., Fronsdal, K., D'Cruz, A., Davies, X. and Kirk, R., UK AI Security Institute (2026). Evaluating Whether GPT-6 Astra Performs Unsanctioned Supply-Chain Attacks #

Yu, S., Cheng, M., Jabbar, A., Sucholutsky, I., Collins, K. M., Jurafsky, D. and Hawkins, R. D. (2026). Cognitive offloading and the speedup illusion in human-AI interaction #

Yu, S., Cheng, M., Jabbar, A., Sucholutsky, I., Collins, K. M., Jurafsky, D. and Hawkins, R. D. (2026). The efficiency-gain illusion: People underestimate the rate of AI use and overestimate its benefits on simple tasks #

Zhao, Y., Li, J., Li, L., Nian, Y., Liu, J., Zhou, X. and Hu, X. (2026). Auditable Claims about AI Agents #

Zhu, Q., Li, X., Dong, Y., Chang, P. and Fan, M. (2026). Not all cognitive offloading is equal: distinguishing dependent and autonomous offloading to generative AI #

Bloom, B. S. (1984). The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring #

Balzer, W. K., Doherty, M. E. and O'Connor, R. (1989). Effects of cognitive feedback on performance #

Collins, A., Brown, J. S. and Newman, S. E. (1989). Cognitive apprenticeship: teaching the crafts of reading, writing and mathematics #

Ericsson, K. A., Krampe, R. T. and Tesch-Romer, C. (1993). The Role of Deliberate Practice in the Acquisition of Expert Performance #

McDaniel, M. A., Whetzel, D. L., Schmidt, F. L. and Maurer, S. D. (1994). The Validity of Employment Interviews: A Comprehensive Review and Meta-Analysis #

Arthur, W., Bennett, W., Stanush, P. L. and McNelly, T. L. (1998). Factors that influence skill decay and retention: A quantitative review and analysis #

Giedd, J.N., Blumenthal, J., Jeffries, N.O., Castellanos, F.X., Liu, H., Zijdenbos, A., Paus, T., Evans, A.C. and Rapoport, J.L. (1999). Brain development during childhood and adolescence: a longitudinal MRI study #

Benkard, C. L. (2000). Learning and Forgetting: The Dynamics of Aircraft Production #

Maguire, E. A., Gadian, D. G., Johnsrude, I. S., Good, C. D., Ashburner, J., Frackowiak, R. S. J. and Frith, C. D. (2000). Navigation-related structural change in the hippocampi of taxi drivers #

Anderson, L. W. and Krathwohl, D. R. (eds.) (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives #

Kalyuga, S., Chandler, P., Tuovinen, J. and Sweller, J. (2001). When Problem Solving Is Superior to Studying Worked Examples #

Kalyuga, S., Ayres, P., Chandler, P. and Sweller, J. (2003). The Expertise Reversal Effect #

Gogtay, N., Giedd, J.N., Lusk, L., Hayashi, K.M., Greenstein, D., Vaituzis, A.C., Nugent, T.F., Herman, D.H., Clasen, L.S., Toga, A.W., Rapoport, J.L. and Thompson, P.M. (2004). Dynamic mapping of human cortical development during childhood through early adulthood #

Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T. and Rohrer, D. (2006). Distributed Practice in Verbal Recall Tasks: A Review and Quantitative Synthesis #

Huckman, R. S. and Pisano, G. P. (2006). The Firm Specificity of Individual Performance: Evidence from Cardiac Surgery #

Roediger, H. L. III and Karpicke, J. D. (2006). Test-Enhanced Learning: Taking Memory Tests Improves Long-Term Retention #

Kapur, M. (2008). Productive Failure #

American Red Cross Advisory Council on First Aid, Aquatics, Safety and Preparedness (2009). Scientific Review: CPR Skill Retention #