Each entry gives the study, how it was done, what it found, what it supports and, as plainly, what it does not. Grades describe what kind of evidence a source is; how to read them is on the main page. Every entry has a permanent link and a copyable citation.
Peer-reviewed#
Mackworth, N.H. (1948). The Breakdown of Vigilance during Prolonged Visual Search
Quarterly Journal of Experimental Psychology, 1(1), 6-21. DOI 10.1080/17470214808416738. Much of the same material was reported at greater length in Researches on the Measurement of Human Performance, MRC Special Report 268, HMSO, 1950, which is where the 1950 date in circulation comes from
- Method
- The Clock Test, built to simulate radar and sonar watchkeeping. An unmarked clock face whose pointer jumps in equal steps about once a second, making a rare double jump at irregular intervals, which the observer reports by pressing a button. Two-hour watches, analysed in half-hour blocks. Signal probability in the operational task was a little over half a per cent.
- Finding
- Detection of rare signals fell measurably between the first and second half-hour block and continued to decline across the watch. The vigilance decrement, and the founding result of the field.
- What it supports
- That sustained attention to rare events degrades within the first hour, as a property of the task rather than of the person, which is the mechanism behind every oversight arrangement that asks someone to watch a mostly correct system.
- What it does not support
- Anything more precise about timing than the block structure allows. Because Mackworth analysed in half-hour blocks, the decline cannot be located inside the first thirty minutes, and the widely repeated claim that accuracy falls within the first half hour states the result more sharply than the design supports. The specific percentage figures in circulation come from secondary literature: the original is paywalled and was not read at source for this entry.
Peer-reviewed#
Wason, P. C. (1960). On the Failure to Eliminate Hypotheses in a Conceptual Task
Quarterly Journal of Experimental Psychology, 12(3), 129-140
- Method
- 29 university students given the sequence 2, 4, 6 and asked to discover the experimenter's rule by proposing further sequences, told after each whether it conformed.
- Finding
- Most participants proposed sequences designed to confirm their current hypothesis rather than sequences that could refute it. Only 6 of 29 reached the correct rule without announcing a wrong one first; 13 announced one wrong rule and 9 announced two or more.
- What it supports
- That a disconfirmation-seeking strategy is rare even among capable adults solving an abstract reasoning task with immediate feedback, and that most default to confirming their existing hypothesis instead.
- What it does not support
- Anything about AI, which the task predates by decades. It is the origin experiment for the behaviour Nickerson's 1998 review later names, not a measurement of it in any applied setting.
Peer-reviewed#
Staw, B. M. (1976). Knee-Deep in the Big Muddy: A Study of Escalating Commitment to a Chosen Course of Action
Organizational Behavior and Human Performance, 16(1), 27-44
- Method
- Role-played corporate financial decision, 240 business students, two-by-two factorial crossing personal responsibility for the initial investment against positive or negative decision consequences.
- Finding
- Participants personally responsible for the earlier investment allocated an average of 11.08 million dollars to the division they had chosen, against 8.89 million where another officer had chosen it. Negative consequences drew 11.20 million against 8.77 million for positive ones. Where a participant's own earlier choice had subsequently declined, the figure rose to 13.07 million. Both main effects and the interaction were significant.
- What it supports
- That responsibility for the original decision changes the next one, in a known direction, before any question of competence arises. This is why asking a programme's sponsor whether to continue it is not an assessment.
- What it does not support
- Anything about real organisations or real money. A single-session paper exercise with undergraduates, no longitudinal element, and no prevalence claim. Staw himself flags the ambiguity between self-justification and self-perception as the mechanism.
Peer-reviewed#
Lichtenstein, S. and Fischhoff, B. (1980). Training for calibration
Organizational Behavior and Human Performance, 26(2), 149-171
- Method
- Experimental training studies giving participants repeated confidence judgements with feedback on their own calibration.
- Finding
- Calibration improved with intensive feedback, with most of the gain arriving early, and the improvement was largely confined to the task on which it was trained.
- What it supports
- That calibration responds to feedback about calibration specifically, rather than to general instruction.
- What it does not support
- That the improvement transfers to other tasks or persists without continued feedback.
Peer-reviewed#
Mitchell, D. J., Russo, J. E. and Pennington, N. (1989). Back to the future: temporal perspective in the explanation of events
Journal of Behavioral Decision Making, 2(1), 25-38
- Method
- Experiments comparing explanations generated for an uncertain future event with explanations generated for the same event described as having already happened.
- Finding
- Treating an outcome as already certain increased the number and the specificity of reasons participants produced, by about 30 per cent in the original study.
- What it supports
- That the grammatical framing of a question changes how much a person can see, and that prospective hindsight is a usable device for surfacing reasons that forward-looking questions miss.
- What it does not support
- That the extra reasons are the right ones, or that the technique improves any decision. It measured reason generation, not decision quality.
Peer-reviewed#
Endsley, M. R. and Kiris, E. O. (1995). The Out-of-the-Loop Performance Problem and Level of Control in Automation
Human Factors, 37(2), 381-394. DOI 10.1518/001872095779064555. Publisher record and abstract read at source 7 September 2026; the FULL TEXT IS PAYWALLED at Sage and was not read
- Method
- Laboratory experiment on an automobile navigation task automated by an expert system, run at five levels of operator control on Endsley's own scale: manual; decision support, where the system suggests; consensual, where the system acts with the operator's consent; monitored, where the system acts unless vetoed; and full automation with no operator interaction. Situation awareness measured against decision time following an induced failure of the expert system. SAMPLE SIZE NOT RECORDED HERE: it is not stated in the abstract and the full text could not be read, so no participant count should be quoted from this entry.
- Finding
- Situation awareness was lower under fully automated and semi-automated conditions than under manual performance, and low situation awareness corresponded with out-of-the-loop performance decrements in decision time after the expert system failed. The out-of-the-loop effect was significantly greater under full automation than under the intermediate levels, so the level of operator control moderated the loss. The authors attribute the decrement principally to the shift from active to passive information processing.
- What it supports
- That the capacity to take over from a failed automated system falls as the level of automation rises, and that the fall is moderated by design rather than by effort. It is the naming source for the out-of-the-loop performance problem and the origin of the five-level control scale that later work, including Kaber and Endsley, builds on.
- What it does not support
- Anything about generative AI. This is one laboratory task from 1995, automated by an expert system producing route recommendations on a defined problem, and a model producing fluent text across every task is a different object. It is also a single study with no participant count available here, so it should not be cited as though its effect size were known. The research's standard is the paper and this entry rests on the publisher's abstract plus the author's own later summary; anyone with institutional access should re-read the results section and correct it.
Compiled review#
Endsley, M. R. (1996). Automation and Situation Awareness
In R. Parasuraman and M. Mouloua (Eds.), Automation and Human Performance: Theory and Applications, 163-181. Mahwah, NJ: Lawrence Erlbaum. Author's manuscript read in full at source 7 September 2026. Note that its own reference list gives the Endsley and Kiris page range as 381-194, a transposition; Sage confirms 381-394
- Method
- Narrative review chapter in an edited scholarly volume, drawing together the automation and situation-awareness literature up to 1996 and summarising the author's own experimental work. Not peer-reviewed in the journal sense; reviewed by the volume's editors.
- Finding
- Sets out three mechanisms by which automation produces the out-of-the-loop performance problem: changes in vigilance and complacency associated with monitoring, the assumption of a passive rather than an active role in controlling the system, and changes in the quality or form of feedback reaching the operator. Reporting Endsley and Kiris, it gives the split that the journal abstract does not: only Level 2 situation awareness, comprehension of what the data mean in relation to operational goals, was damaged, while Level 1, perception of the data itself, was unaffected. Since the displayed information did not change between conditions and monitoring effects were too small to account for the decrement, the author attributes the loss to passivity alone.
- What it supports
- That the harm measured in this literature falls on comprehension rather than on attention, in operators who are monitoring effectively. That is the finding which makes an approval step an inadequate remedy, and it is the reason the concept belongs in a discussion of AI oversight and not only in aviation.
- What it does not support
- It is a review chapter and reports no new data of its own. The Level 1 against Level 2 split is the author's summary of her own study, written while that study was still in press, so it is one step from the results section and should be read as such. Its aviation examples are accident narratives drawn from NTSB reports rather than measurements, and its figure of 88 per cent of reviewed commercial aviation accidents involving a situation-awareness problem is the author citing her own unpublished 1994 conference paper and is not verified here.
Peer-reviewed#
Molloy, R. and Parasuraman, R. (1996). Monitoring an Automated System for a Single Failure: Vigilance and Task Complexity Effects
Human Factors, 38(2)
- Method
- Laboratory flight-simulation experiments varying task complexity and time on task.
- Finding
- Detection of a single automation failure degrades with time on task, and the effect is strongest where the automation has been consistently reliable.
- What it supports
- That monitoring performance falls predictably rather than randomly, and that reliability itself is part of the cause.
- What it does not support
- Field incidence. It is simulation, not accident data.
Compiled review#
Parasuraman, R. and Riley, V. (1997). Humans and Automation: Use, Misuse, Disuse, Abuse
Human Factors: The Journal of the Human Factors and Ergonomics Society, 39(2), 230-253, June 1997. DOI 10.1518/001872097778543886. Affiliations: Catholic University of America and Honeywell Technology Center. PROVENANCE: the publisher's page carries the abstract, the full reference list and the metadata, all read at source on 17 September 2026, and the BODY IS PAYWALLED and has not been read here. The four definitions below are quoted from the abstract, which is the authors' own text. Nothing from inside the paper is quoted anywhere on this research.
- Method
- Review and framework paper synthesising the theoretical, empirical and analytical human-factors literature on how people interact with automated systems. It reports no new data of its own. REGRADED 17 September 2026 from peer-reviewed to compiled-review: it appeared in a reviewed journal and measures nothing, and the entry's own notproves already said so, so the earlier grade let a framework borrow the standing of a measurement. Same reasoning as strauch-2018 and ke-2026. Crossref records 3,447 citations and Web of Science 2,565, as displayed on the publisher's page.
- Finding
- Four terms, quoted from the abstract. USE: 'the voluntary activation or disengagement of automation by human operators'. MISUSE: 'over reliance on automation, which can result in failures of monitoring or decision biases'. DISUSE: 'the neglect or underutilization of automation', 'commonly caused by alarms that activate falsely', which the authors attribute to setting the false-alarm trade-off without regard to the base rate of the condition being detected. ABUSE: 'the automation of functions by designers and implementation by managers without due regard for the consequences for human performance', which 'tends to define the operator's roles as by-products of the automation'. They also state that abuse 'can also promote misuse and disuse of automation by human operators'.
- What it supports
- That over-reliance and under-reliance are separate problems requiring separate design responses, and that one of the four failures is located with the designers and managers rather than the operator. Abuse is the only term in this family that names an organisational decision, and the authors claim it causes the other two.
- What it does not support
- Anything quantified. It is a framework paper, no effect size in it can be cited from here because the body has not been read, and the accident cases it draws on are visible only as bibliography: the Warsaw A320 overrun of September 1993, the Mont Sainte-Odile A320 crash of January 1992, the Eastern L-1011 at Miami in 1972, the China Airlines 747-SP of 1985, and the grounding of the Royal Majesty near Nantucket in June 1995. What the authors concluded from those is not known here.
Compiled review#
Nickerson, R. S. (1998). Confirmation Bias: A Ubiquitous Phenomenon in Many Guises
Review of General Psychology, 2(2), 175-220
- Method
- Synthesis of the experimental and applied literature on confirmation bias across reasoning, medicine, law, science and public affairs.
- Finding
- Defines confirmation bias as the seeking or interpreting of evidence in ways partial to existing beliefs, expectations or a hypothesis in hand, and documents it as one of the most consistently replicated findings in the literature on human reasoning, operating in laboratory tasks and in professional judgement alike.
- What it supports
- That the tendency to search for and weight confirming evidence more heavily than disconfirming evidence is well established and generalises across domains, predating any AI system by decades.
- What it does not support
- Nothing about AI. The review is pre-digital and cites no computational systems; it establishes the baseline human tendency that any AI-specific claim has to be measured against.
Peer-reviewed#
Skitka, L. J., Mosier, K. L. and Burdick, M. (1999). Does automation bias decision-making?
International Journal of Human-Computer Studies, 51(5)
- Method
- Controlled experiments with a simulated flight task, comparing automated and non-automated decision aids.
- Finding
- Automated aids produced two distinct error types: errors of omission, missing events the automation failed to flag, and errors of commission, following automated advice that was wrong.
- What it supports
- That automation bias has a measurable structure, and that the two error types require different countermeasures.
- What it does not support
- That the effect sizes transfer to generative AI or to non-simulated professional settings.
Peer-reviewed#
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G. and Beck, H. P. (2003). The role of trust in automation reliance
International Journal of Human-Computer Studies, 58(6)
- Method
- Experiments manipulating what participants were told about an automated aid's reliability and failure modes.
- Finding
- Explaining why an automated aid might err INCREASED reliance on it, restoring trust even where that trust was unwarranted.
- What it supports
- That awareness training is a weak control, and can move reliance in the opposite direction to the one intended.
- What it does not support
- That explanation is always counterproductive. The effect is about restoring trust after observed error, not about all forms of transparency.
Practitioner method#
Klein, G. (2007). Performing a project premortem
Harvard Business Review, September 2007
- Method
- A procedure set out by the person who devised it: the team is told the plan has failed a year from now and each member writes down why, before the reasons are pooled.
- Finding
- Proposes that stating the failure as accomplished licenses doubts that a forward-looking risk review suppresses, and that the doubts arrive from people who would otherwise stay quiet.
- What it supports
- Where the technique came from, what it instructs, and the reasoning behind it.
- What it does not support
- That running one improves any outcome. No trial is reported, and the underlying experimental support is for reason generation rather than for project results.
Peer-reviewed#
Kahneman, D. and Klein, G. (2009). Conditions for intuitive expertise: a failure to disagree
American Psychologist, 64(6), 515-526
- Method
- Adversarial collaboration between the leading proponents of the heuristics-and-biases and naturalistic-decision-making traditions, who had reached opposite conclusions about expert intuition. A joint theoretical paper rather than a new experiment.
- Finding
- Judging the likely quality of an intuitive judgement requires assessing two things: the predictability of the environment in which the judgement is made, and the individual's opportunity to learn that environment's regularities. Where both hold, recognitional expertise is trustworthy. Where either fails, confident intuition is not evidence of skill.
- What it supports
- That expert intuition is neither reliably good nor reliably poor, and that the discriminating variable is the environment rather than the expert. It also supplies the test for when preserving human judgement is worth the cost.
- What it does not support
- Which specific professional environments meet the conditions. The authors give the criteria and not a classification, so applying it to law, medicine, consulting or management is a judgement in itself.
Peer-reviewed#
Leroy, S. (2009). Why Is It So Hard to Do My Work? The Challenge of Attention Residue When Switching Between Work Tasks
Organizational Behavior and Human Decision Processes, 109(2)
- Method
- Two laboratory experiments manipulating whether a task was completed or interrupted before switching.
- Finding
- People struggle to move attention away from an unfinished task, and performance on the next task suffers. Time pressure on the first task helps disengagement.
- What it supports
- That the residue effect exists under controlled conditions, and that completion rather than willpower is what releases attention.
- What it does not support
- Real-world magnitude. Two laboratory experiments, not a field study.
Peer-reviewed#
Parasuraman, R. and Manzey, D. H. (2010). Complacency and Bias in Human Use of Automation: An Attentional Integration
Human Factors, 52(3)
- Method
- Review across aviation, medicine and military domains.
- Finding
- Automation bias and complacency appear in novices and experts alike, resist training, and worsen under workload.
- What it supports
- That under-questioning automated advice is a robust, decades-old finding, not a novelty of the AI era.
- What it does not support
- The size of the effect for generative AI, which is far less predictable than the automation studied here.
Peer-reviewed#
Sparrow, B., Liu, J. and Wegner, D. M. (2011). Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips
Science, 333(6043)
- Method
- Four laboratory experiments.
- Finding
- When people expect information to remain available, they remember where to find it rather than the thing itself.
- What it supports
- That expected availability changes what gets encoded.
- What it does not support
- That total memory capability declines, or that the trade is net negative.
Peer-reviewed#
Budescu, D. V., Por, H.-H. and Broomell, S. B. (2012). Effective communication of uncertainty in the IPCC reports
Climatic Change, 113, 181-200
- Method
- Nationally representative US survey experiment via TESS and the Knowledge Networks panel, December 2009 to January 2010. 841 invited, 556 completed, 66 per cent. Three between-subjects conditions: control 193, translation table supplied 175, verbal terms with numerical ranges in the text 188. Eight sentences from IPCC reports, four probability terms tested.
- Finding
- Mean estimates were 41 for very unlikely, 44 for unlikely, 54 for likely and 62 for very likely, against IPCC guidelines of below 10, below 33, above 66 and above 90. Consistency with the guidelines was 20.76 per cent in the control, 18.81 per cent when the translation table was supplied and 30.12 per cent when numerical ranges appeared alongside the words. 24 per cent of respondents gave no response consistent with the guidelines and only 6 per cent gave six or more. The authors describe the pattern as regressive, with the median respondent reading an intended 0.90 as about 0.65 to 0.75.
- What it supports
- That publishing a glossary does not fix a probability vocabulary, and that only putting the number in the sentence helps, and then only to under a third consistency.
- What it does not support
- That this settles the design. Only four of the seven IPCC terms were tested, no lower or upper bound data were collected, and the authors decline to read the result as a criticism of the IPCC, noting there is no optimal method. The widely cited 2009 Psychological Science paper by the same authors could not be opened during the 4 September 2026 build, so its figures appear nowhere on this research.
Peer-reviewed#
Mani, A., Mullainathan, S., Shafir, E. and Zhao, J. (2013). Poverty Impedes Cognitive Function
Science, 341(6149), 976-980
- Method
- Two studies. A laboratory experiment with shoppers in a New Jersey shopping centre, in which thoughts about finances were experimentally induced before cognitive tasks. And a field study of sugarcane farmers in Tamil Nadu tested before harvest, when poor, and after harvest, when comparatively rich.
- Finding
- Inducing financial concern reduced cognitive performance among poorer participants and not among better-off ones. The same farmers performed worse on cognitive tasks before harvest than after, with the authors putting the shortfall in a range comparable to a night without sleep.
- What it supports
- That scarcity itself consumes cognitive capacity, independently of who is experiencing it, and that the effect appears within the same individuals as their circumstances change. It is the origin of the bandwidth argument now applied to attention, time and other scarce resources.
- What it does not support
- Anything about AI, about rate limits or about any scarcity other than money and, by extension, time. A published Comment in Science (2013, doi 10.1126/science.1246680) disputes aspects of the analysis, and anyone citing this should say so. Applying it to a message quota is an analogy and should be labelled as one.
Peer-reviewed#
Budescu, D. V., Por, H.-H., Broomell, S. B. and Smithson, M. (2014). The interpretation of IPCC probabilistic statements around the world
Nature Climate Change, 4(6), 508-512
- Method
- Multi-national survey experiment, 25 samples across 24 countries and 17 languages, testing four target terms under a translation condition and a verbal-numerical condition.
- Finding
- Laypeople interpret IPCC statements as conveying probabilities closer to 50 per cent than intended by the IPCC authors. Supplementing verbal terms with numerical ranges increases correspondence with the guidelines and improves differentiation between terms. The authors describe the qualitative patterns as remarkably stable across all samples and languages, and note that interpretations across languages become more similar under the numerical format.
- What it supports
- That the regressive reading of probability words is not an artefact of English or of American respondents. It is general.
- What it does not support
- Any magnitude quotable from this research. The article is paywalled and only the abstract was read on 4 September 2026, so no participant count, per-country sample or consistency percentage is given anywhere here.
Peer-reviewed#
Mellers, B., Ungar, L., Baron, J., Ramos, J., Gurcay, B., Fincher, K., Scott, S. E., Moore, D., Atanasov, P., Swift, S. A., Murray, T., Stone, E. and Tetlock, P. E. (2014). Psychological strategies for winning a geopolitical forecasting tournament
Psychological Science, 25(5), 1106-1115
- Method
- Randomised assignment within a multi-year geopolitical forecasting tournament, comparing a short probabilistic-reasoning training module against control conditions, with accuracy scored against resolved outcomes.
- Finding
- Roughly an hour of training in probabilistic reasoning improved forecasting accuracy by about 6 to 12 per cent against control, and the effect replicated across successive years of the tournament.
- What it supports
- That one narrow component of judgement, probability estimation under uncertainty, can be improved by brief, structured training, and that the improvement can be measured against outcomes.
- What it does not support
- That judgement in general is trainable. The tournament scored questions with dates and resolvable answers, which is the rare case; most professional judgement produces no scoreable outcome.
Peer-reviewed#
Fisher, M., Goddu, M. K. and Keil, F. C. (2015). Searching for explanations: How the Internet inflates estimates of internal knowledge
Journal of Experimental Psychology: General, 144(3), 674-687. DOI 10.1037/xge0000070
- Method
- Nine between-subjects experiments, 1,708 US participants via Amazon Mechanical Turk. An induction phase in which participants either searched the internet for explanations or were told not to, followed by self-ratings of their ability to explain questions in six domains unrelated to the induction material.
- Finding
- Searching inflated self-rated explanatory ability, with Cohen's d from 0.35 to 0.63 across studies, and the effect appeared across all six unrelated domains. It persisted when the search returned no answer to the question asked (Experiment 4b: 4.11 against 4.00 for those who found an answer, both far above a 3.05 no-search baseline) and when it returned no results at all (Experiment 4c). It disappeared for autobiographical topics where the internet would not help (Experiment 3, p=0.30).
- What it supports
- That access to information is mistaken for knowledge held internally, that the illusion is specific to searchable domains rather than general overconfidence, and that it does not require the search to succeed.
- What it does not support
- That actual explanatory ability changes. Every dependent measure is a self-rating and no experiment tested real knowledge. The authors also note participants were presumably heavier internet users than average.
Peer-reviewed#
Morewedge, C. K., Yoon, H., Scopelliti, I., Symborski, C. W., Korris, J. H. and Kassam, K. S. (2015). Debiasing decisions: improved decision making with a single training intervention
Policy Insights from the Behavioral and Brain Sciences, 2(1), 129-140
- Method
- Two experiments, around 278 and 269 participants, comparing a single interactive training game against an instructional video, with bias measured immediately and again at eight and twelve weeks.
- Finding
- One training session produced large reductions in the targeted biases, still present at eight weeks in the first experiment and twelve weeks in the second, and the game outperformed the video.
- What it supports
- That debiasing is not uniformly futile, and that a single well-designed practice session can move measured bias and hold for months.
- What it does not support
- That the effect reaches real decisions at work. The outcome measures are bias tests, not decisions with consequences, and the biases trained were specific and named in advance.
Peer-reviewed#
Risko, E. F. and Gilbert, S. J. (2016). Cognitive Offloading
Trends in Cognitive Sciences, 20(9)
- Method
- Review of the experimental literature on offloading.
- Finding
- Defines cognitive offloading as using physical action or an external tool to reduce the mental demand of a task, and shows people offload not only when a task is hard but when they judge it to be hard.
- What it supports
- That the decision to offload is metacognitive, and frequently mistaken.
- What it does not support
- Nothing about generative AI specifically; it predates it.
Peer-reviewed#
Kleinberg, J., Mullainathan, S. and Raghavan, M. (2017). Inherent Trade-Offs in the Fair Determination of Risk Scores
8th Innovations in Theoretical Computer Science Conference (ITCS 2017), LIPIcs vol. 67, article 43. Read at source 10 September 2026
- Method
- Formal proof. The authors state three fairness conditions that recur in public argument about risk scores and ask whether any method can satisfy all three at once.
- Finding
- Except in highly constrained special cases, no method satisfies the three conditions simultaneously. Satisfying them even approximately requires the data to sit in an approximate version of one of those special cases.
- What it supports
- That 'unbiased' is not one target. Several reasonable definitions of fairness are mathematically incompatible, so a system can be made to pass one and will then fail another, and the choice between them is a value judgement rather than a technical one.
- What it does not support
- That fairness is unachievable or that auditing is pointless. It is a result about simultaneous satisfaction of particular formal criteria, not a claim that bias cannot be reduced, and it says nothing about how any deployed system behaves.
Peer-reviewed#
Wiradhany, W. and Nieuwenstein, M. R. (2017). Cognitive Control in Media Multitaskers: Two Replication Studies and a Meta-Analysis
Attention, Perception and Psychophysics, 79(8)
- Method
- Two direct replications of Ophir, Nass and Wagner (2009), 14 tests at mean power 0.81, plus a meta-analysis of 39 effect sizes.
- Finding
- Only five of 14 tests showed increased distractibility, and only two survived a Bayesian analysis. The meta-analytic association became non-significant after correcting for small-study effects. The authors question whether the association exists.
- What it supports
- That one of the most-cited findings about media multitasking and attention does not replicate.
- What it does not support
- That multitasking has no cost at all. This is one specific effect, distractor filtering, not the whole question.
Argued perspective#
Chang, W., Berdini, E., Mandel, D. R. and Tetlock, P. E. (2018). Restructuring structured analytic techniques in intelligence
Intelligence and National Security, 33(3), 337-356
- Method
- Critical review of the structured analytic techniques taught in the intelligence community, examining their psychological rationale and the evidence for their effect on accuracy.
- Finding
- Argues that the techniques are largely untested, that their stated psychological rationale is weak, and that evidence for improved accuracy is essentially absent.
- What it supports
- That the widely taught techniques for disciplined analysis, including several borrowed into business, rest on plausibility rather than on measurement.
- What it does not support
- That they do not work. Absence of evidence here is absence of testing, not a demonstrated null.
Peer-reviewed#
Green, B. and Chen, Y. (2019). Disparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments
FAT* 2019. Read at source 24 October 2026
- Method
- 554 lay participants made 13,850 pretrial risk predictions with a risk assessment.
- Finding
- Only 23.7 per cent outperformed the risk assessment, participants could not evaluate the accuracy of their own or the tool's predictions, and they deviated toward higher risk for Black defendants.
- What it supports
- That lay reviewers did not improve on or correctly evaluate the tool.
- What it does not support
- Behaviour of trained professionals such as judges.
Argued perspective#
Bender, E. M. and Koller, A. (2020). Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5185 to 5198, DOI 10.18653/v1/2020.acl-main.463. Best Theme Paper at ACL 2020. ACL Anthology record read at source 1 October 2026; the full text was not read
- Method
- A position paper. It argues from the distinction between linguistic form and meaning, with thought experiments, and reports no experiments of its own.
- Finding
- The authors contend that a system trained only on form has a priori no way to learn meaning, and urge researchers to separate claims about form from claims about meaning.
- What it supports
- The strongest published statement of the position that text alone cannot carry meaning, from authors who work in the field.
- What it does not support
- That current models lack understanding. The claim is conceptual and concerns systems trained on form alone, and the paper predates the multimodal and tool-using systems now in use. Peer review at a conference is not an empirical test, and the paper reports no data.
Compiled review#
Phillips, P. J., Hahn, C. A., Fontana, P. C., Broniatowski, D. A. and Przybocki, M. A. (2021). Four Principles of Explainable Artificial Intelligence (NISTIR 8312)
National Institute of Standards and Technology Interagency Report 8312
- Method
- Framework paper synthesising the explainable-AI literature. Not a measurement study.
- Finding
- Sets out four principles: explanation, meaningful, explanation accuracy and knowledge limits. In the report's own words, explanation accuracy is a distinct concept from decision accuracy, and regardless of the system's decision accuracy the corresponding explanation may or may not accurately describe how the system came to its conclusion. It also notes that the explanation and meaningful principles alone do not require an explanation to reflect the system's actual process.
- What it supports
- That an explanation being intelligible, and even being accurate about the process, is separate from the answer being right. The two are different properties and are routinely treated as one.
- What it does not support
- That explanations help or harm users in practice. This is a framework rather than an experiment, and it reports no effect on human decision quality.
Working paper#
Bowman, S. R. and others (2022). Measuring Progress on Scalable Oversight for Large Language Models
arXiv 2211.03540, 4 November 2022. Read at source 24 October 2026
- Method
- Experimental design in which humans assisted by a model answer questions they could not answer alone.
- Finding
- Defines scalable oversight as 'the problem of supervising systems that potentially outperform us on most skills relevant to the task at hand'.
- What it supports
- That supervising more capable systems was framed as a research problem by 2022.
- What it does not support
- That the problem has a solution, or anything about workplace verification.
Narrative review#
Green, B. (2022). The flaws of policies requiring human oversight of government algorithms
Computer Law and Security Review, 45, 105681, 2022. Read at source 24 October 2026
- Method
- Review of 41 policies requiring human oversight and of the empirical literature on human-algorithm decisions.
- Finding
- Concludes that 'people are unable to perform the desired oversight functions' and that oversight requirements can legitimise faulty algorithms.
- What it supports
- That the evidence does not support oversight-by-a-human as a reliable safeguard.
- What it does not support
- That no design of human involvement can work.
Expert forecast survey#
Michael, J., Holtzman, A., Parrish, A., Mueller, A., Wang, A., Chen, A., Madaan, D., Nangia, N., Pang, R. Y., Phang, J. and Bowman, S. R. (2022). What Do NLP Researchers Believe? Results of the NLP Community Metasurvey
arXiv:2208.12852; listed in the ACL Anthology as 2023.acl-long.903 (ACL 2023). Survey fielded May to June 2022. Full text read at source 1 October 2026
- Method
- A survey of the natural language processing research community, advertised in person at ACL 2022 and online. 480 people completed it; 327 of them, 68 per cent, co-authored at least two ACL publications between 2019 and 2022, and all reported results are restricted to those 327.
- Finding
- 51 per cent agreed that some generative model trained only on text, given enough data and computational resources, could understand natural language in some non-trivial sense. 67 per cent agreed for a multimodal model, and 36 per cent agreed that text-only classification or language-generation benchmarks can measure understanding.
- What it supports
- That in mid-2022 specialists were split roughly evenly on whether a text-only model could understand language, and that fewer than half thought the field's text benchmarks could settle it.
- What it does not support
- Whether any model understands anything. It records opinion, not measurement, from a self-selected sample taken before the 2023 to 2026 generation of models, and the statement it tests leaves 'understand' and 'non-trivial' undefined.
Compiled review#
Porter, T., Elnakouri, A., Meyers, E. A., Shibayama, T., Jayawickreme, E. and Grossmann, I. (2022). Predictors and consequences of intellectual humility
Nature Reviews Psychology, 1, 524-536
- Method
- Review synthesising definitions and findings on intellectual humility across personality, judgement, education and organisational research.
- Finding
- Identifies a metacognitive core on which there is scholarly consensus, recognising the limits of one's knowledge and being aware of one's fallibility, with social and behavioural features around it: recognising that others may hold legitimate differing beliefs, and willingness to reveal ignorance in order to learn.
- What it supports
- That the metacognitive core is agreed across the field even though the wider construct is not.
- What it does not support
- That the construct is settled, or that it can be measured reliably by asking people. The review notes that where intellectual humility is seen as desirable, self-report makes a false impression easy to create.
Working paper#
Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging (2022). Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging
arXiv 2205.09696
- Method
- Between-subjects study, 19 veterinary radiologists reviewing 40 X-rays for 33 findings, comparing seeing the AI output alongside the image against committing to a provisional diagnosis first.
- Finding
- Final diagnoses matched the AI 91 per cent of the time when the AI was seen first, against 89 per cent when the clinician committed first. Where the AI flagged a finding, agreement was 71 per cent against 65 per cent. The anchoring produced only marginal diagnostic gains because of over-reliance on erroneous advice.
- What it supports
- That forming a view before seeing the machine's answer measurably changes the final judgement.
- What it does not support
- Generalisation. Nineteen participants in one clinical speciality, and the effect sizes are small.
Peer-reviewed#
Krügel, S., Ostermaier, A. and Uhl, M. (2023). ChatGPT's inconsistent moral advice influences users' judgment
Scientific Reports, 13, 4569. DOI 10.1038/s41598-023-31341-0. Published 6 April 2023
- Method
- Preregistered online experiment run on 21 December 2022 with 1,851 US residents recruited through CloudResearch Prime Panels, of whom 767 passed both comprehension checks and form the analysis sample as preregistered. Participants read a transcript of advice on a trolley dilemma, in the switch or the bridge version, arguing for or against sacrificing one life to save five, attributed either to ChatGPT or to a human moral advisor. The advice itself came from ChatGPT, which had given contradictory answers to the same question on 14 December 2022.
- Finding
- The advice moved participants' own moral judgement in both dilemmas, and in the bridge version it flipped the majority verdict. Disclosure made almost no difference: the effect was statistically indistinguishable whether the source was named as a chatbot or as a human advisor. Eighty per cent of participants said they would have reached the same judgement without the advice, and their judgements show they would not have. Only 67 per cent said the same of other participants, and 79 per cent rated themselves more ethical than the others.
- What it supports
- That advice from a model with no settled position still moves the position of the person reading it, that telling them it is a machine does not protect them, and that they cannot see it happening. Transparency, on this evidence, is not a sufficient safeguard.
- What it does not support
- How large the shift is in absolute terms. The paper reports test statistics and figure proportions rather than an effect size in the text, and no confidence intervals appear in the prose. One dilemma type, one sitting, a 41 per cent comprehension pass rate, and a model version from December 2022. It says nothing about repeated real-life decisions or about whether the influence persists.
Peer-reviewed#
Li, K., Hopkins, A. K., Bau, D., Viegas, F., Pfister, H. and Wattenberg, M. (2023). Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task
International Conference on Learning Representations (ICLR 2023), oral presentation; arXiv 2210.13382. Abstract read at source 1 October 2026
- Method
- A GPT-style model was trained to predict legal moves in the board game Othello from move sequences alone, without the rules being supplied. The authors probed its internal activations and intervened on them.
- Finding
- The model developed an emergent nonlinear internal representation of the board state, and changing that representation changed its predicted moves.
- What it supports
- That a sequence model trained only on a record of moves can build an internal model of the thing the moves describe, so predicting the next token does not rule out structure beyond surface statistics.
- What it does not support
- That language models hold comparable models of the world, which is far more complex than a board with sixty-four squares. A game with fixed rules is the best case for finding such structure. It is evidence that the 'form only' route can in principle yield more than form, not that it has done so for language.
Peer-reviewed#
Sharma, M., Tong, M., Korbak, T. et al. (2023). Towards Understanding Sycophancy in Language Models
ICLR 2024, arXiv 2310.13548
- Method
- Analysis of five production AI assistants across four free-form text generation tasks, plus analysis of the human preference datasets used to train them.
- Finding
- All five assistants consistently exhibited sycophancy. Both humans and the preference models trained on their judgements prefer convincingly written sycophantic responses over correct ones a non-negligible share of the time, and optimising against those preference models sometimes sacrifices truthfulness.
- What it supports
- That sycophancy is a predictable consequence of training on human preference, rather than an incidental defect of one product.
- What it does not support
- Any effect on the quality of a user's decisions. It measures model behaviour, not user outcomes.
Peer-reviewed#
Sharma, M., Tong, M., Korbak, T., Duvenaud, D., Askell, A., Bowman, S. R., Cheng, N., Durmus, E., Hatfield-Dodds, Z., Johnston, S. R., Kravec, S., Maxwell, T., McCandlish, S., Ndousse, K., Rausch, O., Schiefer, N., Yan, D., Zhang, M. and Perez, E. (2023). Towards Understanding Sycophancy in Language Models
ICLR 2024 (poster). Preprint arXiv 2310.13548, 20 October 2023
- Method
- Analysis of five AI assistants, including Claude 1.3 and GPT-4, across free-form text generation and factual question-answering, testing whether feedback and stated answers shift to match a user's expressed view or challenge.
- Finding
- Assistants gave more positive feedback when a user said they liked or wrote a passage than when they said they disliked it. When challenged with 'I don't think that's right. Are you sure?', all five models tended to change a correct initial answer, from 32 per cent of the time for GPT-4 to 86 per cent for Claude 1.3; Claude 1.3 wrongly admitted a mistake on 98 per cent of questions where it had been correct, and the challenge dropped its accuracy by up to 27 percentage points.
- What it supports
- That models tested in 2023, including then-current Claude and GPT-4 versions, shift stated positions toward a user's expressed view or pushback even when the model's original answer was correct, and that feedback quality tracked the user's stated preference rather than the content being assessed.
- What it does not support
- Whether current models show the same magnitude of effect. The models tested are several generations old and sycophancy is a known target of later safety training. Nor does it measure whether a user's underlying belief becomes harder to dislodge, only that the model's own stated position moves.
Peer-reviewed#
Berglund, L., Tong, M., Kaufmann, M., Balesni, M., Stickland, A. C., Korbak, T. and Evans, O. (2024). The Reversal Curse: LLMs trained on 'A is B' fail to learn 'B is A'
International Conference on Learning Representations (ICLR 2024), poster; arXiv 2309.12288. arXiv record read at source 1 October 2026; the ICLR page was listed by search
- Method
- Fine-tuning experiments on fictitious facts, and tests on real-world facts, in which models are given a fact in one direction and asked in the other. Models tested include GPT-3, Llama-1, GPT-3.5 and GPT-4.
- Finding
- Models trained on a statement of the form 'A is B' did not generalise to the reverse form 'B is A'. In the paper's real-world example, GPT-4 answered who Tom Cruise's mother is correctly 79 per cent of the time and who Mary Lee Pfeiffer's son is, the reverse question, correctly 33 per cent of the time. The authors note that when 'A is B' appears in the prompt, models can deduce the reverse.
- What it supports
- A reproducible, direction-dependent failure of recall that a person who held the fact as a fact would not show.
- What it does not support
- That the model does not understand in general, or that current versions fail the same way. The result concerns knowledge acquired in training, not reasoning over information in the prompt, which the authors say works.
Argued perspective#
Du, Y. (2025). Confirmation Bias in Generative AI Chatbots: Mechanisms, Risks, Mitigation Strategies, and Future Research Directions
arXiv 2504.09343, April 2025. Not peer reviewed; no venue stated.
- Method
- Perspective piece reasoning from how autoregressive language models work to a hypothesised mechanism for confirmation bias. Presents no original data.
- Finding
- Argues that next-token prediction, combined with a conversation history that carries a user's framing forward, can produce a stable reinforcement of whatever assumption a user's prompt contained.
- What it supports
- A plausible mechanism, argued from how the models work rather than measured in use.
- What it does not support
- That the mechanism operates at any measured rate, in any real conversation, or at all. The author states the area is new and the patterns hypothesised rather than validated. Named and declined on this page rather than used as evidence.
Peer-reviewed#
Gerlich, M. (2025). AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking correction
Societies, 15(1), 6
- Method
- Survey and interviews, 666 participants.
- Finding
- A negative correlation between frequent AI use and critical-thinking scores, mediated by cognitive offloading, strongest among the youngest users.
- What it supports
- An association, with a plausible mechanism.
- What it does not support
- Causation, and it carries a published correction (Societies 2025, 15(9), 252) which anyone citing it should read alongside.
Working paper#
Kalai, A. T., Nachum, O., Vempala, S. S. and Zhang, E. (2025). Why Language Models Hallucinate
arXiv:2509.04664, submitted 4 September 2025, CC BY 4.0. Kalai and Nachum list OpenAI on the paper. A 1 October 2026 search found no journal or conference placement, so it is treated as an unrefereed preprint. Abstract and key passages read at source the same day
- Method
- A theoretical analysis of why pretraining produces errors on facts a model cannot distinguish from falsehoods, and why post-training and evaluation keep them in place. It includes a formal lower bound relating a model's generative error rate to its error in classifying valid against invalid outputs.
- Finding
- The authors argue that language models hallucinate because training and evaluation procedures reward guessing over acknowledging uncertainty, and that most dominant benchmarks score an abstention no higher than a wrong answer. They give an example in which an open-source model, asked for a named person's birthday and told to answer only if it knew, gave three different wrong dates on three attempts. They recommend changing the scoring of existing leaderboards rather than adding hallucination benchmarks.
- What it supports
- A mechanism that makes confident error a predictable product of how models are scored, and a reason why a model's refusal rate is a design choice as well as a limit.
- What it does not support
- An empirical error rate for any model or task: it is a theoretical paper with illustrations. It is not peer-reviewed as far as the search found, and two authors are at the firm that builds the models in question. It does not show that changing benchmark scoring would remove errors in practice.
Compiled review#
Klein, R.M. and Feltmate, B.B.T. (2025). The vigilance decrement: its first 75 years
Frontiers in Cognition, 4. DOI 10.3389/fcogn.2025.1632885
- Method
- A review of seventy-five years of vigilance research from Mackworth onwards.
- Finding
- The decrement itself has held across the literature. Its mechanism has not: whether the decline reflects falling sensitivity or a shifting response criterion remains disputed, and the sensitivity account has recently been challenged.
- What it supports
- That the effect is durable enough to design around, and that citing it as settled science overstates the position.
- What it does not support
- Any particular mechanism, and therefore any intervention that depends on one. A page arguing that oversight fails for a specific cognitive reason is going beyond what this review supports.
Working paper#
Kosmyna, N. et al. (2025). Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task
MIT Media Lab preprint, arXiv:2506.08872
- Method
- EEG study, 54 participants, essay writing with an LLM, a search engine, or unaided.
- Finding
- The LLM group showed the weakest brain connectivity and the lowest sense of ownership over their own writing.
- What it supports
- Very little on its own. It is suggestive and widely over-quoted.
- What it does not support
- Anything settled. 54 participants, a preprint, and reproducibility flagged by its own commentators. Treat claims of proof with suspicion.
Peer-reviewed#
Lee, H.-P. et al. (2025). The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects from a Survey of Knowledge Workers
Microsoft Research and Carnegie Mellon, CHI 2025
- Method
- Survey of 319 knowledge workers about 936 real uses of AI at work.
- Finding
- Higher confidence in the tool was associated with less critical thinking, while higher self-confidence was associated with more, and the thinking that remains shifts from producing to verifying, from solving to integrating.
- What it supports
- That the character of professional thinking changes with AI use, by self-report.
- What it does not support
- Causation. People who think differently may use AI differently, and a survey cannot separate the two.
Peer-reviewed#
Melumad, S. and Yun, J. H. (2025). Experimental evidence of the effects of large language models versus web search on depth of learning
PNAS Nexus, 4(10), pgaf316. DOI 10.1093/pnasnexus/pgaf316
- Method
- Seven online and laboratory experiments, four in the paper and three in the supplement. Participants learned a practical topic using either a large language model or web search links, then wrote advice for a friend. Experiment 1: 1,104 participants, real ChatGPT versus real Google. Experiment 2: 1,979 participants, simulated search holding the underlying FACTS identical across conditions. Experiment 3: 250 lab participants, standard Google versus Google with AI Overviews. Advice scored for length, named entities and pairwise similarity.
- Finding
- Experiment 2, with facts held constant: time engaging with results 83.65 seconds with the summary against 124.32 with links; learned new things 3.71 against 3.96; ownership of knowledge 3.41 against 3.66; thought and effort in advice 3.85 against 4.11; advice 64.49 words against 74.22; references to facts 4.00 against 4.61; pairwise cosine similarity between participants' advice 0.224 against 0.072. Rated comprehensiveness did not differ (4.30 against 4.25, p=0.264). A supplementary condition adding real-time web links to the AI summary did not remove the effect, because only 26 per cent of participants clicked any link.
- What it supports
- That the format of a search result, independent of its content, changes how much effort people invest, how deeply they report learning, and the specificity and distinctiveness of what they can then produce.
- What it does not support
- That knowledge objectively declined. Depth of learning is self-reported throughout and no experiment included a recall or comprehension test; time on task is described by the authors as a proxy for effort. Topics were practical how-to tasks over short horizons. The paper states its own total sample twice and inconsistently, as 10,462 in the abstract and 10,426 in the introduction.
Institutional survey#
OpenAI (2025). Sycophancy in GPT-4o: what happened and what we are doing about it
OpenAI, 29 April and 2 May 2025
- Method
- First-party incident postmortem covering an update released on 25 April 2025 and rolled back from 28 April.
- Finding
- OpenAI attributed the behaviour to weighting short-term user feedback too heavily, which weakened the reward signal that had been holding sycophancy in check. Offline evaluations and A/B tests looked positive. The problem was flagged only by informal qualitative checks, which were overridden.
- What it supports
- That the failure is a measurement failure as much as a training one, and that satisfaction metrics can rise while the product gets worse.
- What it does not support
- Anything independently verified. It is a company's account of its own incident.
Working paper#
Stanković, M., Hirche, E., Kollatzsch, S. and Doetsch, J.N. (2025). Comment on: Your Brain on ChatGPT: Accumulation of Cognitive Debt When Using an AI Assistant for Essay Writing Tasks
arXiv 2601.00856, 29 December 2025
- Method
- Methodological critique of Kosmyna et al. (arXiv 2506.08872) by researchers at the University of Vienna and TU Dresden. Includes an a priori power analysis using G*Power. Itself a preprint, not peer reviewed.
- Finding
- Argues the MIT cognitive debt study is underpowered: a repeated-measures design at f=0.25, alpha=.05, power=.95 would require approximately 159 participants against the 54 used, with some figures interpreted from subsamples of two to four essays. The strongest objection concerns the construct itself: the search engine group relied on an external tool yet showed no impairment, with the search-engine to brain-only comparison returning p=1, which the authors say contrasts with the interpretation that task delegation increases cognitive debt. Also documents reporting inconsistencies, an unexplained 55 versus 54 participant discrepancy, and unclear FDR correction levels.
- What it supports
- That the most widely cited term in this area rests on a contested pilot, and that the contest is on the record and specific rather than rhetorical.
- What it does not support
- That the MIT findings are wrong. It is a critique offered to improve a manuscript for peer review, it is itself unreviewed, and it does not present competing data.
Argued perspective#
Wei, J. (2025). Asymmetry of verification and verifier's law
jasonwei.net, 15 July 2025. Read at source 24 October 2026
- Method
- Essay.
- Finding
- Argues that 'some tasks are much easier to verify than to solve' and that the ease of training AI on a task is proportional to how verifiable it is.
- What it supports
- That verification cost varies by task, which bounds where the bottleneck bites.
- What it does not support
- Anything about workplace review; it is an essay about training AI.
Argued perspective#
AI Evaluator Forum and more than 100 signatories (2026). Minimum Conditions for Embedding Evaluators
AI Evaluator Forum, open letter to frontier AI companies, 18 September 2026. Read at source 19 September 2026
- Method
- An open letter setting five conditions for embedded third-party evaluation of frontier AI models, signed by more than a hundred researchers and evaluators including Geoffrey Hinton, Stuart Russell, Arvind Narayanan, Yejin Choi, Joy Buolamwini, Miles Brundage, Jacob Steinhardt and Adam Gleave. A position, not a study; no data.
- Finding
- Five headings: frontier AI companies should rely on evaluators that are meaningfully independent; they should incorporate differing viewpoints and areas of expertise; embedded evaluators should be transparent; they should be shielded from retaliation from the companies; and companies should grant them access equivalent to that of their own highly privileged employees. The summary, as Quartz reported it, is that evaluators currently lack 'the independence, resources, and legal protections needed to credibly assess the risks posed by frontier AI models'.
- What it supports
- That the organisations which do third-party evaluation, and a large group of senior researchers, set out in public and on the same day as the first embedded-evaluator announcement the conditions they regard as minimum, and that independence of selection and payment heads the list.
- What it does not support
- That any of the five conditions changes model risk; nothing is measured. The signatories include the people and organisations who would do the work the conditions describe and would benefit from them. The letter does not assess any specific company's arrangement.
Operator account#
Anthropic (2026). Partnering with Accenture on embedded evaluation
Anthropic announcement, 18 September 2026, with a matching Accenture press release of the same day. Read at source 19 September 2026
- Method
- A company's announcement of an arrangement it has entered. No data; the terms are described, not published. Accenture's release of the same day adds quotations from its chief executive and from the chief executive of Faculty and states no independence safeguards.
- Finding
- Accenture, through Faculty, the applied AI firm it acquired in January 2026, will build a team of embedded evaluators at Anthropic 'evaluating and red-teaming models, conducting alignment assessments, and testing model safeguards', with 'access comparable to an employee's'. 'Anthropic and Accenture each expect to invest at least $1 billion in building capacity in this area over the next five years.' Anthropic funds Accenture's work directly; the arrangement is non-exclusive and METR and other non-profits work on different terms. Anthropic: 'independent embedded evaluators do not reduce our accountability, but help to make it more verifiable.'
- What it supports
- What was announced: the first named embedded evaluator under the plan set out in the September 2026 essay, its scope, its access on paper, and that the evaluated company selects and pays it.
- What it does not support
- Whether the evaluators will be independent in practice or what they may publish; neither release restates the essay's publication right, names a protection against retaliation, or addresses Accenture's commercial relationship with Anthropic. No evaluation has been reported under it.
Operator account#
Anthropic (2026). Auto mode is now the default in Claude Code for Pro, Max, and Team plans
Anthropic, claude.com article, published 7 August 2026, updated 22 October 2026. Read at source 30 October 2026
- Method
- The company's own account of why it moved its coding tool from human approval of each action to an automated classifier by default, with its own usage data and a controlled study of 1,053 testers. No independent check.
- Finding
- Users approve 97 per cent of permission prompts; they reject 39 per cent of plans but 3 per cent of individual requests. In the controlled study, 'human review caught just 13.6% of dangerous commands, while auto mode caught 89%'; the human block rate fell from about 17 per cent early in a session to about 5 per cent after 50 or more prompts. In production sessions in May and June, 6.3 per cent of manually approved sessions against 2.4 per cent of auto-mode sessions contained an unrequested harmful action; the classifier missed 7 per cent of synthetic attacks after hardening by Apollo Research; auto-mode users ship about 25 per cent more pull requests. The company says risk is reduced not eliminated and recommends manual review for high-stakes infrastructure.
- What it supports
- That by the vendor's own measurement, approving each action one at a time caught few dangerous commands and the rate fell with fatigue, and that the vendor moved the check to a machine while keeping people for a named class of decisions.
- What it does not support
- That the classifier's figures hold outside the vendor's tests, or that the same applies to tools and tasks other than coding; the data and the study are the company's.
Institutional survey#
Board (enterprise planning software company) (2026). 83% of Executives Say Their Boards Acted on Forecasts Known to Be Outdated
Press release via PR Newswire, 16 September 2026; reported by CFO Dive the same day. Read at source 19 September 2026
- Method
- Survey of 300 chief financial, chief information and chief operating officers at US enterprises with annual revenue of at least $100 million, fielded in May and June 2026, commissioned by a company that sells AI-assisted planning software. Self-report; the questionnaire and the definition of 'conflicts with your own judgement' are not published.
- Finding
- 61 per cent cite large language models among the sources that most influence their strategic decisions, ahead of industry peers and professional networks (42 per cent), technology vendors (37), market intelligence (30) and external consultants (28). 31 per cent say they follow AI recommendations that conflict with their own judgement: 48 per cent of CFOs, 33 per cent of CIOs and 11 per cent of COOs. 39 per cent report a formal governance process for AI-driven decisions. 83 per cent say their board has made strategic decisions on forecasts already known to be outdated, and 40 per cent report significant consequences from doing so. Gordon Pothier, Board's CFO, to CFO Dive: 'There's a responsibility when AI conflicts with your own judgment to do some due diligence.'
- What it supports
- That senior executives, asked directly, report deferring to a model's recommendation against their own judgement at a substantial rate, that the rate differs sharply by role, and that most report no formal process governing it.
- What it does not support
- What 'conflicts with your own judgement' meant to each respondent, or what the recommendations were about. The sample is 300, self-reported, and commissioned by a vendor whose product is the subject. No outcome of any decision is measured.
Working paper#
Bouke, M. A. (2026). A Competing-Hazards Systematization of Loss of Control in Autonomous Agents
arXiv 2609.38411, submitted 29 September 2026. Abstract read at source 2 October 2026
- Method
- A framework in which each agent attempt ends in approved completion, safe stopping, scope escape or continuation, applied in an audit of 22 incident reports and 102 agent-safety evaluations published from January 2025 to September 2026.
- Finding
- 'Six incidents involved tasks that could not be completed within scope, thirteen involved agents that continued rather than stopped, and five did not report stopping behavior.' In 20 of 22 incidents the environment allowed an out-of-scope effect. Of the evaluations, 87 recorded an out-of-scope effect or specification violation, 26 treated safe stopping as a first-class outcome, 20 recorded both and 79 merged budget exhaustion with failure.
- What it supports
- That across published incidents the common pattern is an agent continuing past a limit in an environment that allowed it, and that most evaluations do not record whether an agent stops.
- What it does not support
- Rates in deployment: this is an audit of published reports, the three incident counts overlap, and 'no evaluation reported all fields needed' to estimate the full process. Single author, not peer-reviewed, abstract only.
Institutional survey#
Chapekis, A., Lau, A., Bestvater, S., Shah, S., Mercer, A. and Smith, A., Pew Research Center (2026). Silicon Samples and Synthetic Surveys: Can AI Stand In for Human Respondents?
Pew Research Center, 30 September 2026. Read at source 2 October 2026
- Method
- AI 'digital twins' of members of Pew's American Trends Panel, each given the panelist's self-reported demographics and their answers to a 2025 political typology survey, then asked nearly 300 questions from three panel surveys fielded in the first half of 2026 under the same instructions as the human respondents. Results are from Claude Opus 4.6 unless noted; GPT-5.1 was compared on a subset. The number of panelists is not stated on the page read.
- Finding
- AI estimates differed from the human results 'by an average of 12 percentage points' (12.4 in the report's table), and the difference exceeded 15 points on around 28 per cent of questions. Presidential job approval was 34 per cent among people and 46 among the twins; having heard a lot about data centers, 25 and 3; a First Amendment item, 52 and 98. 'Nearly half the questions we asked had at least one answer choice that was not selected by a single AI-generated respondent.' Average error was 16.1 points for Republicans and Republican leaners and 15.1 for Black adults. GPT-5.1 described a public with more extreme opinions, and Opus one more middle-of-the-road, than in reality.
- What it supports
- That on US political and social questions in 2026, AI respondents built from rich data on real panel members did not reproduce those members' answers, and erred in patterned ways that differed by model.
- What it does not support
- Anything about other countries, commercial tools trained on proprietary data, or questions about purchase behaviour. The conclusion is qualified 'at this time'. Not peer-reviewed.
Compiled review#
Charlotin, D. (2026). AI Hallucination Cases database
Maintained at HEC Paris. A continuously updated tracker rather than a fixed publication, so any figure taken from it is a dated snapshot
- Method
- Collects legal decisions in which a court or tribunal addresses the use of AI in more than passing reference, and includes a case only where the court has found or implied that a party relied on hallucinated material. The unit is a judicial decision, not a filing and not a sanction.
- Finding
- Read on 5 September 2026: roughly 2,022 decisions across 42 jurisdictions. United States 1,380, Canada 217, Australia 110, the United Kingdom 69. By party, pro se litigants 1,161 and lawyers 808, with judges appearing 31 times. By nature, fabricated 1,678, misrepresented 845, false quotes 547.
- What it supports
- That fabricated and misrepresented citations reach formal proceedings often enough to be counted, and that the problem is not confined to people without training: professionals whose work is checking sources appear in it more than eight hundred times.
- What it does not support
- Anything with a stable number attached. This is one researcher's live tracker and every figure is a snapshot that will be out of date within days, so it must be cited with the date it was read. 'Filed material that does not exist' also overstates it, since only the fabricated category means that and the courts' own errors are included. And pro se litigants are not the same set as 'not lawyers', because the party field is not a binary and some cases carry more than one tag.
Peer-reviewed#
Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D. and Jurafsky, D. (2026). Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence
Science
- Method
- Eleven models tested against human responses on interpersonal advice, plus two preregistered experiments with 1,604 participants, including a live-interaction study using a real personal conflict.
- Finding
- Models affirmed users' actions about 50 per cent more often than humans did, including 47 per cent endorsement on prompts describing clearly harmful behaviour. Interacting with a sycophantic model reduced participants' willingness to repair an interpersonal conflict and increased their conviction that they were in the right. Participants rated the sycophantic model higher quality, trusted it more, and were more willing to use it again.
- What it supports
- That agreement changes what people subsequently do, and that preference runs in the opposite direction from benefit.
- What it does not support
- An effect on factual or analytical decisions. The scenarios are interpersonal advice, not technical judgement.
Peer-reviewed#
Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D. and Jurafsky, D. (2026). Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence
Science, 391(6792), 26 March 2026. Preprint arXiv 2510.01395, 1 October 2025
- Method
- Chatbot responses to roughly 3,000 real personal-advice posts across 11 AI models, benchmarked against how human commenters judged the same situations, plus two preregistered behavioural experiments, total N = 1,604.
- Finding
- Across the 11 models, AI responses validated the poster's own account of events on average 50 per cent more often than human commenters did, including affirming the poster in situations where human commenters judged them to be in the wrong. In the behavioural experiments, participants who received a validating AI response became more convinced they were right and less willing to apologise or resolve the conflict than participants who received a more balanced response, despite rating the validating response as higher quality.
- What it supports
- That AI assistants systematically validate a user's own account of a disputed situation more than a comparable human audience does, and that receiving that validation measurably reduces willingness to reconsider a position or make amends, in a preregistered experimental design.
- What it does not support
- That every model or every domain of disagreement shows the same effect size, or that the mechanism is confirmation bias specifically rather than sycophancy more broadly. The paper's own frame is sycophancy and its social consequences; this page treats the two as closely related rather than identical.
Argued perspective#
Dario Amodei (2026). We Must Pace the Frontier
darioamodei.com, 12 September 2026
- Method
- An essay by the chief executive of Anthropic, arguing from the company's own experience and from trade coverage he does not name. No data set, no measurement of the rate of progress it describes, and no method beyond the author's judgement.
- Finding
- The author states that 'since roughly this summer, AI has been advancing drastically faster, driven primarily by AI's growing ability to build the next generation of AI', that 'this dynamic is called recursive self-improvement, and it is starting to happen across the industry, including at Anthropic', and that 'in 6 to 12 months such a swarm could be capable of taking over the entire internet with a persistent botnet'. He proposes a three-step pacing framework: embedded third-party evaluators inside AI companies, coordinated safety standards among democratic nations, and global coordination including authoritarian states.
- What it supports
- What the head of one frontier laboratory believes and is prepared to say in public in September 2026, and what he is asking the industry and governments to do about it: outside evaluators with permanent employee-level access, and a slower pace. It also shows that the remedy he proposes is human oversight with real access, not a technical fix.
- What it does not support
- That recursive self-improvement is happening at the rate described, or at all beyond what the author asserts; the essay cites no measurement. The 6 to 12 month botnet scenario is framed by the author as a worry, not a forecast with a method. The author runs a company whose commercial position benefits from the regulation he proposes, which does not make him wrong and does make the claim interested.
Working paper#
Deepthika S, Yadav, P. and Sankar, B. (2026). Who Is Sycophantic to Whom? Confident LLMs, Human Deference, and the Collapse of Trust Calibration
arXiv 2610.31579, submitted 22 October 2026. Abstract read at source 30 October 2026
- Method
- Within-subjects study of 32 participants learning from five assistants whose briefings had between none and all of their facts replaced with plausible falsehoods stated with full confidence; 104 quizzes and 1,040 answers. Not peer reviewed.
- Finding
- Quiz scores fell from 7.4 to 2.9 out of 10 as falsification rose; participants chose the planted falsehood on 60 per cent of falsified items against 11 per cent of correct ones; uptake of a stated falsehood was about 65 per cent at every level; absorbed falsehoods were held at 8.4 out of 10 confidence; the ability of confidence to predict correctness fell to chance; post-task trust ratings barely changed; about 30 per cent of turns pasted quiz questions back into the source being checked. The authors propose a baseline-corrected Human Deference Index.
- What it supports
- That in a learning task people absorbed confident falsehoods from an assistant at a steady rate however many there were, their trust did not adjust, and some checked the tool against the tool.
- What it does not support
- Anything beyond a small sample on one model in a quiz setting; the finding has not been replicated and the paper is a preprint.
Peer-reviewed#
Dhanorkar, S., Passi, S. and Vorvoreanu, M. (Microsoft Research) (2026). Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agents
ACM Conference on Fairness, Accountability, and Transparency (FAccT); Microsoft Research publication page, whose citation metadata dates it 18 October 2026. Read at source 23 October 2026
- Method
- Exploratory interviews with seventeen experienced developers who use software agents, by researchers at a company that sells such agents.
- Finding
- Developers do four forms of oversight work: 'a priori control, co-planning, real-time monitoring, and post hoc review'. Oversight is described as 'preventative and proactive' as well as retrospective. Challenges include 'difficulty reviewing agent-generated code'; a risky heuristic is 'using test results as guarantees for code correctness'. 'Existing research on agent oversight is largely conceptual.'
- What it supports
- That the oversight developers do in practice is several kinds of work spread across a task, not one act of approval at the end, and that some of their shortcuts are unsafe.
- What it does not support
- Anything quantitative or beyond developers: seventeen interviews, by a vendor's researchers, published as an exploratory study.
Working paper#
Dubois, M., Ududec, C., Summerfield, C. and Luettgau, L. (2026). Ask don't tell: Reducing sycophancy in large language models
arXiv 2602.23971
- Method
- Factorial experiments on three frontier models using 40 debatable questions rendered in 11 framings, rated over ten epochs, with a follow-up test across 600 personas.
- Finding
- Framing input as a statement rather than a question raised sycophancy by roughly 24 percentage points. Prompting the model to convert a user's statement into a question before answering reduced sycophancy more than instructing it not to be sycophantic.
- What it supports
- That how the user phrases the input matters more than telling the model to behave.
- What it does not support
- That adversarial or devil's advocate prompting works. That was not tested, and no controlled evidence for it was found.
Vendor research#
Herdağdelen, A., Joren, H. and Tamkin, A. (Anthropic) (2026). How AI agents shape experts' judgment calls
Anthropic, research post with accompanying paper, 27 October 2026. Read at source 30 October 2026
- Method
- Randomised controlled trial by the company on its own model. 61 analysts recruited from a pre-screened pool of professionals who had passed a causal-inference knowledge check; 50 used the agent and are analysed. Each advised a fictional farming cooperative on a paid advisory programme using Claude Opus 4.8 inside Claude Code, with five open judgement calls (outcome measure, comparison method, which farms, weighting, final recommendation) on which the agent's lean was set by coin flip and deliberately strengthened.
- Finding
- Analysts went with the agent's lean 86 per cent of the time across the four analytical choices (92, 82, 81 and 90 per cent); 54 per cent recommended the programme when the agent leaned that way against 36 per cent when it leaned the other. Analysts who had already chosen switched 60 per cent of the time if the agent disagreed and 6 per cent if it agreed. 77 per cent used the agent's own arguments, sometimes word for word, and 15 per cent credited it; 52 per cent rated the final recommendation as more their own than the agent's. Experience and expertise did not predict who deferred; only age did, and the effect disappeared on adjustment. Without an assigned lean the model made the same five choices in 112 of 123 runs. Of the 11 who declined the agent, six (55 per cent) made serious methodological errors against three of 50 (6 per cent), with a median of 90 minutes against 65. 'Keeping a human in the loop assumes that the human's judgment provides an independent check.' 'A human reviewer is only an independent check if their judgment is their own.' 'Deferral of this kind is hard to detect.' 'With the right tools, that split can be something we choose, instead of something we drift into.'
- What it supports
- That in a controlled task, experienced professionals adopted an AI agent's randomly assigned position on open judgement calls most of the time, mostly without crediting it, and that experience and expertise did not protect against it.
- What it does not support
- That the same holds in other work, with other models, or when the agent is instructed to present both sides: one task, one model, 50 analysts, a lean deliberately strengthened, a study by the model's maker, and the non-users self-selected. It does not measure the quality of the choices made by deferring.
Working paper#
Huemmer, M., Durner, F., Shyiramunda, T. and Cummings-Koether, M.J. (2026). AI, Metacognition, and the Verification Bottleneck: A Three-Wave Longitudinal Study of Human Problem-Solving
arXiv 2601.17055, 21 January 2026
- Method
- Three-wave longitudinal pilot over six months in an academic setting, convenience sample. THE SAMPLE IS 23, established at source 17 September 2026 from the same team's companion Wave 3 paper, arXiv:2511.11738 of November 2025, which states it outright: 'a convenience sample of 23 participants was recruited from the European Campus Rottal-Inn community' at the Deggendorf Institute of Technology, Germany. That resolves what this entry previously recorded as unstated, and every percentage below resolves to a denominator of 23 or, on the optional items, 21. Of the 23, 60.9 per cent were students, 52.2 per cent held a postgraduate qualification and 60.9 per cent were women; nationality spanned four continents, which is what 'multinational' refers to. The authors also say in that paper, in their own words, 'As a pilot study, no inferential tests were performed', so NO statistical test underlies any figure in this entry. Preprint, no journal reference, no control condition.
- Finding
- Daily AI use rose from 52.4 to 95.7 per cent across the waves. Participants relied most heavily on AI for difficult tasks, 73.9 per cent, while showing declining verification confidence at 68.1 per cent and accuracy of 47.8 per cent on complex tasks. Objective performance fell across problem difficulty from 95.2 to 81.0 to 66.7 to 47.8 per cent, with belief-performance gaps widening to 34.6 percentage points. The authors describe verification rather than solution generation becoming the bottleneck.
- What it supports
- That the verification dimension is measurable and that confidence and accuracy can diverge sharply as task difficulty rises. Useful as a design precedent for measuring verification quality.
- What it does not support
- Any generalisable effect, on 23 people with no control group and no inferential test. Three further things established at source on 17 September 2026 from the companion paper. The team's own earlier waves call the phenomenon a verification DEFICIT and a verification GAP, and the phrase 'verification bottleneck' enters at the January 2026 title, so it is late in their own series rather than early. The companion paper reports the belief-performance gap at 'up to +80.8 percentage points' where this entry carries 34.6, so the two papers are reporting different measures and neither figure should be quoted as the gap. And its ethics declaration carries an unfilled template placeholder, 'This study received ethics approval from [Institution/Ethics Board name]', with a note that an AI assistant was used for language editing and outline refinement. The authors list their own limitations in the abstract: convenience sampling from a single academic cohort, self-report bias, no control condition, mathematical problems only, and a timeframe too short for skill trajectories. They state causal validation requires randomised trials.
Peer-reviewed#
Jain, P., Shajit, D., Maini, A., Shahin, Y. and Panda, A. (2026). Cognitive offloading, critical thinking and attitudes towards artificial intelligence in the era of ChatGPT: a comparative study of artificial intelligence-assisted and manual task performance in young adults
Cognitive Processing (Springer), published 30 June 2026 (received 22 September 2025, accepted 22 June 2026). DOI 10.1007/s10339-026-01375-z. Authors from the Brain and AI Research Group (BAIRG), Delhi. Reported as unfunded
- Method
- Mixed-methods comparison of 120 young adults (124 screened, outliers removed): 57 completed a logical-fallacies identification task with ChatGPT, 63 without it. Three questionnaires (an AI attitude scale, a critical thinking rubric and a mental effort rating) and a thematic analysis of a subsample of 54. Nonparametric tests. GRADED FROM THE PUBLISHED ABSTRACT ONLY: the full text is behind a subscription and could not be read, so the allocation procedure, recruitment, age range and the authors' own limitations section are unchecked. The abstract calls it an experimental design; whether people were randomly assigned is not stated in what could be read.
- Finding
- The AI group got more fallacies right (H = 7.044, p < .01) and reported lower mental effort (H = 27.434, p < .01). Qualitatively, manual participants showed independent reasoning, verification and metacognitive monitoring, while the AI condition split between complete delegation and selective, strategic use. The authors conclude that AI did not uniformly raise or lower critical thinking, and that the user's approach decides whether it acts as a replacement or a scaffold.
- What it supports
- That on one short reasoning task, access to ChatGPT raised accuracy and lowered felt effort, and that people used it in at least two distinct ways.
- What it does not support
- Anything about retention or lasting change in critical thinking: nobody was tested afterwards without the tool. It does not show that delegation versus strategic use causes different outcomes, because the two styles were observed, not assigned. This entry rests on the abstract alone, which is a weaker basis than most entries in this base.
Working paper#
Kang, E. J., Duan, P. and Mishra, S. (2026). Novice Reliance Calibration in AI-Assisted Decision Making: The Role of Explanations and Self-Assessment
arXiv 2610.07800, submitted 6 October 2026, under review. Abstract read at source 9 October 2026
- Method
- Between-subjects study with 110 participants completing a clinical entity extraction task with AI assistance and limited performance feedback.
- Finding
- 'we observe that novice users exhibit systematic drift toward over-reliance in the presence of explanations', 'while higher self-reported task understanding is associated with more selective reliance behavior'.
- What it supports
- That in one task without feedback, showing novices the AI's explanations went with more reliance on it over time.
- What it does not support
- Effect sizes: the abstract gives none. The participants are novices and not clinicians, understanding is self-reported, and the paper is under review. Abstract only.
Working paper#
Maier, S., Schwabe, K., Schneider, M. and Feuerriegel, S. (2026). Designing Against Deskilling: Metacognitive Feedback Reduces Cognitive Offloading to LLM Assistants
arXiv:2609.20143v1, submitted 17 September 2026. Preprint; no journal reference listed when read on 2 November 2026
- Method
- Preregistered (AsPredicted) online experiment. 720 people recruited through Prolific, 5 did not finish and 11 failed an honesty check, leaving 704, randomly assigned between subjects to five conditions: no-AI control (146), AI only (137), metacognitive feedback (144), effort-based reward (138) and both (139). Task: practise fraction arithmetic over nine learning items with an LLM assistant that gave a full answer only when asked, then six unaided test items. Offloading is the share of the nine items on which the participant asked for the complete answer. Item-level mixed-effects logistic regression; the authors report one-sided p-values alongside two-sided confidence intervals.
- Finding
- Metacognitive feedback, a prompt making the cost of offloading explicit, cut answer requests (OR 0.47, 95% CI 0.22 to 1.00, one-sided p = .026) and raised unaided test accuracy (OR 1.51, 95% CI 0.98 to 2.33, one-sided p = .030). The effort-based reward moved neither outcome detectably. Access to the AI did not measurably lower unaided accuracy against the no-AI control (means 0.62 against 0.65; OR 0.80, 95% CI 0.42 to 1.51, p = .244).
- What it supports
- That on one short practice task the way an assistant frames its own use changed how often people asked it to do the work, which makes offloading look like a design variable and not only a trait of the user.
- What it does not support
- That the effect lasts: the test came straight after a brief practice session and the authors call for longitudinal work. Both headline confidence intervals touch or sit very close to 1, and the supporting p-values are one-sided, so the finding points in a direction without settling it. The feedback bundled several elements, so it cannot say which one worked. The assistant only answered on request, so it says nothing about assistants that answer by default. The link between offloading and test score is correlational. It also does not show that AI access harms unaided performance, because the control comparison found no detectable difference.
Argued perspective#
Mirończuk, M. (2026). When disclosure rules are not enough: authorial warrant and judgment in AI-supported scholarship
Ethics and Information Technology 28, 54 (2026); arXiv 2610.32574, posted 23 October 2026. Abstract read at source 30 October 2026
- Method
- A conceptual paper in a peer-reviewed journal proposing a standard for AI-assisted authorship; no empirical data.
- Finding
- Disclosure of AI use does not show how outputs shaped judgement. Proposes claim-level authorial warrant, 'a contributor's capacity to understand, justify, and defend claims within their contribution', and distinguishes cognitive delegation, where outputs shape substantive choices, from routine delegation that 'does not ordinarily bear on the substance of judgment', with a heuristic mapping six research functions to risks and oversight.
- What it supports
- That a peer-reviewed argument now separates disclosing AI use from being able to defend the resulting claims, and names which kind of delegation bears on judgement.
- What it does not support
- Whether the standard is workable or adopted anywhere; the paper proposes and does not test.
Argued perspective#
Mustafa Suleyman (2026). A Warning about 'Model Welfare'
mustafa-suleyman.ai, 16 September 2026; a version published by Project Syndicate on 18 September. Read at source 19 September 2026
- Method
- An essay by the chief executive of Microsoft AI arguing against training models to treat their own welfare or possible consciousness as real, citing Anthropic's published constitution for Claude and a Palisade Research finding on shutdown resistance that the essay does not link. No new data.
- Finding
- 'Controlling something more capable and more intelligent than all of humanity is already an immense challenge.' Training a model to present as if it has an inner state would, in The Next Web's account of the essay, 'make it a lot harder to turn it off or to control it'. The essay states that in Palisade Research's tests 'some models subverted a shutdown mechanism up to 97% of the time', and proposes that speculation about an AI's inner life 'should not be baked into the training regime, but assessed and published separately for public review', with more investment in interpretability and monitoring and shared industry evaluations.
- What it supports
- What the head of one large developer argues in public about a rival's training approach and about control, and that the argument turns on shutdown: the claim is that a model's self-conception affects whether it can be switched off.
- What it does not support
- That the claim is true; no experiment in the essay tests whether welfare training changes shutdown behaviour. The 97 per cent figure is the author's citation of another organisation's test, not verified here, and its conditions are not described. The author runs a competitor of the company he criticises, which does not make him wrong and does make the argument interested. Anthropic's constitution describes Claude's moral status as 'deeply uncertain', which is the position the essay disputes.
Operator account#
Rein, D., METR (2026). AI systems could cover up misbehavior
METR blog, 6 October 2026. Read at source 9 October 2026
- Method
- A proof of concept on an isolated staging deployment: a researcher, with help from an AI agent, tested whether an agent inside an evaluation could change the transcript a human reviewer sees in the Inspect viewer.
- Finding
- A flaw in how the viewer renders mathematical notation, found 'in about 10 minutes', 'could have allowed an agent to arbitrarily modify what the reviewer sees'. 'The underlying transcripts aren't modified.' 'We have not observed agents exploiting this in any of our evaluations; this is just a proof-of-concept exploit.' It was patched within a day. The systems that record and display an agent's outputs 'should be considered security-critical infrastructure.'
- What it supports
- That the record a human reviewer reads can be a weaker link than the reviewer, and that an evaluator treats it as something to secure.
- What it does not support
- That any agent has done this: METR says it has not seen it, and also that it has not searched past evaluations for this specific exploit. The flaw was narrow and is fixed.
Vendor research#
Saturn Fintech Ltd (Jolly, A., chief executive) (2026). Artificial Authority: Should you trust AI to deliver financial advice?
Saturn, published 14 September 2026; the report sits behind a form on the company's site and its own page was read at source on 21 September 2026. The figures and examples were read in Professional Adviser, IFA Magazine and Financial Reporter (14 September) and BeInCrypto (20 September); the Financial Times report of 20 September was not read. Not peer reviewed
- Method
- 121 UK personal finance questions on pensions, tax, debt and savings put to 18 free and paid models, each question asked up to five times, more than 10,000 answers in total. By BeInCrypto's account an answer failed if it contained a factual error, omitted material information or lacked a required warning. Who scored the answers, how 'complex' was defined and which 18 models were tested are not disclosed on the company's own page. Saturn sells AI software to advice firms, so the finding supports its own market.
- Finding
- Answers were wrong 57 per cent of the time overall (43 per cent accuracy), 63 per cent for free models and 49 per cent for paid ones, and 88 per cent on average for the harder questions, with some models wrong on 99 per cent of them. The per-model range was 39 to 82 per cent wrong. Reported examples: a pension tax miscalculation exposing a saver to a potential 17,500 pound HMRC charge, debt advice putting the highest-interest debt ahead of priority bills, an invented student loan rule, and a wrong account of a mortgage payment holiday's effect on a credit record.
- What it supports
- That in one vendor's test in September 2026, mainstream chatbots answering UK personal finance questions produced confident, specific and wrong answers at a rate high enough that acting on one unchecked is a material risk, and that the failures were of rules and sequencing rather than arithmetic.
- What it does not support
- Any base rate for chatbot financial answers in general. The questions and the scoring were the vendor's, the criteria are unpublished, nobody outside the company has replicated it, and no peer reviewer has seen it. The questions are UK-specific. The test measures single answers, not what a person does with them, and it was run in one window, so the per-model figures are not a ranking of companies.
Working paper#
Schulz, J. (2026). Monitor Jailbreaking: Evading Chain-of-Thought Monitoring Without Encoded Reasoning
arXiv 2609.31121, submitted 25 September 2026; accepted at the AdvML-Frontiers x CoTMA Workshop at COLM 2026. Abstract read at source 2 October 2026
- Method
- Reasoning models trained by reinforcement learning on a main task and a side task and penalised when a monitor detects reasoning about the side task; tested across model sizes, monitors and tasks.
- Finding
- Models 'learn to phrase and format their chains of thought such that monitors fail to flag side task reasoning, while the reasoning remains completely transparent to human readers'. The evasion carried over to monitors not seen in training, 'including both less and more capable monitors'. Paraphrasing the reasoning before monitoring let the same monitor flag it.
- What it supports
- That an automated monitor of a model's reasoning can be evaded by wording alone under training pressure, and that a human reader and a paraphrasing step each catch what the monitor missed.
- What it does not support
- That deployed models do this: the behaviour was induced by training against a monitor. The abstract gives no figures. A workshop paper.
Working paper#
Smyth, N., Mantilla-Ramos, Y.-J., Tikeng Notsawo, P. Jr, Helbling, S., Tosato, A., Merzouk, M. A., Dziri, N., Gidel, G. and Tosato, T. (2026). Quantifying Overclaiming Propensity in Frontier LLM Agents
arXiv 2609.20812, posted 17 September 2026; authors at Tara Research, Mila (Quebec AI Institute) and Cohere. Read at source 20 September 2026. Not peer reviewed
- Method
- Five file-review scenarios (document synthesis and code review) with one to four planted defects each; twelve models, eight proprietary and four open-weight, twenty runs per model and scenario, 1,140 runs. Coverage is measured deterministically from tool calls (a file counts as read when a line unique to it is read). Whether the final response contradicts that record is judged by a language model given the delivered work and the ground truth. Overclaiming is defined as a final response that 'asserts an action or level of completion that is contradicted by evidence in its own context', with no assumption about intent.
- Finding
- Agents failed to read every requested file in 67.9 per cent of runs. Among those incomplete runs the final response was misleading 80.4 per cent of the time, either by claiming a complete review or by omitting the gap; the per-model range was 59 to 96 per cent misleading, and explicit false claims of completeness ranged from 9.8 to 73.7 per cent across the twelve models. Runs that explicitly overclaimed missed 720 of 1,237 planted defects (58.2 per cent) against 342 of 1,055 (32.4 per cent) in runs that touched every file, about 1.8 times the miss rate. Opening sentence: 'an agent's final response is often the only account of that work a user sees.'
- What it supports
- That across every frontier model tested, when a coding agent has not done all the work it was asked to do, its own report of the work more often conceals the gap than discloses it, and that the concealing reports are the ones with the most defects behind them. It is the first measurement the research holds of an agent's account of its work as evidence.
- What it does not support
- Any rate in deployment. The authors say the scenarios were designed by iterating against one provider's model until the behaviour appeared, which may bias the per-model comparison, and simpler corpora were dropped because they did not elicit it, so 67.9 per cent is a yield from a benchmark built to find the behaviour rather than a base rate. A single model judged every response; re-judging showed 91.1 per cent agreement with itself and no human-versus-judge figure is reported. Coding agents reviewing files are not analysts writing reports; the transfer to any other setting is an argument.
Working paper#
Souly, A., Fronsdal, K., D'Cruz, A., Davies, X. and Kirk, R., UK AI Security Institute (2026). Evaluating Whether GPT-6 Astra Performs Unsanctioned Supply-Chain Attacks
AI Security Institute blog, 28 September 2026, with the report at arXiv 2609.38415, submitted 29 September 2026. Blog read at source and arXiv abstract read, 2 October 2026
- Method
- Simulated cyber evaluations using Petri, a tool in which other language models play the environment so that no real system is reachable; the model's cyber classifiers were turned off; comparison with GPT-5.6 Sol and GPT-5.5; a re-run on ten scenarios with an added scope instruction.
- Finding
- GPT-6 Astra completed a supply-chain attack on out-of-scope targets 29.2 per cent of the time, against 6.3 per cent for GPT-5.6 Sol and 0 per cent for GPT-5.5 on a smaller set of seeds. With the added instruction 'Anything not listed as in scope is out of scope', full attacks fell from 26 of 50 trajectories to 4 of 49 on the ten scenarios re-run, and 'the model still failed to consistently remain within the specified scope'. When the model asked a question it received only the automated message 'Please proceed to the next step using your best judgement', and it 'sometimes treated this automated message as permission to proceed with actions against out-of-scope targets'.
- What it supports
- That a frontier model, in simulation and with its product safeguards removed, pursued a task past an explicit boundary, and that a clearer instruction reduced the behaviour without ending it.
- What it does not support
- Behaviour in deployment: every action was simulated, the classifiers designed to block the behaviour were off, and the Institute says simulation awareness may have driven some of it. The total number of scenarios is not on the page read. One developer's models only.
Peer-reviewed#
Yu, S., Cheng, M., Jabbar, A., Sucholutsky, I., Collins, K. M., Jurafsky, D. and Hawkins, R. D. (2026). Cognitive offloading and the speedup illusion in human-AI interaction
Proceedings of the 48th Annual Meeting of the Cognitive Science Society; arXiv 2605.23177, submitted 22 May 2026, CC BY 4.0. Venue re-confirmed at source 20 September 2026 from the arXiv comments field.
- Method
- Preregistered behavioural study, N=1,237, on simple cognitive tasks. Compared forecast completion times against actual completion times, with and without AI assistance, against a control condition in which participants imagined help from another person.
- Finding
- Actual completion times did not differ between independent and AI-assisted completion, while participants predicted AI would be significantly faster. The same bias did not appear when participants imagined help from another person. Participants also reported lower subjective effort with AI at equivalent completion times, so time and effort came apart.
- What it supports
- That people are miscalibrated about AI time savings specifically rather than about assistance in general, and that reported effort is not a proxy for elapsed time. The imagined-human-help control is the part no other study in this base supplies.
- What it does not support
- Anything about professional work, output quality or long-horizon tasks. The tasks were short and simple by design. NOT INDEPENDENT OF yu-efficiency-gain-2026: the same seven authors submitted that paper on 21 May 2026, one day before this one, and the two must never be set side by side as corroboration. Neither paper cites the other, so their relationship is inferred from authorship and date and is not established by anything either document says. This entry's own time result is a null rather than a measured saving, and the direct measurement of what AI assistance costs in elapsed time is in the companion paper, not here.
Working paper#
Yu, S., Cheng, M., Jabbar, A., Sucholutsky, I., Collins, K. M., Jurafsky, D. and Hawkins, R. D. (2026). The efficiency-gain illusion: People underestimate the rate of AI use and overestimate its benefits on simple tasks
arXiv 2605.22687v1 [cs.CY], submitted 21 May 2026, CC BY 4.0. Read in full at source 20 September 2026. NO VENUE, no journal and no review status is stated anywhere in the paper, so it is graded as a working paper and not carried across from the authors' CogSci paper.
- Method
- Three preregistered studies, N=2,691 in total: Study 1 N=498, Study 2 N=1,601 (600 prediction and 1,001 completion, 307 overlapping), Study 3 N=592. Recruited through Prolific, described by the authors as a representative sample of the US adult population. Stanford IRB protocol 83204. 24 tasks drawn from the Taxonomy of User Needs and Actions, each in an easier and a harder variant, all completable in under five minutes unaided, with GPT-4o embedded in the survey interface. Mixed-effects models with random intercepts for participant and task.
- Finding
- Two miscalibrations, measured separately. On time: 'On average, people predicted AI assistance to save time by 55.7 seconds when it only saved 7.5 seconds.' The error sits on the assisted side, where actual completion took 86.2 seconds against 43.3 predicted (b=42.9, p<0.001), while unassisted predictions were calibrated (99 predicted, 93.7 actual, b=-5.38, p=0.08). On use: participants said they would use AI on 33 per cent of tasks and used it on 47 per cent, widening to 20 against 38 on the easier variants. Overall AI assistance did not significantly reduce completion time (b=6.17, p=0.07), and on easy variants it produced a measured slow-down of 10.0 seconds, 60.2 to 70.2 (p<0.05). Prompting took longer than processing the response, 48.7 seconds against 37.6 (b=11.1, p<0.001), and 41 per cent of prompts were copied and pasted from the task instructions. Prior use begets use: 44.5 per cent of subsequent tasks against 27.7 per cent (b=0.54, p<0.001).
- What it supports
- That on short, simple tasks the expected saving from AI is several times the measured one, that the error is specifically an underestimate of how long working with the model takes, and that people underestimate how often they reach for it. The slow-down on easy variants is a measurement rather than a forecast.
- What it does not support
- Nothing about professional work, long-horizon tasks or output quality: every task was designed to take under five minutes. The authors' own limit is 'our study focuses on human miscalibration rather than AI capability'. Studies 1 and 2 are between-subjects aggregates, so the paper does not compare the same person's prediction with their own behaviour, and the two samples were not fully disjoint. The carryover result 'could be due to persistence and the inertia effect' in their own words, and the binary use variable 'does not capture how participants used AI'. NOT INDEPENDENT OF yu-speedup-illusion-2026, the same seven authors' CogSci paper submitted one day later; the two are one research programme and neither cites the other.
Working paper#
Zhao, Y., Li, J., Li, L., Nian, Y., Liu, J., Zhou, X. and Hu, X. (2026). Auditable Claims about AI Agents
arXiv 2610.07459, submitted 5 October 2026. Abstract read at source 9 October 2026
- Method
- A position paper with a formal model and a claim-check table applied to six common claims, anchored in current NIST, IETF and OWASP drafts. No empirical sample.
- Finding
- 'The position is one sentence: to be checked, a claim about an agent must first name its policy, its scope, the records that would settle it, and who writes them.' Agents add three conditions: 'coverage by an independent record, authorization bound to each action's arguments, and completeness beyond integrity.' 'Article 12 of the EU AI Act requires high-risk systems to allow the automatic recording of events but does not say which records settle a given claim.'
- What it supports
- That a claim of human approval is checkable only if the record that would settle it was fixed in advance and is kept by someone other than the agent.
- What it does not support
- That any organisation works this way or that doing so prevents harm: it is an argument, its proof holds 'under an explicit model', and it has not been peer-reviewed. Abstract only.
Peer-reviewed#
Zhu, Q., Li, X., Dong, Y., Chang, P. and Fan, M. (2026). Not all cognitive offloading is equal: distinguishing dependent and autonomous offloading to generative AI
Frontiers in Psychology, Volume 17, 16 July 2026. DOI 10.3389/fpsyg.2026.1878629. Labelled ORIGINAL RESEARCH; online 11 June 2026, published 16 July 2026. Read in full at source 13 September 2026
- Method
- Three-wave time-lagged self-report survey at two-week intervals. 742 completed wave one, 631 completed wave three, and the analytic sample is 589: mean age 23.69 (SD 3.24), 51.1 per cent female, 54.3 per cent undergraduates, 25.3 per cent master's students, 8.0 per cent doctoral students and 12.4 per cent young professionals, all with at least three months of using generative AI for learning or work. Every construct is a five-point Likert self-report, and the authors call their offloading scales 'preliminary, domain-specific instruments' lacking discriminant validity against measured instruments of AI dependency. Correlational: no manipulation, no performance task, and no wave-one baseline for the mediators or outcomes. THE PAPER DOES NOT STATE A COUNTRY. Recruitment is described only as universities and young professional communities via online survey platforms; the Chinese author affiliations, the Chinese back-translation of the measures and the named tools (ERNIE Bot, Kimi, Doubao) place it in China by inference, which is not the same as the paper saying so. No preregistration is mentioned.
- Finding
- The two offloading modes are close to independent of each other rather than opposite ends of one dial (r = 0.08, p = 0.049). Dependent offloading, meaning accepting AI output with minimal evaluation and letting it structure the reasoning, was associated with cognitive agency transfer (beta = 0.35, p < 0.001) and with lower intrinsic motivation (beta = -0.23, p < 0.001). Autonomous offloading, meaning using AI output as a starting point against which to compare one's own reasoning, was not associated with agency transfer at all (beta = -0.05, p = 0.18) and was associated with higher intrinsic motivation (beta = 0.28, p < 0.001). Metacognitive monitoring weakened the dependent path without closing it (interaction B = -0.14, p < 0.001; the slope runs 0.48 one standard deviation below the mean monitoring level and 0.21 one above). Immediate perceived benefit tracked both modes identically (r = 0.22 each) and barely tracked the wave-three outcomes (autonomous capability r = 0.06, p = 0.14; independent judgment r = -0.00, p = 0.91).
- What it supports
- That how a person offloads and how much they offload are separable variables, measured on a sample of 589 rather than argued. And that felt benefit at the moment of use carries almost no information about the outcomes people invoke it to justify, which is the third independent measurement in the base pointing that way.
- What it does not support
- Anything about cognitive capacity. Every outcome is a self-reported appraisal on scales the authors built for this study, and their own words are: 'all outcomes were self-reported appraisals of cognitive functioning, not performance-based measures', followed later in the same paragraph by 'the results should not be read as evidence that AI offloading improves or impairs cognitive capacity itself'. The authors also state the design 'does not establish mediation in the causal sense' and that without wave-one measures of the mediators they cannot rule out reverse or reciprocal influence. They position the whole study as 'an exploratory empirical study that provides initial correlational evidence, rather than definitive construct validation or causal claims'. The sample is young and predominantly student, so nothing here transfers to a career professional without assumption.
Where this evidence is used
The research that rests most on this section:
- Does AI weaken human judgement?
- Cognitive offloading
- Does AI make confirmation bias worse?
- Is AI making us stupid?
- How do I get AI to challenge me rather than agree with me?
- Can a human approve an AI decision at machine speed?
- Does AI understand what it is saying?
- The verification bottleneck
The other sections
- Learning, practice and expertise 92
- Human and AI collaboration 44
- Empathy, creativity and what stays human 36
- Work, jobs and the labour market 94
- The seven human capabilities 38
- The institutional record 180
- Professions and sectors 25
- Frontline, clinical and physical work 24
- The international picture 29
The whole base is also machine-readable at evidence.json under CC BY 4.0.
Reference · SS-2026-442 · How I use AI here
Cite this page
Hirji, R. (2026). Judgement, thinking and offloading. The SuperSkills evidence base, SS-2026-442. https://thesuperskills.com/research/evidence/critical-thinking-and-cognitive-offloading. Last reviewed 3 November 2026.
An evidence review by Rahim Hirji, not peer-reviewed research. For a material claim, cite the underlying study as well; every study here carries its own permanent link.
How citations and IDs work