Each entry gives the study, how it was done, what it found, what it supports and, as plainly, what it does not. Grades describe what kind of evidence a source is; how to read them is on the main page. Every entry has a permanent link and a copyable citation.
Compiled review#
Transportation Safety Board of Canada (2019). Aviation Investigation Report A19P0112, Seair Seaplanes, Addenbroke Island
Transportation Safety Board of Canada
- Method
- Single-accident investigation report. The passage on skill degradation sits in section 1.8 and reports concerns gathered from air-taxi operators in a separate TSB Safety Issue Investigation, not findings about this accident.
- Finding
- Surveyed air-taxi operators expressed concern that 'dependence on technology was causal in degradation of basic piloting skills', and numerous operators commented that over-reliance on GPS navigation 'may contribute to the decision to fly into adverse weather conditions'.
- What it supports
- That an industry regulator was recording practitioner-reported skill degradation from automation dependence, in an operational setting, before generative AI existed. It also captures the double bind: the sector's stated problem is too little technology and its observed failure mode is over-reliance on it.
- What it does not support
- That automation dependence caused this crash. The accident's own findings as to causes cite weather, terrain-alerting ambiguity and fatigue. This is industry survey commentary quoted inside an accident report, and citing it as a causal finding would misrepresent it.
Peer-reviewed#
Dratsch, T. and others (2023). Automation Bias in Mammography: The Impact of Artificial Intelligence BI-RADS Suggestions on Reader Performance
Radiology, 2023; RSNA press release 2 May 2023. Read via the release 24 October 2026
- Method
- 27 radiologists read mammograms with AI suggestions, some deliberately wrong.
- Finding
- With wrong suggestions, accuracy of the most experienced readers fell from 82 per cent to 45.5 per cent, and of inexperienced readers from almost 80 per cent to under 20 per cent.
- What it supports
- That wrong AI suggestions pull even experienced experts off the right answer.
- What it does not support
- Usual-case performance, since the wrong suggestions were deliberate.
Peer-reviewed#
Education Endowment Foundation and National Foundation for Educational Research (2024). ChatGPT in lesson preparation: a Teacher Choices trial
EEF project record and evaluation report, 12 December 2024, project completed August 2026. Co-funded by the Hg Foundation; the ChatGPT guide was developed by Bain and Company's Social Impact practice
- Method
- Two-arm school-randomised Teacher Choices trial, 259 Year 7 and Year 8 science teachers across 68 state-funded secondary schools in England, ten weeks in the summer term of 2024. One arm used ChatGPT with a written guide; the other was asked to use no generative AI. Weeks one to five were a familiarisation period and planning time was recorded in weeks six to ten. Resource quality was assessed by an expert panel blinded to condition. Independent evaluation by NFER.
- Finding
- Weekly lesson and resource preparation time was 56.2 minutes in the ChatGPT arm against 81.5 minutes in the comparison arm, a saving of 25.3 minutes and a reduction of 31 per cent, given a high security rating. The blinded panel found no noticeable difference in resource quality. The proportion of ChatGPT-arm teachers who felt they spent too much time on preparation fell from 49 to 26 per cent, with no similar fall in the comparison arm. Frequency of use, and consultation of the guide, both declined over the trial.
- What it supports
- That light, largely unsupported use of a general assistant reduces teacher preparation time measurably, without a detectable cost to the quality of the materials produced, in one subject at one key stage.
- What it does not support
- Anything about teaching or about pupils. Preparation time was the outcome; classroom effect was outside scope. Time was self-recorded rather than observed. The sample over-represents schools in London and the South East and schools rated Outstanding, which the report states. The 31 per cent excludes a five-week familiarisation period.
Peer-reviewed#
Wilson, K. and Caliskan, A. (2024). Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval
Proceedings of the 2024 AAAI/ACM Conference on AI, Ethics, and Society; preprint arXiv 2407.20371
- Method
- Resume audit study run through a document retrieval framework simulating job candidate selection. Massive Text Embedding models tested across nine occupations using over 500 publicly available resumes and over 500 job descriptions, with 120 first names associated with male, female, Black and white candidates. Code published by the authors.
- Finding
- The embedding models significantly favoured White-associated names in 85.1 per cent of cases and female-associated names in 11.1 per cent, with a minority of comparisons showing no statistically significant difference. Black male candidates were disadvantaged in up to 100 per cent of cases. Document length and the corpus frequency of a name also affected selection. Three hypotheses of intersectionality were validated.
- What it supports
- That the representation layer underneath commercial screening tools carries a large, measurable and intersectional name-based bias before any product logic is added, and that some of the effect is an artefact of name frequency in training data rather than anything about a candidate.
- What it does not support
- The behaviour of any deployed product. These are open embedding models in a simulated pipeline, not the proprietary systems vendors sell, which add filters and thresholds and cannot be independently tested. It measures ranking behaviour rather than who was hired.
Compiled review#
Divisional Court of England and Wales (Dame Victoria Sharp P and Johnson J) (2025). Ayinde v London Borough of Haringey and Al-Haroun v Qatar National Bank
[2025] EWHC 1383 (Admin), judgment of 6 June 2025
- Method
- Two referrals under the Hamid jurisdiction, heard together, arising from fabricated citations placed before the court. In Ayinde, five cited authorities did not exist. In Al-Haroun, a claim for £89.4 million, eighteen of forty-five cited authorities did not exist.
- Finding
- At [6] the court held that freely available generative AI tools trained on a large language model are not capable of conducting reliable legal research. At [7] those using them have a professional duty to check accuracy against authoritative sources, which the court names. At [8] the duty extends to lawyers relying on others' AI-assisted work. At [81] a lawyer is not entitled to rely on their lay client for the accuracy of citations. At [23] the available powers run from public admonition and wasted costs to contempt and referral to the police, and at [31] admonishment alone is unlikely to suffice save in exceptional circumstances. At [9] leadership responsibility falls on heads of chambers and managing partners, and the court states it will inquire in future hearings whether that responsibility was fulfilled.
- What it supports
- That in England and Wales the verification duty is settled, non-delegable and extends upward to those who supervise. It also establishes that the court will treat a fabricated citation as a possible supervision failure rather than only as an individual one.
- What it does not support
- Anything about jurisdictions other than England and Wales, or about a lawyer who checked competently and was still misled, which no reported case has yet tested. It is a judgment rather than an empirical study, and it measures nothing.
Institutional modelling#
Financial Reporting Council (2025). AI in audit: Illustrative example and documentation guidance
Financial Reporting Council, June 2025, published 26 June 2025
- Method
- First FRC guidance on artificial intelligence in statutory audit, developed with the FRC's Technology Working Group. Two parts: a worked example of an unsupervised machine learning tool used for journal risk assessment, and principles for documenting tools that use AI on the audit file.
- Finding
- The scope is deliberately broad, covering 'both traditional machine learning techniques and deep learning models, including generative AI'. On explainability the FRC declines to set a threshold: 'what constitutes appropriate explainability will vary widely based on context', and appropriate explanations 'may, particularly in relation to tools that rely on neural networks, be approximate or post hoc explanations that seek to explain how inputs influence outputs rather than the internal features and workings of the model'. Automation bias appears once, as something training material should carry 'strategies to mitigate'. Engagement teams are required to understand why the tool flagged an item and to stay alert to the possibility that its assessment is systemically flawed for that entity. The guidance states that it is not prescriptive and that 'the requirements against which firms will be assessed remain only those in the ISQMs and ISAs (UK)'.
- What it supports
- That a professional regulator can bring generative AI inside an existing evidence and documentation standard without writing new rules, and that it can do so while accepting post hoc explanation rather than model transparency.
- What it does not support
- Anything about practice. It is guidance issued in 2025, not a finding about what firms do, and the FRC says it creates no new requirements. It contains nothing on junior auditors, training pipelines or skills: the words junior, trainee and graduate do not appear.
Compiled review#
Financial Reporting Council (2025). Thematic Review: Certification of Automated Tools and Techniques
Financial Reporting Council, June 2025, published 26 June 2025. Fieldwork Q2 2024 to autumn 2024
- Method
- Review of the processes and controls by which the six largest UK audit firms, named as BDO, Deloitte, EY, Forvis Mazars, KPMG and PwC, certify automated tools and techniques before use in audits. Information request issued April 2024, firm meetings summer 2024, feedback autumn 2024. A snapshot of process, not an inspection of audits.
- Finding
- All six firms had certification processes, but 'the maturity of these processes was found to vary and in some cases were not supported by formal documented policies'. Only two of the six set out the limitations of a tool or restrictions on its use in the certification documentation. Three captured assessment of the supporting IT control environment. One enforced a minimum recertification frequency, of three years. Generally the firms had no key performance indicators for tool usage and monitoring. The finding that carries furthest: 'There was no formal monitoring performed by the firms to quantify the audit quality impact of using ATTs.' At the time of review, generative AI use was limited to productivity aids such as chatbots rather than tools producing audit evidence.
- What it supports
- That the profession whose function is verification had, as at 2024, deployed the tools that produce its evidence without measuring their effect on the quality of that evidence, by its regulator's own account.
- What it does not support
- That audit quality has fallen. The review measures process rather than outcome, covers only the six largest firms, and its counts are of documentation practice rather than of tools or audits. Nothing here is a sample of engagements.
Compiled review#
Fletcher, J. and Verckist, D. (BBC and European Broadcasting Union) (2025). News Integrity in AI Assistants: An international PSM study
BBC and European Broadcasting Union, published 21 October 2025
- Method
- Twenty-two public service media organisations across 18 countries and 14 languages put 30 shared news questions, drawn from questions audiences had actually asked, to the free consumer versions of ChatGPT, Copilot, Perplexity and Gemini between 24 May and 10 June 2025. Each prompt opened a new chat and asked the assistant to use the participating organisation's sources where possible. Assistants were anonymised and 271 journalists graded 2,709 core responses against accuracy, sourcing, separation of opinion from fact, editorialisation and context, marking each as no issues, some issues, significant issues or don't know.
- Finding
- Forty-five per cent of responses carried at least one significant issue, and 81 per cent carried an issue of some kind. Sourcing was the largest single cause at 31 per cent, then accuracy at 20 per cent and insufficient context at 14 per cent. Gemini recorded significant issues in 76 per cent of responses against 37 per cent for Copilot, 36 per cent for ChatGPT and 30 per cent for Perplexity, driven by sourcing, where Gemini's rate was 72 per cent against 24, 15 and 15. Of responses that cited a participating broadcaster's content, 15 per cent misrepresented it. Where the same BBC-only comparison could be run against the 2025 first round, significant issues fell from 51 per cent to 37 per cent.
- What it supports
- That misattribution and unsupported sourcing, rather than outright fabrication, is the dominant failure mode when general assistants answer news questions, and that it holds across languages, territories and platforms rather than being an artefact of one market or one model.
- What it does not support
- Current performance of any named product. These were free consumer versions tested in mid-2025 and all four have shipped new defaults since. It is not adversarial testing and difficulty was not controlled, so the rate is not a worst case; equally, per-organisation samples were around 120 responses, which the authors say is too small to compare countries or languages. Nothing here measures what readers then believed or did. The assessors work for the organisations whose own journalism is the subject matter and who have a stake in the finding. NOT INDEPENDENT OF ebu-bbc-news-integrity-2025: the same study, read there from the EBU release and toolkit rather than from the report, and graded separately on that narrower source. Counting both would double one study.
Institutional survey#
Government Digital Service (2025). Microsoft 365 Copilot Experiment: Cross-Government Findings Report
Government Digital Service, June 2025
- Method
- Cross-government trial of M365 Copilot from 30 September to 31 December 2024, 20,000 licences across twelve organisations, each committing at least 1,000. Usage data from the Microsoft dashboard for 14,500 users; survey of 7,115 users; five focus groups. Time savings were self-estimated by selecting a band, and the average was computed from band midpoints with the largest savings estimated at 60 minutes.
- Finding
- Average self-reported saving 26 minutes a day. Adoption reached 83 per cent and held around 80 per cent. 17 per cent noticed no clear saving; more than a third reported over half an hour. Drafting documents 24 minutes, creating presentations 19, scheduling meetings 9. 82 per cent said they would not want to return to pre-Copilot conditions; satisfaction 7.7 and recommendation 8.2 out of 10; 85 per cent agreed it provided good value; 63 per cent believed their productivity would decline without it. The conclusions state it was not possible to identify how the saved time was spent.
- What it supports
- That a very large public sector deployment achieved high adoption and strongly positive user sentiment, and that accessibility benefits for disabled and neurodivergent users were a consistent theme.
- What it does not support
- Any measured productivity effect. The headline is a self-reported estimate, bucketed, with its top band capped by the analysts, and with no control group, no baseline task timing and no measure of output quality or decision quality. The report itself flags inconsistent user experience across departments and a festive-period disruption.
Compiled review#
Indeed Hiring Lab (2025). AI at Work Report 2025: How GenAI is Rewiring the DNA of Jobs
Indeed Hiring Lab, September 2025
- Method
- Assessment of almost 2,900 individual work skills commonly found in US Indeed job postings, each rated on a five-point scale for problem-solving demand and physical necessity, then classified by how far current generative AI could transform the skill. Results are weighted to skills as they appear in US postings; the postings window is not given in the summary fetched.
- Finding
- 46% of skills in a typical US job posting fall into the hybrid transformation or full transformation categories, but less than 1% of skills are fully transformable today. In software development 81% of listed skills fall into hybrid transformation; in nursing 68% fall into minimal transformation. 26% of jobs posted on Indeed could be highly transformed by GenAI and 54% moderately transformed.
- What it supports
- That the exposure of advertised skills to generative AI is wide but shallow, and that near-total substitution of a skill is rare on this method.
- What it does not support
- Transformability ratings are analyst judgements of capability, not observed adoption or job loss. Job postings measure employer demand text, not work done. Indeed is a job-advertising business.
Peer-reviewed#
Magesh, V., Surani, F., Dahl, M., Suzgun, M., Manning, C. D. and Ho, D. E. (2025). Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools
Journal of Empirical Legal Studies, 22(2), 216-242. DOI 10.1111/jels.12413
- Method
- First preregistered empirical evaluation of retrieval-augmented legal research tools. Over 200 handwritten legal queries across four categories, preregistered with the Open Science Foundation in March 2024, run against Lexis+ AI, Westlaw AI-Assisted Research, Ask Practical Law AI and GPT-4, graded on whether responses were correct and grounded in the sources cited.
- Finding
- The three commercial legal tools each hallucinated between 17 and 33 per cent of the time. Lexis+ AI was accurate on 65 per cent of queries and incomplete on 18 per cent; Westlaw AI-Assisted Research was accurate 42 per cent of the time with a hallucination in one-third of responses; Ask Practical Law AI was incomplete on 62 per cent of queries. Westlaw produced the longest answers, averaging 350 words against 219 and 175, which the authors link to both its higher hallucination rate and the verification burden it imposes.
- What it supports
- That retrieval-augmented generation reduces hallucination relative to a general chatbot without eliminating it, and that provider claims of hallucination-free citation were overstated. Longer generated answers carry more falsifiable propositions and require more checking.
- What it does not support
- Current performance of any named product. These are specific versions tested in 2024 and providers update continuously. It also does not measure legal outcomes, only response accuracy against expert grading.
Institutional survey#
Arzilli, F., Lynch, C. S. and Page, L. (2026). An Evaluation of DWP's Microsoft 365 Copilot Trial
Department for Work and Pensions, published 29 January 2026
- Method
- Mixed-methods post-implementation evaluation of a trial running October 2024 to March 2025 with 3,549 licensed staff. Survey of users (1,716 responses) and of a random stratified comparison group of non-users (2,535 responses from 9,300 sampled), 19 qualitative interviews, and seemingly unrelated regression controlling for demographic, occupational and AI-keenness variables. No baseline; licences allocated first come, first served.
- Finding
- Estimated saving of 19 minutes a day across eight routine tasks, statistically significant across all specifications, with the largest task effects on searching for information (26 minutes), writing emails (25) and summarising (24). 90 per cent of users said it saved time. Job satisfaction rose 0.56 points and perceived work quality 0.49 points on seven-point scales; 73 per cent reported better quality outputs and 65 per cent felt more fulfilled.
- What it supports
- That a regression-based estimate on a large departmental sample, controlling for observable differences, still finds a positive and significant self-reported effect on efficiency, satisfaction and perceived quality.
- What it does not support
- A measured time saving. The evaluation's own limitations chapter names the absence of baseline data, post-treatment bias, self-selection towards AI enthusiasts through first-come first-served allocation which it says may lead to overestimation, non-response bias and acquiescence bias on the time question. Nothing about decision quality or citizen outcomes was measured.
Peer-reviewed#
Bean, A. M., Payne, R. E., Parsons, G., Kirk, H. R., Ciro, J., Mosquera-Gomez, R., Hincapie M, S., Ekanayaka, A. S., Tarassenko, L., Rocher, L. and Mahdi, A. (2026). Reliability of LLMs as medical assistants for the general public: a randomized preregistered study
Nature Medicine 32(2), 609 to 615, published 9 February 2026. A Publisher Correction later fixed the y-axis labels of Figure 2a and nothing else. Read at source 30 September 2026
- Method
- Randomised, preregistered study of 1,298 UK adults given ten medical scenarios written by doctors. Participants used GPT-4o, Llama 3 or Command R+, or any method they would normally use at home (the control group), to identify the condition and decide what to do.
- Finding
- Tested alone, the models identified conditions in 94.9 per cent of cases and the right disposition in 56.3 per cent on average. Participants using the same models identified relevant conditions in fewer than 34.5 per cent of cases and the right disposition in fewer than 44.2 per cent, both no better than the control group. In 16 of 30 sampled interactions the first message held only partial information, and the models suggested 2.21 conditions per interaction of which 34.0 per cent were correct.
- What it supports
- That in this design a strong model score did not carry through to lay users, and that the gap sat in the interaction.
- What it does not support
- That any current model or product fails the same way, or how people behave with real symptoms, stress and urgency: the scenarios were vignettes and the authors call their figures a lower bound for newer models. Not a comparison with a search engine or a pharmacist.
Institutional modelling#
Board of Governors of the Federal Reserve System, Federal Deposit Insurance Corporation and Office of the Comptroller of the Currency (2026). Supervisory Guidance on Model Risk Management (SR 26-2)
Federal Reserve supervisory letter SR 26-2, 17 April 2026, 12 pages. Supersedes SR 11-7 (4 April 2011) and SR 21-8 (9 April 2021)
- Method
- Interagency supervisory guidance for United States banking organisations, most relevant to those above $30 billion in total assets. Replaces the 2011 model risk management framework after fifteen years of supervisory experience.
- Finding
- The definition of a model is narrowed to 'a complex quantitative method, system, or approach that applies statistical, economic, or financial theories to process input data into quantitative estimates', expressly excluding simple spreadsheet arithmetic, deterministic rule-based processes and software with no such theory underpinning it. Footnote 3 states that generative and agentic AI models 'are novel and rapidly evolving' and 'are not within the scope of this guidance', while the principles do apply to traditional quantitative models and to non-generative, non-agentic AI. Effective challenge is retained and defined as critical analysis by objective experts with expertise, independence and the organisational standing to force change. The guidance sets no enforceable standards and says non-compliance will not itself draw supervisory criticism.
- What it supports
- That the most mature oversight regime any profession has for machine-produced numbers has, on its own initiative and in 2026, placed generative AI outside its scope. Anyone citing model risk management as the ready-made precedent for governing generative AI in finance is citing a document that declines the job.
- What it does not support
- That generative AI in banks is ungoverned. The same footnote directs firms to their own risk management and governance practices for tools outside scope, and other supervisory expectations, consumer protection law and third-party risk guidance still apply. It is United States banking supervision only, and guidance rather than rule.
Compiled review#
Charlotin, D. (2026). AI Hallucination Cases database
damiencharlotin.com, updated daily. Figures read from the update of 27 August 2026
- Method
- Curated database of legal decisions worldwide in which a court or tribunal has explicitly found or implied that a party relied on hallucinated material. Excludes mere allegations, with a small stated exception. Coverage begins in the second quarter of 2023.
- Finding
- 1,963 cases identified as at 27 August 2026. By jurisdiction: United States 1,345, Canada 214, Australia 98, United Kingdom 62, Israel 57, with more than thirty other countries represented. By party responsible: self-represented litigants 1,127, lawyers 784, judges 29, expert witnesses 15. By nature: fabricated material 1,634, misrepresented authority 816, false quotations 528, outdated advice 33.
- What it supports
- That fabricated legal authority reaching courts is a documented, dated, worldwide phenomenon rather than an anecdote, and that self-represented litigants account for more recorded instances than lawyers do.
- What it does not support
- The true rate. The database counts decisions where a court addressed the point, so instances nobody noticed, or resolved without a written decision, are invisible by construction; its author states this. It is a curated compilation rather than a sampled study, so it cannot support a denominator or a trend rate.
Peer-reviewed#
Derksen, C. and colleagues (Queen Mary University of London) (2026). Use of AI scribes in UK primary care: a survey of general practitioners
npj Digital Medicine, 16 May 2026. Read at source 26 September 2026
- Method
- Cross-sectional online survey of 598 UK general practitioners, September to October 2025, recruited through a market research company.
- Finding
- About 40 per cent of respondents were using AI scribes and a further 23 per cent had used them; users applied them to a mean of 60 per cent of consultations (range 5 to 100). Over 75 per cent endorsed timeliness benefits such as real-time documentation. Over 60 per cent agreed there are risks of inaccuracies, errors, misinterpretations and associated medicolegal threats. Adoption was higher among men, private practitioners, more experienced GPs and GP trainers.
- What it supports
- That by late 2025 a substantial share of UK GPs had adopted ambient documentation tools, and that the same population that adopted them largely agreed they carry a risk of error, so use and the expectation of checking arrive together.
- What it does not support
- Whether GPs check the notes, or how often the notes are wrong; the survey measured use and attitudes, not accuracy or behaviour. The sample over-represents younger and male GPs relative to the workforce, so the proportions are indicative rather than national.
Peer-reviewed#
Education Endowment Foundation and National Foundation for Educational Research (2026). AI tool reduced primary teachers' lesson planning time by a quarter, but views on the tool were mixed
EEF press release on its trial of Oak National Academy's Aila, 6 October 2026. Read at source 9 October 2026; the evaluation report was not read
- Method
- Randomised controlled trial commissioned by the EEF and carried out by NFER: 464 Key Stage 2 teachers in 108 primary schools over ten weeks in autumn 2025. One group was encouraged to plan with Aila; the comparison group was told not to use Aila and was free to use other AI tools. Use was measured through weekly teacher diaries, interviews and school visits.
- Finding
- Teachers using Aila spent an average of two and a half hours a week planning against three hours and 19 minutes, a saving of around 49 minutes (24 per cent). NFER's Ben Styles: 'Lessons planned with the assistance of Aila were found not to be of lower quality'. 81 per cent of the Aila group used it at least once and use declined over the ten weeks; fewer than one in five felt it meaningfully influenced their teaching and planning. 'New teachers and teachers with less confidence in their subject knowledge were more likely to experience reduced lesson planning workload.'
- What it supports
- That a specialist AI planning tool saved primary teachers time against business as usual, other AI tools included, without a measured loss of lesson quality, and that newer teachers saw the larger effect on workload.
- What it does not support
- What newer teachers learn about planning when the tool plans with them: the trial did not measure it. How quality was assessed is not on the page read, and the grade here follows the EEF's earlier evaluation record and not a reading of this report. Oak National Academy, which makes Aila, is quoted in the release.
Compiled review#
Kissin, E. (Financial Times) (2026). Junior consultants called back to office as AI increases need for human skills
Financial Times, 27 August 2026
- Method
- News reporting. On-the-record interviews with named executives at EY, KPMG, Accenture and Azets, with reference to policies at Deloitte, PwC, BCG, Microsoft and JPMorgan. Not a study, and no measurement of skill or outcome.
- Finding
- Consulting leaders report that AI has raised the value of interpersonal skills and are considering requiring junior staff in the office more often to develop them. EY's UK head of consulting, Sayeh Ghanbari, is quoted saying firms will "have to reduce flexibility, but in order to help the human skills", and that firms dropped training in empathy, storytelling and leadership during the remote-working period while prioritising AI and technical skills. KPMG describes reinventing in-person training; BCG is expanding office social activities; Azets has lowered its degree requirement and is encouraging four days a week. Deloitte and PwC began extra coaching for their youngest UK recruits in 2023 after finding weaker teamwork and communication than earlier cohorts.
- What it supports
- That the apprenticeship-erosion argument is now being acted on by the largest professional services firms, named and on the record. It is strong evidence of institutional belief and of policy change.
- What it does not support
- That AI caused the deficit, or that office attendance repairs it. No measurement appears anywhere in the reporting, the cohort effects described in 2023 are attributed to pandemic lockdowns rather than to AI, and EY as a firm restated its existing flexibility policy alongside its executive's comments. Testimony from interested parties is not evidence of a mechanism.
Peer-reviewed#
Mass General Brigham and Harvard Medical School researchers (research letter) (2026). Clinician use of a generative AI clinical reference tool: more than 270 million queries
JAMA, published 15 October 2026. Read through Healthcare IT News (Andrea Fox, 19 October 2026); the paper itself was not read
- Method
- Analysis of more than 270 million queries to the OpenEvidence tool from about 950,000 clinicians over two and a half years, with a stratified sample of queries classified by a model rather than by people. Read at second hand.
- Finding
- As reported: treatment and management questions were 33.1 per cent of physician queries and diagnostic questions 13.9 per cent; monthly active clinical users rose from 355 in January 2024 to 703,265 in June 2026. The authors wrote that 'many of the tasks used to evaluate clinical AI may not match the tasks for which physicians actually use it' and that 'Evaluation and oversight frameworks should be weighted accordingly'.
- What it supports
- That clinicians using one tool at scale ask it mostly about treatment and management, the decisions current evaluations test least, on the authors' account.
- What it does not support
- Anything about the quality of the answers or of the resulting care: the study counts queries, classified by a model, and was read through a news report.
Compiled review#
National Transportation Safety Board (2026). Highway Investigation Report HIR-26-02, Ford BlueCruise collisions
National Transportation Safety Board, 31 March 2026
- Method
- Investigation of two fatal crashes in which Ford BlueCruise-equipped vehicles struck stationary vehicles at highway speed: San Antonio, 24 February 2024, and Philadelphia, 3 March 2024.
- Finding
- Overreliance on the partial automation system appears in the probable cause for both crashes, not merely in discussion: distraction 'stemming from overreliance on the vehicle's hands-free partial automation system' in San Antonio, and 'overreliance on and misuse of' it in Philadelphia. The board recommended that driver monitoring systems detect and warn about 'accumulated short distractions' over a prolonged period.
- What it supports
- That an investigator placed automation over-reliance in the causal chain rather than treating it as context, and that the monitoring system was defeated by ordinary attention behaving ordinarily rather than by anyone circumventing it.
- What it does not support
- Anything about knowledge work. Driving is a continuous manual-control task with a monitoring system watching the human, which is not the shape of AI-assisted professional judgement.
Argued perspective#
Parsons, A. S. and Rodman, A. (2026). Risk, Accountability, and the Clinician's Role in the Artificial Intelligence Era
Annals of Internal Medicine, 2026, doi 10.7326/ANNALS-26-02478. The journal page could not be opened; read through the University of Virginia Health release of 12 October 2026 and its syndication at News-Medical on 14 October 2026, both read at source 16 October 2026
- Method
- A commentary by two physicians, at the University of Virginia School of Medicine and Harvard Medical School, in a peer-reviewed journal, reporting no new data.
- Finding
- The clinician's lasting role, they contend, is accountability: 'owning the risk of acting under uncertainty and answering for the decision that follows.' 'A clinician who knows they will answer for the result weighs options differently than a system with nothing to lose.' 'Deviating from the technically correct answer is often good medicine, and someone must answer for that deviation.' AI 'cannot take ownership of what happens next'.
- What it supports
- That two clinicians in a reviewed venue locate what stays with the doctor in accountability for decisions under uncertainty rather than in diagnostic accuracy, where they accept AI may match or exceed doctors.
- What it does not support
- That clinicians in fact weigh options differently when accountable, which is argued and not measured. Read through a press release, not the article.
Argued perspective#
Solicitors Regulation Authority (2026). SRA seeks views on updates to solicitor competence framework
SRA press release, 8 October 2026, with the linked consultation page. Read at source 9 October 2026
- Method
- A regulator's consultation on proposed updates to the Statement of Solicitor Competence, introduced in April 2015, and to the functioning legal knowledge that candidates for the Solicitors Qualifying Examination must understand and apply. Open from 8 October to 3 December 2026.
- Finding
- 'Areas where the Statement of Solicitor Competence is potentially to be updated include the use of technology and AI, supervision, health and wellbeing and professional conduct and ethics.' The consultation page proposes 'a separate section setting out the competences required for effective supervision of others'. Final versions are due in April 2027 and would take effect in September 2027.
- What it supports
- That the regulator of solicitors in England and Wales proposes to write the use of AI and the supervision of others into its definition of a competent solicitor.
- What it does not support
- Any decision: these are proposals out for consultation. Nothing about how firms train their juniors today.
Self-selected survey#
The Clearing (2026). AI Fatigue in 2026: The State of the Engineer
The Clearing, April 2026. A commercial site for AI-fatigued engineers, which forecasts in the same report that AI boundary and wellness tools will become a product category. Read in full at source 11 September 2026 and re-read 13 September 2026
- Method
- 2,147 respondents, collected through The Clearing's own AI Fatigue Quiz between January and March 2026, who then voluntarily completed an optional demographic supplement. Recruited through the site's newsletter, r/cscareerquestions, r/ExperiencedDevs, r/programming and Twitter. Cross-sectional, one wave, no control group, no randomisation and no measured skill: every figure is self-reported perception. The report's own FAQ sets all of this out without being asked, to its credit, and its own sentence is that results 'reflect the experiences of engineers who found and chose to participate in the survey'. The report asks to be cited, in a box headed 'For Journalists and Researchers': 'This report is designed to be cited.'
- Finding
- 63 per cent 'report measurable decline in at least one core skill they had before adopting AI tools'; 71 per cent agree with 'I often feel like a middleman between AI output and actual results'; 67 per cent spend the majority of coding time reviewing AI output; 58 per cent 'cannot fully explain the code they shipped last month without AI assistance'; 44 per cent have seriously considered leaving their role because of how AI changed the work, while 68 per cent would choose the career again. By declining skill area, debugging from first principles leads at 58 per cent. The report names the pattern 'the Competence Illusion': 'when AI tools make you productive without making you more competent. You ship more. You understand less.'
- What it supports
- That a population of engineers describing themselves this way exists and is large enough to fill a survey, and that the vocabulary they reach for is loss of authorship rather than fear of replacement. The word for it, the middleman problem, is theirs and is used here as theirs.
- What it does not support
- Any rate. The frame selects on the outcome: respondents arrived by taking a quiz about AI fatigue, so the proportion reporting AI fatigue cannot be read as a prevalence. The word 'measurable' in 'measurable decline' does no work, because nothing in the report measures a skill; all seven skill-area figures are perceptions. And the intervention table prints effect sizes with confidence labels, 'no-AI coding blocks -2.1 points, Evidence Strength: Strong' and 'taking a break from AI tools entirely -2.8 points, Very Strong', on a cross-sectional self-selected survey with no control arm, which presents a correlation as a measured effect. The report also contradicts itself on one date, placing the r/programming language-model ban in the 2025 row of its own timeline and in April 2026 in the prose beneath it.
Advisory instrument#
The Law Society of England and Wales (2026). Using artificial intelligence in legal practice (practice note)
The Law Society of England and Wales, practice note, 21 October 2026. The note's page would not load on 21 to 23 October 2026; read through Artificial Lawyer (21 October), Legal Futures and City AM (22 October). Every quotation here is as those outlets gave it
- Method
- Guidance from the representative body for solicitors in England and Wales on choosing, supervising, disclosing and charging for AI in legal work. It is not a rule of the regulator and the Law Society said it knew of no positive duty to use AI.
- Finding
- As reported: 'AI is not a legal entity'; lawyers 'cannot transfer professional responsibility' to an AI tool; 'professional duties and obligations apply in the same way whether AI or other technologies were used'; AI is 'no substitute for junior lawyers' and must not 'replace opportunities' for juniors to learn to 'identify errors, omissions and unsupported propositions'; 'You should apply an appropriate level of human involvement and safeguards to mitigate risk'; 'it may be helpful to treat AI outputs with the same level of professional scepticism and independent judgement' as 'work produced by a trainee or junior colleague'; firms 'should not bill for the time that a task would have taken had AI not been used'. President Brett Dixon: 'Over-reliance on AI systems may result in a catastrophic gap of future legal skills and knowledge.'
- What it supports
- That the solicitors' representative body in England and Wales has stated, in guidance, that responsibility for AI-assisted work stays with the solicitor, that AI outputs need human checking at the standard applied to a junior's work, and that juniors' development must be protected.
- What it does not support
- Anything about practice: the note binds no one, the SRA has issued no equivalent rule, and the note was not read at source, so its wording beyond the quoted fragments is not established here. It does not say how a firm would measure a skills gap.
Compiled review#
United States Court of Appeals for the Fifth Circuit (Smith, Haynes and Engelhardt JJ) (2026). Jackson Federation of Teachers v. Fitch, No. 25-60496
Fifth Circuit, published opinion, 22 October 2026; coverage by Law News (UK), 27 October, and Bloomberg Law, 22 October. Read at source 30 October 2026
- Method
- An appellate court's review of a district court's orders and record, vacating an injunction and ordering reassignment. Graded as the court's compiled review of a record, the grade this research gave the Ayinde judgment; the underlying dispute is outside this research's lane and is not assessed.
- Finding
- The district court's orders contained false quotations purporting to be from the statute, plaintiffs who were not parties, declarants who 'do not appear in the record at all and may not even exist', and a cited case that 'does not even exist; it is an apparent AI hallucination'. The proceedings 'were a botch job, to say the least'; the judge 'carelessly exercised his Article III power by issuing sloppy edicts based on falsehoods and fabrications'; the court refers to the district court 'or perhaps, being charitable, its AI agent'. The judge had called the errors clerical and attributed them to a law clerk's use of AI. Reassignment, which the court calls extraordinary, was ordered because 'the public esteem for the judiciary would be diminished by sending this case back to the same judge.'
- What it supports
- That an appellate court has removed a case from a judge over AI-fabricated material in his orders, and has treated the delegation of drafting to a clerk's AI as the judge's own failure.
- What it does not support
- Anything about the frequency of AI-tainted judicial orders, or about courts outside the United States.
Where this evidence is used
The research that rests most on this section:
- Capability debt
- How will AI change law?
- Will human judgement protect lawyers from AI?
- How will AI change accounting and audit?
- How will AI change human resources?
- What happens to medical and legal training?
- What are the risks of using AI at work?
- What It Would Take to Leave
The other sections
- Judgement, thinking and offloading 79
- Learning, practice and expertise 92
- Human and AI collaboration 44
- Empathy, creativity and what stays human 36
- Work, jobs and the labour market 94
- The seven human capabilities 38
- The institutional record 180
- Frontline, clinical and physical work 24
- The international picture 29
The whole base is also machine-readable at evidence.json under CC BY 4.0.
Reference · SS-2026-451 · How I use AI here
Cite this page
Hirji, R. (2026). Professions and sectors. The SuperSkills evidence base, SS-2026-451. https://thesuperskills.com/research/evidence/professions-and-sectors. Last reviewed 3 November 2026.
An evidence review by Rahim Hirji, not peer-reviewed research. For a material claim, cite the underlying study as well; every study here carries its own permanent link.
How citations and IDs work