Over-reliance on AI

Over-reliance is depending on a system beyond the point at which you could tell it was wrong. The definition turns on capacity rather than on quantity. That is the reason the question people usually ask, how much AI use is too much, has no answer. Someone who uses a model forty times a day and verifies competently is not over-relying. Someone who uses it once, on a judgement they have no way to check, is. The term is established in human-factors and human-computer interaction research and is not a SuperSkills coinage.

What it looks like at work, and what it does not#

In a workplace the pattern is recognisable by what the person can no longer do, not by how much they use the tool. An analyst approves a model-written summary of a contract she could not have drafted and has no way to check. A manager forwards an AI-generated report because it reads well. A team lead has never overridden the system and has never been asked to. Each is a case of acceptance without the capacity or the habit of verification. A colleague who drafts everything with AI and then checks every claim against the source is using the tool heavily and is not over-relying. If a question asks which description fits, the answer is the one about unverified acceptance, and an option built on frequency of use describes something else.

The measurement that makes it concrete#

Kim, Liao, Vorvoreanu, Ballard and Wortman Vaughan ran the cleanest available demonstration. They gave 404 participants eight yes-or-no medical questions and an AI system whose answers were correct on exactly half of them, in a pre-registered design. Participants with access to the system agreed with it 80.9 per cent of the time. Participants without access answered correctly 74.2 per cent of the time; participants with access answered correctly 63.9 per cent of the time.

Access to the machine made people more than ten points worse at a task they could otherwise do. That is the shape over-reliance takes when it is measured rather than described. It also explains why the phenomenon cannot be diagnosed from usage logs. Nothing in a usage log distinguishes the participant who was helped from the participant who was hurt.

Three named mechanisms underneath it#

Over-reliance is the outcome. The human-factors literature names the routes to it separately, and the distinction is worth holding because they need different responses.

• Automation bias. Accepting automated output without applying the scrutiny you would apply to the same claim from a person. Skitka, Mosier and Burdick split it into errors of commission, acting on a wrong recommendation, and errors of omission, missing what the system did not flag. Almost every review process is built to catch the first kind.
• Automation complacency. The decay of monitoring when a system is usually right. Molloy and Parasuraman showed detection of an automation failure falling sharply with time on task, which is a property of attention rather than of motivation.
• Disuse. Parasuraman and Riley named the opposite failure in 1997 and put it in the same family: rejecting or ignoring a system that would have helped, usually because its false-alarm rate has taught the operator to discount it. An organisation can hold both at once, over-relying where the tool is confident and ignoring it where it warns.

Why telling people to be careful does not work#

Parasuraman and Manzey's review found the effect present in experts as readily as in novices, resistant to training, and worse under time pressure and high workload. Dzindolet and colleagues found something more awkward in 2003: explaining to people why an automated aid might fail increased their reliance on it. Transparency about limitations restored trust rather than calibrating it.

Bucinca, Malaya and Gajos tested what does work. Cognitive forcing functions, interface changes that require the person to commit to a view before the system shows its answer, significantly reduced over-reliance compared with conventional explainable-AI designs. Participants rated those designs the least favourably of any they were shown. The intervention that works is the one users dislike, which makes it a governance problem rather than a design problem, and the same trade appears on the uncertainty page, where the hedging that improved accuracy also reduced intention to use.

Two 2026 preprints on designing the reliance down#

Two preprints from this autumn test the same idea from different ends: how much a person hands over depends partly on how the tool is set up. Both are version-one preprints without a journal reference when read, and neither is settled.

Maier, Schwabe, Schneider and Feuerriegel randomised 704 online participants to practise fraction arithmetic with an assistant that gave a full answer only when asked. A prompt that spelled out what asking would cost them cut requests for the complete answer (odds ratio 0.47) and raised accuracy on an unaided test afterwards (odds ratio 1.51). A points reward for using the assistant less moved neither measure. The confidence intervals on both effects touch or approach 1 and the reported p-values are one-sided, so each is a lead and no more. Access to the assistant did not measurably lower unaided accuracy against a control group without it, which is a reminder that a brief session on a simple task is a poor place to look for harm.

Chen, Koedinger and colleagues followed 153 students in a university proof course at Carnegie Mellon, randomised to early or later access to an AI tutor. Early access lifted homework by 2.56 points and left the next exam statistically unchanged. Once everyone had access, each extra 100 chatbot messages went with a 3.02-point lower score on the third midterm, and students with lower self-efficacy used the chatbot more. That second part is observational: students chose how much to use it, and the authors describe offloading as their interpretation, since they did not measure mental effort. The feedback interface that asked for a proof attempt before responding showed no reliable link to lower scores.

Read together with Bucinca and colleagues, the pattern is consistent with the forcing-function finding: the arrangements that ask the person to commit or reflect first are the ones with the better record. Neither preprint tests whether the effect survives months, and the Chen result describes who ends up using the open chatbot most, which an assignment experiment would have to confirm before anyone treats it as a cause.

The diagnostic that replaces counting hours#

Because over-reliance is defined by capacity to detect error, the useful question is not about frequency. Three questions get closer, and all three are answerable without instrumentation.

• Could you do this unaided, and when did you last try? Not whether you could learn to. Whether you could now. An answer of "probably" that has not been tested in a year is a no.
• What would make you reject this output? Decided before you see it. A criterion invented after reading the answer is a rationalisation of the answer.
• When did you last disagree with it? A process in which nobody ever overrides is indistinguishable from a process in which nobody is checking. The disagreement rate is the cheapest available instrument and almost nobody records it.

What the term does not cover#

Over-reliance describes a relationship between a person and a system at a moment. It says nothing about what repeated reliance does to the underlying ability over time, which is a separate question with separate evidence and is covered under deskilling and capability debt. Nor does it imply that reliance is wrong. Every professional relies on instruments they cannot personally validate. The claim is narrower: reliance without the capacity to detect failure is a different arrangement from reliance with it, and only one of the two is a control.

Related SuperSkills research#

On the mechanisms, automation bias, automation complacency and algorithm aversion. On the personal version, am I becoming dependent on AI and using AI without dependency. On the organisational version, why human in the loop is not a safeguard, who supervises work they cannot do and meaningful human oversight. On what to do about it, how to know when AI is wrong and when to override AI.

Key research and primary sources

• Kim, S. S. Y., Liao, Q. V., Vorvoreanu, M., Ballard, S. and Wortman Vaughan, J. (2024). "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust. FAccT 2024.
• Parasuraman, R. and Manzey, D. H. (2010). Complacency and Bias in Human Use of Automation: An Attentional Integration. Human Factors, 52(3).
• Skitka, L. J., Mosier, K. L. and Burdick, M. (1999). Does automation bias decision-making? International Journal of Human-Computer Studies, 51(5).
• Parasuraman, R. and Riley, V. (1997). Humans and Automation: Use, Misuse, Disuse, Abuse. Human Factors, 39(2).
• Molloy, R. and Parasuraman, R. (1996). Monitoring an Automated System for a Single Failure. Human Factors, 38(2).
• Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G. and Beck, H. P. (2003). The role of trust in automation reliance. International Journal of Human-Computer Studies, 58(6).
• Bucinca, Z., Malaya, M. B. and Gajos, K. Z. (2021). To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making. CSCW 2021.
• Maier, S., Schwabe, K., Schneider, M. and Feuerriegel, S. (2026). Designing Against Deskilling: Metacognitive Feedback Reduces Cognitive Offloading to LLM Assistants. arXiv preprint 2609.20143.
• Chen, E., Koedinger, K. and colleagues (2026). Supporting or Replacing Students' Reasoning? Cognitive Offloading in AI-Assisted Proof Learning. arXiv preprint 2610.31481.

About this research#

Rahim Hirji is the author of SuperSkills (Kogan Page, 2026), keynote speaker on AI and human capability, and founder of The SuperSkills Intelligence Company. Over-reliance, automation bias, automation complacency and disuse are established terms from human-factors research and belong to the researchers cited above. Nothing on this page is a SuperSkills coinage. Missed reps is his; capability debt is used here without any claim of first use. Both appear only as the separate question of what repeated reliance does over time.

How this research works  ·  Reviewed quarterly  ·  Found an error? Tell me and it is corrected on the page.