← Research
Research

What counts as a serious AI incident?

The law now defines one for high-risk systems. Most organisations have no definition of their own, so the first incident is also the first argument about whether it was one.

Last reviewed: 15 September 2026

The EU AI Act defines a serious AI incident and requires providers of high-risk systems to report one. Most organisations have no definition of their own, so the first incident is also the first argument about whether it was one. A working definition, what happens when one occurs, which failures reach the board, and why a review that never disagrees is the incident nobody logs. An evidence review by Rahim Hirji; every figure resolves to a graded entry in the evidence base that says what it does not show.

Questions this page answersAll 811 questions this research covers

The law now says what a serious AI incident is, for high-risk systems, and most organisations have not said what one is for themselves. That matters because the first incident is also the first argument about whether it was one, conducted by people who would rather it was not, while the system that caused it is still running. A working definition, an owner and a stop button are the whole of incident readiness, and the three of them fit on a page. This page gives the legal floor, a definition an organisation can adopt above it, what happens in the first day, and which failures have to reach the board.

The answer, in one line

Under Article 3 of the EU AI Act, an incident or malfunction of an AI system that directly or indirectly leads to death or serious harm to health, a serious and irreversible disruption of critical infrastructure, an infringement of fundamental rights obligations, or serious harm to property or the environment.

Share as a card

Article 3 of the EU AI Act defines a serious incident as an incident or malfunctioning of an AI system that directly or indirectly leads to death or serious harm to a person's health, a serious and irreversible disruption of critical infrastructure, an infringement of obligations under Union law intended to protect fundamental rights, or serious harm to property or the environment. Article 73 requires providers of high-risk systems to report such incidents to the market surveillance authority of the member state where they occurred, within set periods, and deployers to inform the provider. Annex III lists the high-risk uses: recruitment, promotion, termination, credit, access to education, among others. That is the floor. It covers death, rights and infrastructure. It does not cover the credit decision that was wrong for a month, the shortlist that excluded everyone over fifty, or the agent that sent the wrong contract, unless somebody argues them across the line.

A definition an organisation can adopt#

A serious AI incident is any case in which a system the organisation has allowed to recommend or execute a consequential decision did so wrongly, for more than one person or for more than one day, without a human catching it. Three parts. Consequential, which the organisation's own allocation defines: the decisions it has marked as ones the machine may recommend or execute. Wrongly, by the organisation's own standard for that decision. And uncaught, which is the part that turns a fault into an incident, because a fault a human caught is the oversight working and a fault nobody caught is the oversight failing. Under that definition the incident is a property of the review, not of the model, which is where responsibility sits.

What happens when one occurs#

In the organisation that has prepared: the person named as able to stop the system stops it, and does not have to ask. The decisions the system took since the fault are identified from the log, which Article 12 requires for high-risk systems and the organisation should keep for any consequential one; the record a decision has to leave is at decision provenance. The people affected are told. Where the system is high-risk, the provider is notified so that it can meet its own reporting duty. And the owner of the decision the system was making, not the owner of the system, explains to the board why the organisation made that decision, which is the question a human in the loop exists to answer. In the organisation that has not prepared, the first hour is spent establishing whether anybody can turn it off.

Rules Before Tools put the incident playbook and a drill in the first-quarter list in August 2025, alongside naming the five failures the organisation most fears and who owns each. The drill is the part organisations skip and the part that finds out whether the stop button is connected to anything.

Which failures reach the board#

Four kinds. Any incident meeting the organisation's own definition, with the decision owner's account. Any meeting the legal one, with the report as filed. Any reversal or constraint of an AI deployment, and why, because deployment is not a ratchet and a board should know when management has stepped back. And, quarterly, the disagreement rate for every consequential system: how often the humans reviewing its output reached a different answer. That last one is the incident that never gets logged. Kim and colleagues found people agreeing with a half-accurate system 81 per cent of the time; a review whose disagreement rate has fallen to zero has stopped being a review, no fault has been recorded, and the next uncaught error is already in the pipeline.

What nobody has measured#

There is no public dataset of enterprise AI incidents under a consistent definition, and that absence is what makes the argument about whether something counts so easy to have. The legal definition is new and its reporting periods have not yet produced a body of cases. The organisational definition above is a proposal, drawn from the model risk regime's treatment of uncaught errors, and its three parts are chosen so that the organisation's own allocation decides the scope rather than a lawyer after the event.

Key sources

On what oversight has to be able to do before it counts, meaningful human oversight. On the audit trail, how do you audit an AI-assisted decision. On what management reports to the board each quarter, what board oversight of AI looks like. On the allocation that defines the scope, AI leadership.

About this research#

Rahim Hirji is the author of SuperSkills (Kogan Page, 2026), keynote speaker on AI and human capability, and founder of The SuperSkills Intelligence Company. He has run, grown, bought and advised businesses with AI in them. Findings are attributed to the studies and statements that produced them and kept separate from the interpretation. This is a living reference, reviewed and updated as significant new evidence appears.

How this research works  ·  Reviewed quarterly  ·  Found an error? Tell me and it is corrected on the page.

Evidence review · SS-2026-251 · Graded against the published rubric

Cite this page

Hirji, R. (2026). What counts as a serious AI incident?. The SuperSkills evidence base, SS-2026-251. https://thesuperskills.com/research/what-counts-as-a-serious-ai-incident. Last reviewed 15 September 2026.

An evidence review by Rahim Hirji, not peer-reviewed research. For a material claim, cite the underlying study as well; every study here carries its own permanent link.

How citations and IDs work
Questions answered on this page

What counts as a serious AI incident?

Under Article 3 of the EU AI Act, an incident or malfunction of an AI system that directly or indirectly leads to death or serious harm to health, a serious and irreversible disruption of critical infrastructure, an infringement of fundamental rights obligations, or serious harm to property or the environment. Article 73 requires providers of high-risk systems to report such incidents to the market surveillance authority. That is the legal floor; an organisation needs its own definition above it, covering the consequential decisions its own systems take.

What happens when an AI incident occurs?

In an organisation that has prepared: the system is stopped by the person with authority to stop it, the decisions it took since the fault are identified from the log, the people affected are told, the provider is notified where the system is high-risk, and the owner of the decision the system was making explains to the board why the organisation made it. In an organisation that has not, the first hour is an argument about whether it counts.

Which AI failures must be reported to the board?

Any incident that meets the organisation's own definition; any that meets the legal one; any reversal of an AI deployment and why; and the disagreement rate for consequential systems every quarter, because a rate that has fallen to zero is the incident that never gets logged.

In this hub

Definitions

The terms this field uses, defined against their primary sources.

Ask the evidence
What does the evidence actually show?What should our board be asking about this?Where does Rahim disagree with the consensus?
Bring this into your organisation

If this describes something happening in your teams, say so.

Keynotes, board sessions and advisory work, drawing on research across more than 200 organisations in 30 countries. Tell me the room, the date and the shift you need. A reply within 24 hours.

Start a conversation

Topics and audiences  ·  All research

A definition, an owner and a stop button fit on a page. Writing that page with the executive team, and running the drill, is the engagement. Board advisory.

This argument is one a board usually meets for the first time in the room. There is the boards and leadership version, and the full range of topics and audiences.

Box of Amazing

Rahim’s free weekly letter on AI and human capability

If this was useful, the weekly letter is where the thinking happens first. Most of what ends up on this site starts there. Weekly essays on AI, capability and the future of work. Read by 25,000 people, every week since 2017. Free, and one click to stop.

Opens Substack to confirm. No pitch in it, unsubscribe in one click, and nobody follows up because you read something.