The S Curve
Tue, 13 OctMelbourne · Singapore · New York
Book a call
Insight

Who keeps an AI incident register, and what goes in it

A blank incident ledger, a cracked glowing cube and an empty owner chair

A customer email goes out offering a discount the company does not sell. The draft came from the assistant on the desk. The person who hit send mentions it in the corridor, and by Friday the story has shrunk to a shrug.

The OECD draws a line a shrug erases. An AI incident, in its definition, is an event, circumstance or series of events where the development, use or malfunction of one or more AI systems directly or indirectly leads to harm: injury to health, disruption of critical infrastructure, a violation of human rights or of legal duties protecting fundamental, labour and intellectual property rights, or harm to property, communities or the environment. An AI hazard is the earlier case, where that same use or malfunction could plausibly lead to one of those harms.

What the framework says to record

The OECD's common reporting framework is written for governments, national authorities and other stakeholders. It holds 29 criteria in eight dimensions, drawn from 88 candidate fields across four existing resources. Seven are mandatory, because they carry what a reader needs in order to understand the incident, its impacts and its link to the system. The mandatory set is a title, a description, how the AI system is related to the incident, the submitter's role and affiliation, supporting material, a severity and a harm type.

Severity is a fixed list: hazard, serious hazard, incident, serious incident, disaster or other. Harm type covers physical, psychological, reputational, economic or property, environmental, public interest or critical infrastructure and human or fundamental rights. The link between system and incident can include more than one of direct cause, contributing factor, failure to act, overreliance and intentional misuse, human error or a failure to comply with the law. A submitter can add a link the list missed.

The other criteria are optional. They cover the date first known, the system name and version, who developed or deployed it, who was affected, the business function, whether the data or the model was involved, the task and the autonomy. Autonomy runs from a human doing the work, through a human in the loop, to a human out of the loop. I would refuse to save a row that leaves a mandatory field blank.

Who has to be able to write the row

The framework is a format for a report, and anyone should be able to file one if it meets a quality bar. It asks for the submitter's role, affiliation and relation to the incident: a user of the system, an affected stakeholder, someone at the organisation that developed or deployed it, or someone with first-hand or research knowledge. People at the organisation that developed or deployed the system are the ones invited to record the action taken. Those actions are prevention, mitigation, ceasing or remediation.

The person closest to the miss opens the row the same day. They hold the description and the supporting material. A named owner, close enough to stop the system, sets the severity, the harm type and the action. If only a risk team can create a row, the corridor conversation never becomes a record.

The report stops at the record. It leaves prevention and correction to a later decision, taken by the person who can change the workflow.

What to put on the page this week

  1. Open the log with the seven mandatory fields, plus the system name, the date and the action taken.
  2. Let the person who saw the miss create the row. Name one owner who sets severity and records whether the team prevented, mitigated, ceased or remediated.
  3. Mark the row as an incident or a hazard: harm that has happened, or harm that could plausibly happen.
  4. Read the open rows on a set day. Ask which ones still have no action and no owner.

An AI governance session is one place to try the log on a miss the team already knows. The row still has to be written by the people who were there.

From The S Curve

News and insights for innovation, digital transformation, future of work and L&D leaders.

Stay ahead of learning and development, corporate innovation and digital transformation news. Plus the future of work. For leaders in AU, NZ, HK, SG, the US, the UK and Canada.