AI incident

The event is usually a wrong output that was acted on, rather than a system falling over. Nothing was unavailable, nothing alerted, and the harm came from a plausible answer being trusted. That is why detection so often comes from a person noticing rather than from monitoring, and why the record depends on somebody deciding this was worth writing down.

Near misses carry most of the value and are the ones least likely to be recorded. A wrong output caught by a reviewer before it went anywhere tells you the same thing about the system as one that got through, at no cost. It also feels like nothing happened, which is precisely why it goes unlogged, and an organisation that records only the expensive half learns from a fraction of what it sees.

Existing machinery covers some of it, which is worth knowing before building anything new. Where a platform records what its agents did, the trail needed to reconstruct an event may already exist. An enterprise agent platform in this guide states that outputs in regulated settings need human oversight and audit trails, which is that same requirement arriving as a product caveat.

The recording question is now being asked from outside rather than only internally. Compliance platforms in this guide carry risk and evidence machinery alongside control monitoring, and the roster note for this term observes that some regimes now expect these events to be recorded and reported. What any particular regime requires is its own subject and changes; that the question is being asked is the durable part.

• A platform stating that outputs in regulated settings need human oversight and audit trails, which is the record you would need if something later went wrong.

  Sierra

• Documentation warning that important outputs should be reviewed and verified, which describes the check whose absence turns a wrong answer into an event.

  ChatGPT

• A compliance platform carrying risk and continuous evidence alongside control monitoring, which is where the record of such an event tends to live.

  Vanta

What counts as one?

An occasion where the system caused harm or came close to it, including the near miss. Drawing the line narrowly produces a clean and useless log; drawing it broadly produces noise. The workable test is usually whether anybody would want to know this happened, which sorts most cases without needing a definition.

Who should hear about it?

Whoever owns the system, and whoever owns the arrangement across systems, because the pattern across events is more informative than any single one. A record that reaches only the team that caused it produces local fixes and no learning anywhere else.

How is this different from an outage?

An outage announces itself and ends when service returns. This kind is often discovered after the fact, has no clean start, and is not resolved by a restart. That difference is why an existing process usually needs an explicit route for it rather than being assumed to cover it.

• A wrong output that was acted on, more often than a system falling over.
• Near misses count, carry most of the value, and are the least likely to be recorded.
• Detection usually comes from a person noticing, not from monitoring.
• An outage process fits it badly, so it needs an explicit route in.

• 

  Sierra

  Human oversight and audit trails named for regulated settings.

• 

  ChatGPT

  Review and verification of important outputs, the check whose absence matters.

• 

  Vanta

  Risk and continuous evidence, where such a record tends to live.