Safety and governance

The work of making AI systems behave predictably and within bounds, covering both everyday failures and longer-term concerns.

The problem of getting a system to pursue what people actually intended rather than the literal instruction it was given.

Rules and filters wrapped around a model to keep its output within bounds: refusing harmful requests, staying on topic, not revealing private data.

Deliberately attacking an AI system before its users do: probing for ways to make it leak data, produce harmful content or ignore its instructions.

A request crafted to get a model to ignore its own rules, and the reason guardrails are treated as a control rather than a guarantee.

An organisation's stated commitments on fairness, safety and accountability in how it builds and buys AI, and the practices meant to deliver them.

The rules and roles an organisation sets for how AI may be used, who approves it and who answers for what it does.

The written document telling staff what they may and may not do with AI tools, usually the cheapest first control to put in place.

The specific ways an AI use could cause harm or loss, assessed case by case rather than treated as one general worry.

How far you can tell why a system produced a particular answer, which matters most where a decision has to be justified to someone.

Watching a live system's outputs over time so quality problems are noticed from evidence rather than from complaints.

Noticing when results get worse because the world or the inputs have changed, even though nothing about the system was altered.

The whole path a model takes in your organisation, from selection and testing through live use to eventual replacement.

Filtering what goes into or comes out of a system against a policy, the control behind a product refusing certain requests.

Marking generated content so it can later be recognised as machine-made, more reliable in some formats than in others.

A record travelling with a file describing how it was made and edited, aimed at showing where an image or document came from.

Constraining what a model may produce while it produces it, rather than checking and discarding the answer afterwards.

Structured tests that measure whether an AI system actually does its job, and the honest answer to how you would know if it stopped.

Using one model to score another's output at scale, cheaper than human review and needing its own checking before it is trusted.

An occasion where an AI system causes or nearly causes harm, which some regimes now expect to be recorded and reported.

Systematic unfairness in a system's outputs, learnt from its training data or its design, showing up as worse results for some groups.

A model declining to answer, sometimes correctly and sometimes not, and a common source of complaints when the boundary is drawn poorly.