Glossary
AI safety
AI safety is the work of making these systems behave predictably and within bounds, covering both the everyday failures of tools in use today and longer-term concerns about far more capable ones.
In plain terms
Two conversations share a name. One is about systems being used right now: confidently wrong answers reaching customers, material disclosed to somebody who should not have it, tools talked into things by whatever they read. The other is about what would follow if far more capable systems arrive. Both involve serious people doing serious work, and only the first has any bearing on what your organisation ships this quarter.
Why it matters
Because the phrase has become associated with the long-horizon argument, a great many organisations file the entire subject under speculation and skip the half that applies directly to the tool they deployed last month. That half is not philosophical. It is testable, it has known remedies, and the remedies are ordinary engineering discipline of a kind your organisation already applies elsewhere. Recognising the two halves is what lets you take the near one seriously without having to hold a view on the far one.
How it works
The near half concerns things you can go and test. Systems producing confident answers with no basis, disclosing material the person asking should not see, being steered by instructions hidden in a document they were asked to summarise, or behaving differently for different groups of people. All four are observable, reproducible on your own material, and have known mitigations.
In a deployed system the safety mostly lives outside the model. What the assistant is permitted to reach, what is checked before an action is taken, which actions require a person, what is written down afterwards, and how somebody reports that it did something odd. A capable model in a badly arranged system is unsafe, and a modest model behind good arrangements is generally fine.
Some of it is done during training, and it is partly effective. The stage where people rank answers is where refusal behaviour and tone are installed, and it errs in both directions: harmless requests get declined, and determined effort still gets past it. Useful, worth having, and not a control you would want to be the only one.
The far half is a genuine argument among researchers rather than a settled forecast. Some hold that substantially more capable systems would present problems different in kind from today's, and that the work should begin well before such systems exist. Others hold that the present failures are the ones with evidence behind them and that attention spent on speculative scenarios is attention taken from testable harms. Both positions are held by people with standing.
What an organisation should do follows from the near half alone. Decide which actions need a person in front of them, record what the system did and on what basis, test with your own awkward cases rather than the vendor's, and give people an obvious route to report something strange. None of that requires you to take a view on the longer argument.
One word, and how much of it reaches your organisation
Seen in the wild
Notice where an assistant declines something entirely harmless, which is the same trained caution that blocks genuine misuse, erring in the other direction.
ClaudePut an approval step in front of the one action in a flow that cannot be undone, which is where most of the practical safety in any deployment actually sits.
n8nAsk a document tool something its sources do not cover and see whether it says so, which is a property you can test rather than an assurance you have to accept.
NotebookLM
Common misconceptions
People assume
It is all about far-future risk.
In fact
Most of the working discipline concerns systems already deployed: wrong answers with consequences, material reaching the wrong people, and tools being steered by their own inputs. Filing the whole subject under speculation is precisely how the applicable half gets skipped, and it is the applicable half that produces incidents this year.
People assume
The provider handles safety for us.
In fact
They handle their layer, which is real and stops at their boundary. What your assistant is allowed to reach, which actions run without a person, what happens to a bad answer once it leaves your building: none of that is theirs. The parts with consequences for you are the parts you configured.
People assume
A safer model makes a safer product.
In fact
It helps and it is one layer of several. A model with careful refusal behaviour, given write access to a payments system and no approval step, is an unsafe product. The consequences live in the permissions and the checks, which is why two products on the same model can differ enormously in how much damage they could do.
Telling them apart
AI safety vs Guardrails
AI safety
The whole discipline: what can go wrong, how you would find out, and what you do about it.
One family of mechanisms within it, namely checks and blocks applied around a system in operation.
Guardrails are something you install. Safety is the question of whether you installed the right ones and would notice if they stopped working.
AI safety vs Alignment
AI safety
The practical goal, including everything from a bad answer reaching a customer to who may approve a payment.
The narrower research problem of getting a system to pursue what was actually intended rather than a near miss of it.
The second is a research question and a real one. The first is what you are accountable for on Monday, and most of it is configuration rather than science.
Questions
- Is this about far-future risk or about today?
- Both, under one name, which is the source of most of the confusion. The near half concerns deployed systems and is testable on your own material. The far half is a live argument among researchers with reasonable people on each side. You can act fully on the first without holding any position on the second.
- What should our organisation actually do?
- Four things, none of them exotic. Decide which actions require a person before they happen. Record what the system did and what it based that on. Test with your own difficult cases rather than the vendor's demonstration. And make it obvious how somebody reports that it behaved strangely, then read those reports.
- Is safety not the provider's responsibility?
- Theirs covers what the model will and will not produce. Yours covers what your system may reach, what it may do unsupervised, and what happens to an answer once it leaves your organisation. Those are the parts with contractual and reputational consequences, and no provider can configure them for you.
- How do we test any of this ourselves?
- Collect the awkward cases: questions your documents do not answer, requests that should be refused, material containing something that reads like an instruction, and the operations that would be expensive to get wrong. Run all of them and record what happened. That set is worth more than any assurance and can be rerun after every change.
Key takeaways
- Two conversations share the name, and only the near one bears on what you ship this quarter.
- In a deployed system, most of the safety lives in permissions, checks and approvals rather than in the model.
- Training-stage caution errs in both directions and should never be the only control.
- The long-horizon part is a genuine argument among researchers rather than a settled forecast.
- Everything an organisation should do follows from the near half alone.
Tools that use this
- Claude
Trained caution is observable, including where it over-refuses.
- n8n
An approval step before the irreversible action, which is where safety lives.
- NotebookLM
Answering within supplied sources is a property you can test.
Last checked July 2026