Glossary
Jailbreak
A jailbreak is a request crafted to make a model set aside the rules it was trained or instructed to follow, and its existence is why trained refusal is treated as a control rather than a guarantee.
In plain terms
The rules a model follows are not switches; they are text and training, competing with all the other text in front of it. Somebody who wants it to behave differently is arguing with those rules rather than breaking anything, and arguments are sometimes won. This page deliberately does not describe how, because the useful part for anybody running a system is what follows from it rather than what it looks like.
Why it matters
It changes what you may reasonably rely on. If a model's refusal is the only thing standing between your system and something you would have to explain, then that protection can be argued away by somebody sufficiently motivated, and you will not be told when it happens. That is not an argument for distrusting these tools. It is an argument for putting anything that must hold somewhere an argument cannot reach it.
How it works
The structural reason is worth understanding and gives nothing away. A model's rules arrive as instructions and as trained disposition, and both are weighed alongside everything else it is reading rather than enforced above it. There is no separate layer inside the model that holds a rule the rest of the system cannot touch, which is why refusal is a strong tendency rather than a boundary.
Providers treat each one they learn about as a defect and address it, and the work is genuinely effective and genuinely continuous. What that means for a buyer is that a model's resistance is a moving property rather than a fixed one: it improves over versions, and a new version can also behave differently in ways nobody predicted. It is not a thing you verify once.
The case worth designing against is the one where nobody typed anything. Text inside a document, a web page or an email that a system was asked to process can carry instructions of its own, and a system reading it may act on them. Nobody hostile needs access to your assistant for this; they need only to have written something your assistant later reads.
The conclusion follows directly and is not dramatic. Trained refusal is one layer, useful and improving, and it should never be the layer a serious consequence rests on. What a system is permitted to reach, what it may do without a person, and what is checked after it answers are all decided by you and are not subject to persuasion.
Where the instruction came from, and why it changes the design
Seen in the wild
Notice where an assistant refuses something entirely harmless, which is the same trained caution seen from its other failure direction and a reminder that it is a tendency rather than a rule.
ClaudePut a check outside the model in front of the one action that must never happen, so the rule holds regardless of what the model was persuaded of.
n8nRestrict what an automated step is permitted to reach, so the worst case is bounded by permissions rather than by a refusal.
Make
Common misconceptions
People assume
A good enough model will not be susceptible.
In fact
Resistance improves substantially with each generation and no provider claims the problem is closed. Treating it as solved by capability puts the weight of a real consequence on a property nobody has been able to guarantee. The safer reading is that it will keep improving and should keep not being your only control.
People assume
It requires somebody with access to our assistant.
In fact
Not for the indirect case, which is the one worth designing against. Instructions can arrive inside material your system was asked to read, from anybody who could put text in front of it. That is why validating what a system does, rather than trusting what it was told, matters more than restricting who may talk to it.
Telling them apart
Jailbreak vs Guardrails
Jailbreak
The phenomenon: a model's own rules being argued aside, silently, with no signal that it happened.
The remedy: checks in code that inspect and refuse, unaffected by anything the model was told or read.
The first is the reason the second exists. A rule you would have to defend in a meeting belongs in code, because code does not weigh arguments.
Questions
- What should we actually do about it?
- Stop relying on refusal for anything consequential. Limit what a system may reach, require a person before anything irreversible, check answers before they leave, and treat any material arriving from outside as untrusted input rather than as instructions. None of that depends on how resistant the model turns out to be.
- Should we test whether our own system is susceptible?
- Yes, as part of a red-team exercise on a system you own, with the findings turned into permanent checks. That is testing your own deployment, which is ordinary diligence. It is a different activity from probing somebody else's product, and the difference is whose system it is.
- What if somebody finds one in a product we use?
- Report it to the provider rather than publishing it, which is the same convention security work has used for decades and for the same reason. Providers generally have a route for this. Meanwhile, treat it as evidence about your own design: the question it raises is what would have happened if a consequence had rested on that refusal.
Key takeaways
- A model's rules are instructions and disposition weighed alongside everything else, not a boundary above it.
- Resistance improves with each generation and no provider claims the problem is closed.
- The indirect case matters most: instructions can arrive inside material your system was asked to read.
- Nothing consequential should rest on refusal alone, because you will not be told when it is argued away.
- Permissions, approvals and checks after the answer are decided by you and cannot be persuaded.
Last checked July 2026