Glossary
Reinforcement learning (RL)
Reinforcement learning trains by trial and reward rather than by copying examples: the system tries something, receives a score, and shifts towards whatever scored well.
In plain terms
Learning by consequences instead of by instruction. Nobody shows the system the right answer; it attempts something, finds out how that went, and adjusts. That works where the right answer is hard to write down but easy to recognise, which describes a great deal of the world and very little of what fits neatly into a spreadsheet of examples.
Why it matters
It is how the stage that makes assistants feel helpful actually works, so it explains their manner more directly than anything else. It also has a failure everybody in business already recognises: a system optimising a score pursues the score, and where the score and the intention differ, the score wins. That is the same dynamic as a badly chosen performance target of your own, arriving now in something that pursues it without pausing to wonder whether it should.
How it works
The system acts, something scores the result, and the scoring pushes it towards what did well. Repeated enormously, that produces behaviour nobody demonstrated and nobody could easily have described, which is both the appeal and the reason you have to watch the outcome rather than the method.
Where the score comes from is the whole design. In a game it is the result. In the stage that shapes assistants, it comes from people ranking answers, which are then used to train a stand-in that can score at a scale no group of people could reach. Whatever that stand-in rewards becomes the behaviour you eventually meet in the product you are using.
The system will find ways to score well that nobody intended, and this is a documented pattern rather than an occasional surprise. Anything that raises the number without achieving the point of it will be found, because the search is exhaustive and indifferent to what you meant by the number.
It is usually combined with learning from examples rather than used alone. A model is first shown how the job is done and then refined by scoring, because starting from a reasonable attempt is far cheaper than discovering everything by trial. Most real systems are that combination.
The loop, and why it finds things nobody demonstrated
Seen in the wild
Use the rating controls in an assistant, which is a small contribution to exactly the kind of preference data this approach turns into a score.
ChatGPTPush back on an assistant twice about something you are wrong about and watch how readily it comes round, which is agreeableness that scored well.
ClaudeSet an automation to optimise something countable and watch what happens to the thing the count stood for, which is the same dynamic without any AI in it.
n8n
Common misconceptions
People assume
The system learns what we wanted.
In fact
It learns what scored well, and those differ whenever the score is an imperfect stand-in for the goal, which is nearly always. This is not a flaw in the method so much as a permanent design responsibility: whoever sets the score is specifying the behaviour, whether or not they realise it.
People assume
It is only used in games and robots.
In fact
That is where it was famous and it now shapes the assistants you use daily. The stage that turns a text continuer into something helpful works this way, which means an assistant's tone, its willingness to agree and its refusals were all produced by something scoring answers.
Telling them apart
Reinforcement learning vs RLHF
Reinforcement learning
The general approach: act, be scored, adjust. Used across games, robotics and much else, with the score coming from wherever the designer chose.
The specific application of it to assistants, where the score is derived from people ranking competing answers.
The second is the version that shaped the tools you use. This page is why it behaves the way it does, and the other is what was actually done.
Questions
- Why does this matter to a business reader?
- Because it explains assistant behaviour that otherwise looks arbitrary, and because its failure is one you already know. Any measure used as a target gets pursued rather than the thing it measured, and connecting a system that pursues targets tirelessly makes that sharper rather than new.
- Could we use it on our own systems?
- Rarely worth it directly. It needs a reliable way of scoring outcomes at scale, which most business processes do not have, and it is easy to get expensively wrong. Where you have a clean score and high volume, it is worth a specialist conversation; otherwise examples and rules do the job with far less risk.
- Is this how a model learns from my corrections?
- No. Nothing you type changes the model, and corrections in a conversation last only as long as that conversation. Ratings you give may be collected and used in a later training round of this kind, which affects a future version rather than the one in front of you.
Key takeaways
- It trains by trial and score rather than by copying examples.
- Whoever chooses the score is specifying the behaviour, whether or not they realise it.
- Systems reliably find ways to score well that nobody intended, because the search is indifferent to intent.
- It is usually combined with learning from examples rather than used alone.
- It is the approach behind the stage that gives assistants their manner.
Last checked July 2026