If we only do one thing, what should it be?

Record which version produced each result. Pinning without a record still leaves you unable to explain a change when you eventually move, while recording without pinning at least makes an investigation possible. One field alongside the output is the whole of the work and it is what turns an impression into evidence.

Should we use the general name or a dated version?

The general name for exploration and a specific version for anything somebody relies on. The first keeps you current automatically and makes your behaviour a function of a release schedule you do not control; the second stays put until it is retired. The mistake is using the first for production without noticing you did.

Somebody says quality dropped last month. How do we check?

Compare the recorded versions across the period and re-run a sample of the old inputs. Without that record there is no honest answer, which is why these reports so often end unresolved and are absorbed as a general sense that the tool got worse. The record is what converts the conversation into a test.

How often does the thing behind a general name actually change?

Often enough that it is a real operational fact rather than a theoretical one, and on a schedule set by the provider. That is the point of the general name: it is meant to move. What matters is whether your system's behaviour depending on that movement was a decision somebody made or a default nobody examined.

Glossary

Model versioning

Model versioning records exactly which model produced a result, which is what makes it possible to answer why something that used to work stopped working.

In plain terms

Writing down exactly which version produced a result. It sounds like bookkeeping and it decides whether you can ever answer the sentence people actually say, which is that something used to work and now does not. Without the record, that complaint has no route to a cause and gets absorbed as a general feeling that quality dropped.

Why it matters

Because behaviour changes underneath you and the change is silent. A provider updates what a general name points at, output shifts, and somebody notices weeks later that results feel different, at which point the only honest answer without a recorded version is that nobody can tell. One stored field converts that into a comparison.

How it works

Providers offer both a general name and specific dated versions, and the choice between them is a real trade. The general name always points at something current, which means it improves without you doing anything and changes without telling you; a specific version behaves identically for as long as it exists and eventually stops existing. Neither is safe and only one of them surprises you.

The general name is a moving target, which is exactly why it is the default. It is convenient, it keeps you on improvements you would otherwise have to adopt deliberately, and it makes your system's behaviour a function of somebody else's release schedule. That is fine for exploration and awkward for anything whose output somebody depends on.

The record is worth more than the pinning for most teams. Storing which version produced each result costs one field and makes an investigation possible, whereas pinning without recording still leaves you unable to say what changed when you eventually move. If you do only one of the two, keep the record.

A version change invalidates the work that was done against the old one. Instructions were tuned to its habits, evaluations were run against its output, and thresholds were chosen from its behaviour, so moving is a re-testing exercise rather than a configuration change. Teams that treat it as a setting discover the difference through their users.

The same complaint, with and without a record

What makes this worth insisting on is how the left-hand column fails. It does not produce a wrong answer; it produces no answer, and an unresolvable complaint has a particular corrosive effect on a team's relationship with a tool. Somebody reports that summaries have got worse. Nobody can check. The reasonable positions are then that the person is imagining it or that the tool is quietly degrading, and neither can be tested, so both persist. Over a few months this hardens into a general sense that the feature is unreliable, which is expensive and may be entirely unfounded. The right-hand column costs one field written alongside each result, and it does not require pinning, changing providers or any architectural commitment. It simply means that the next time somebody says it used to be better, the conversation can end with a comparison instead of a shrug.

Seen in the wild

A routing service in front of many providers, where which model answers is a policy decision rather than a property of the code that calls it.
OpenRouter
An open-weight family you download and serve yourself, where the version you run changes only when you decide to change it.
Llama (Meta)
An assistant offering slower, more capable members and faster ones under one subscription, so which one answered is a real question about a result.
ChatGPT

Common misconceptions

People assume

Pinning a version makes us safe.

In fact

It makes you stable until that version is retired, which happens on somebody else's timetable. Pinning buys predictable behaviour and defers a migration rather than removing it, and a pinned version with no record of what changed at the eventual move is the worst of both arrangements.

People assume

It only matters for engineering.

In fact

It decides whether a business complaint can be investigated. When somebody reports that output has got worse, the recorded version is what turns an impression into a comparison, and its absence is why so many of those reports end in nobody being able to say either way.

People assume

Moving up a version is a configuration change.

In fact

It invalidates the tuning around it. Instructions were written against one system's habits, thresholds were chosen from its behaviour and evaluations were run on its output, so a move is a re-testing exercise. Treating it as a setting is how a quiet quality regression reaches users first.

Telling them apart

Model versioning vs Model lifecycle

Model versioning

Knowing exactly which one answered.

Model lifecycle

The whole arc from release to retirement.

Versioning is the record-keeping that makes the lifecycle manageable, which is why teams meet the second painfully when they skipped the first.

Questions

If we only do one thing, what should it be?: Record which version produced each result. Pinning without a record still leaves you unable to explain a change when you eventually move, while recording without pinning at least makes an investigation possible. One field alongside the output is the whole of the work and it is what turns an impression into evidence.
Should we use the general name or a dated version?: The general name for exploration and a specific version for anything somebody relies on. The first keeps you current automatically and makes your behaviour a function of a release schedule you do not control; the second stays put until it is retired. The mistake is using the first for production without noticing you did.
Somebody says quality dropped last month. How do we check?: Compare the recorded versions across the period and re-run a sample of the old inputs. Without that record there is no honest answer, which is why these reports so often end unresolved and are absorbed as a general sense that the tool got worse. The record is what converts the conversation into a test.
How often does the thing behind a general name actually change?: Often enough that it is a real operational fact rather than a theoretical one, and on a schedule set by the provider. That is the point of the general name: it is meant to move. What matters is whether your system's behaviour depending on that movement was a decision somebody made or a default nobody examined.

Key takeaways

Without a recorded version, "it used to work" has no route to a cause.
The general name is a moving target by design; that is what it is for.
If you do one thing, record the version rather than pin it.
A version change invalidates the tuning built around the old one.

Tools that use this

OpenRouter
Which model answers becomes a routing policy, not a code property.
Llama (Meta)
Self-served weights that change only when you decide to change them.
ChatGPT
Several members under one subscription, so which answered is a real question.