What is worth recording during the live stage?

What has been fitted to this particular model: the instructions tuned to its habits, the output shapes assumed downstream, and the checks written around its known weaknesses. That list is the migration plan, it costs almost nothing to keep as you go, and reconstructing it under a deadline is the expensive way.

Does self-hosting remove the problem?

It moves the schedule to you, which is a real difference and not the same as removing it. Nothing is retired underneath you, and in exchange you own the hardware, the updates and the decision about when to move. That suits organisations that would rather choose the timing than avoid the work.

How much should selection effort matter?

Less than it usually gets, given how short the shelf life of the answer is. The durable output of that stage is the written reasoning rather than the choice, because it is what lets the next decision be made quickly instead of restarting the comparison from nothing.

Glossary

Model lifecycle

The model lifecycle is the path a model takes through an organisation, from choosing and testing it to running it and eventually replacing it, on a schedule somebody else sets.

In plain terms

Everything that happens between choosing a model and stopping using it. Choosing, testing, going live, watching it, and eventually moving off it. The unusual part compared with other software is the ending: for a bought model, the date it stops being available is set by the provider and not by you.

Why it matters

Because most organisations plan the beginning carefully and the ending not at all. The selection gets a comparison, a trial and a decision. The replacement arrives as a notice with a date on it, lands on whoever is nearest, and reveals how much work had quietly been fitted to the particular model that is going away.

How it works

Selection is the part that gets attention, and it is the part with the least lasting effect. Comparisons and trials answer a question about today, and the answer has a short shelf life because the field moves. What survives from this stage is not the choice but whatever was written down about why it was made, which is what makes the next decision cheaper.

Live use is where the fitting happens, and it is mostly invisible. Instructions get tuned to how this model responds, output shapes get assumed downstream, and people learn its habits. None of that is recorded as a dependency, which is why the cost of moving is consistently underestimated until somebody actually tries it.

The ending is scheduled externally, and this is the structural difference from other software. A provider retires a version and everything depending on it has to move, on their timetable. Products that put a routing layer over many providers exist substantially because of this: model choice becomes a routing decision rather than a set of separate accounts, which changes what a retirement costs.

Frameworks and platforms have their own version of the same problem, which is worth counting as part of the same lifecycle. An application framework in this guide names it directly: keeping up with the framework's evolution is part of the cost of building on it. That cost is real, recurring, and rarely in the business case that justified building.

Where the effort goes, and where it is needed

The imbalance here is not irrational, because the left-hand column is where the decision feels consequential and where everybody is paying attention. It is also where the effort has the shortest half-life: a careful comparison of two models answers a question that will be asked again within a year, against a different field of candidates, and very little of the analysis carries forward. The right-hand column is unglamorous and durable. A sentence recording why this model was chosen turns the next comparison into a check rather than a project. A short note of what has been tuned to its particular habits is the migration plan, written while the knowledge is free rather than reconstructed under a deadline. Neither requires a process, a tool or anybody's approval, which is probably why neither reliably happens.

Seen in the wild

One key and one balance in front of many models across many providers, so a change of model becomes a routing decision rather than a migration.
OpenRouter
A framework whose own documentation notes that keeping up with its evolution is part of the cost of building on it.
LangChain / LangGraph
Running an open model on your own machine, where nothing is retired on somebody else's schedule and the trade is that hardware is destiny.
Ollama

Common misconceptions

People assume

Once it works, it is done.

In fact

It is done until the version behind it is retired, and that date belongs to the provider. Treating a deployment as finished is what turns an announced retirement into an emergency, because the work of moving was never scoped and the dependencies were never written down.

People assume

Swapping models is straightforward.

In fact

The interfaces are often compatible and the behaviour is not. Instructions tuned to one model's habits do not transfer cleanly, output shapes shift, and the cases that worked by luck stop working. What makes it manageable is knowing what was fitted to the old one, which almost nobody records at the time.

Telling them apart

Model lifecycle vs Model monitoring

Model lifecycle

The whole path, including the parts nobody plans.

Model monitoring

The watching that happens during the live stretch of it.

Monitoring tells you how the current one is doing. The lifecycle is the reason there will be a next one.

Questions

What is worth recording during the live stage?: What has been fitted to this particular model: the instructions tuned to its habits, the output shapes assumed downstream, and the checks written around its known weaknesses. That list is the migration plan, it costs almost nothing to keep as you go, and reconstructing it under a deadline is the expensive way.
Does self-hosting remove the problem?: It moves the schedule to you, which is a real difference and not the same as removing it. Nothing is retired underneath you, and in exchange you own the hardware, the updates and the decision about when to move. That suits organisations that would rather choose the timing than avoid the work.
How much should selection effort matter?: Less than it usually gets, given how short the shelf life of the answer is. The durable output of that stage is the written reasoning rather than the choice, because it is what lets the next decision be made quickly instead of restarting the comparison from nothing.

Key takeaways

The ending is scheduled by the provider, which is the structural difference from other software.
Live use fits work to one model's habits, and none of it is recorded as a dependency.
Routing layers exist substantially because retirement is somebody else's decision.
Self-hosting moves the schedule to you rather than removing it.

Tools that use this

OpenRouter
Model choice as a routing decision rather than a migration.
LangChain / LangGraph
Keeping up with the framework's evolution named as part of the cost.
Ollama
Your own machine, where nothing is retired on somebody else's schedule.