Skip to content

Glossary

Inference endpoint

An inference endpoint is the address software sends a request to in order to reach a particular model, which is the concrete thing behind buying access.

In plain terms

When a product says it gives you access to a model, what it gives you is somewhere to send requests and something to identify yourself with. That address, and the terms attached to it, is the thing being sold. Two endpoints can point at the same model and behave differently, because what surrounds the model is not the same.

01

Why it matters

Because it is the level at which several purchasing questions become concrete. Which exact version answers, whether it changes without notice, where the request is processed, what limits apply and what is retained are all properties of the endpoint rather than of the model. A buyer thinking in terms of models is comparing something that is not what they are buying.

02

How it works

The endpoint names a model version, and whether that name is pinned to one release or follows the provider's updates is the property that matters most. This varies by provider and by generation rather than being a settled convention: some providers now pin every name by default, so there is no floating option to choose, while others still publish names that move. A pinned version is stable and is eventually retired, on notice. Establish which you have rather than assuming either.

The same model reached through different endpoints can behave differently, which surprises people who assume they are buying the model itself. Where the request is processed, what limits apply, what the provider retains and what safety layers are applied around it all vary by route, and each of those is a term of the endpoint rather than a property of the underlying model.

Access is by key, and the key is the security object rather than the address. Anything holding one can spend against your account, so where keys live, who can issue them and whether separate ones exist per application are the practical questions. Endpoints themselves are rarely secret and are not meant to be.

Retirement is a scheduled event rather than an accident, and it is the maintenance obligation this whole arrangement carries. Providers announce that a version will stop being served, and anything pinned to it has to move. Knowing which endpoints your organisation depends on is what turns that from an incident into a task.

Two routes to the same model

Two routes to the same modelSetting these side by side makes the point that the model name, which is what everybody discusses, is the one thing the two routes have in common. It also shows why the comparison is worth making rather than assuming: a team that has satisfied itself about a model's capability has answered a question about the left-hand column of a specification and none of the questions a reviewer will ask. Neither route is better in the abstract. Route A is simpler, follows improvements automatically and needs nobody to maintain a list; route B is stable, answerable about location and retention, and carries an obligation to move before a stated date. What causes trouble is having route A while believing you have route B, which is the usual state of affairs when nobody has looked.Route AFloating version name.Processed wherever the providerdecides.Provider's standard retention.Route BPinned version, with aretirement date.Processed in a named region.Retention agreed in thecontract.Identical model, identicalanswers to most questions, andtwo quite different things tohave bought. Every line ofdifference sits in the routerather than in the model.
Setting these side by side makes the point that the model name, which is what everybody discusses, is the one thing the two routes have in common. It also shows why the comparison is worth making rather than assuming: a team that has satisfied itself about a model's capability has answered a question about the left-hand column of a specification and none of the questions a reviewer will ask. Neither route is better in the abstract. Route A is simpler, follows improvements automatically and needs nobody to maintain a list; route B is stable, answerable about location and retention, and carries an obligation to move before a stated date. What causes trouble is having route A while believing you have route B, which is the usual state of affairs when nobody has looked.
03

Seen in the wild

  • One interface in front of many providers' endpoints, so software can reach several without integrating each separately.

    OpenRouter
  • A locally served endpoint on your own machine, where the address is yours and no key is being billed against.

    Ollama
  • An automation step configured with an endpoint and a key, which is where most teams first meet the concept directly.

    n8n
04

Common misconceptions

People assume

We are buying the model.

In fact

You are buying a route to it on particular terms. Processing location, retention, limits, safety layers and the version served are all properties of the route, and two routes to the same model can differ on every one of them. That is why the comparison worth making is between endpoints rather than between model names.

People assume

The endpoint is the secret.

In fact

The key is. Addresses are ordinarily published and are not intended to be confidential, while anything holding a key can spend against the account it belongs to. The questions worth attention are where keys are stored, who can issue them, and whether one exists per application so a single leak has bounded scope.

05

Telling them apart

Inference endpoint vs API

Inference endpoint

The specific address serving one model, on its own terms.

API

The general way software talks to a service.

A provider's API is the shape of the conversation; the endpoint is which model is at the other end of it.

06

Questions

Why did behaviour change without us changing anything?
If the name you call moves with the provider's updates, it followed one. Where every name is pinned, the change came from somewhere else: a product layer above the model, a changed default, or your own prompt. Establishing which kind of name you hold is the first step, because it decides whether there is anything to pin at all.
Can two endpoints for one model differ?
Yes, in most of the ways that matter for a purchase: where processing happens, what is retained, what limits apply and which safety layers surround the model. That is the practical reason to compare routes rather than model names when the difference has any bearing on what you are doing.
What should we track?
Which endpoints and versions your systems depend on, and where the keys for them live. Both are unglamorous lists that nobody maintains until a version retirement is announced or a key has to be replaced, at which point the absence of the list is the whole of the problem.
07

Key takeaways

  • You buy a route to a model on terms, not the model.
  • Whether a name is pinned or moves varies by provider; establish which you have.
  • The key is the secret; the address ordinarily is not.
  • Version retirement is scheduled, so keep a list of what depends on what.
09

Tools that use this

  • OpenRouter

    One interface in front of many providers' endpoints.

  • Ollama

    A local endpoint, with no key being billed against.

  • n8n

    Where a team usually configures an endpoint and key for the first time.

Last checked July 2026

All glossary terms