When is one worth it?

When more than a couple of separate things call a model, or the first time somebody asks a question nobody can answer: what did we spend on which application, what was asked last Tuesday, which model is that automation actually using. The threshold is the number of callers rather than the size of the organisation.

What does it solve that nothing else does?

A view across applications. Individual products each report their own usage, and no amount of reading those separately produces attribution across them, a single record, or the ability to change a model everywhere at once. That cross-cutting view is the thing only a common path can give you.

What does it cost us?

Something else to run and depend on. It has to be available, maintained and secured, and because everything passes through it, it becomes both a single point of failure and a concentration of credentials. Both are manageable and neither is free, which is why it is worth adopting at the point of need rather than in advance.

Glossary

AI gateway

An AI gateway is one point every request passes through on its way to a model, so keys, spending, logging and model choice are handled centrally rather than per application.

In plain terms

Instead of every application holding its own key and talking to a provider directly, they all talk to one internal service which talks to the providers. Because everything passes through it, that one place can see the spending, keep a record, apply limits and decide which model answers, without any of those having to be built into each application separately.

Why it matters

Because the alternative degrades quietly as an organisation builds more than one thing. Keys spread into several codebases, nobody can say what is being spent where, changing a model means editing every application, and there is no single record of what was asked. Each of those is tolerable alone and together they are the reason organisations reach a point where they cannot answer basic questions about their own AI usage.

How it works

Attribution is usually what justifies it first. When every request carries an identity for the application or team that made it, spending can be broken down and the argument about whose budget an overage came from has facts in it. That question is unanswerable once several applications share one provider account, and it arrives sooner than most teams expect.

Keys stop being distributed, which is the change with the most security value and the least visible effect. Applications hold credentials for the gateway instead of for the provider, so the provider keys live in one place and a compromised application does not carry an account-wide ability to spend. What the gateway then holds is worth protecting accordingly.

Model choice becomes configuration rather than code. Because the gateway decides what actually answers, swapping a model, pinning a version or routing by task can happen without touching applications, which matters most in the situation this whole category keeps returning to: a version being retired, or a cheaper model becoming good enough.

A single record of requests is the part that pays off unexpectedly. Debugging why an answer was poor, investigating an incident, understanding what a cost spike consisted of and sampling quality all need a log that spans applications, and assembling one afterwards from several places is considerably harder than having had it.

It is an additional component to run, and that is the honest cost. Everything now depends on it, it needs to be available and maintained, and below a certain scale the effort is not repaid. One application talking to one provider does not need a gateway; the threshold arrives with the third or fourth thing, or with the first question nobody can answer.

Four applications, two arrangements

What is worth drawing out here is that the left column is not a mistake anybody made. Each application was built at a different time by somebody solving their own problem, and holding a key and calling a provider is the obvious way to do that. The cost is entirely emergent: it appears in the relationships between the four, none of which any individual decision created. That is also why it tends to be addressed reactively, usually after an unexplained bill or a question about what an automation is doing that nobody can answer without reading code. The practical signal to watch for is not a number of applications but the first time somebody asks a question about AI usage across the organisation and the answer requires opening several repositories.

Seen in the wild

A single interface in front of many providers, which is the same centralising idea offered as a service rather than run internally.
OpenRouter
Several automations each holding their own provider credentials, which is the arrangement a gateway replaces.
n8n
Pointing internal applications at a locally served model, where the address is central even without a full gateway.
Ollama

Common misconceptions

People assume

It is only about saving money.

In fact

Routing is one of several things it enables and often not the first to pay off. Attribution of spending, keys held in one place, model changes without code changes and a single record across applications each stand on their own, and organisations usually adopt one for the attribution before they think about routing at all.

People assume

We are too small for one.

In fact

Possibly, and the threshold is about the number of separate things calling a model rather than the size of the organisation. A small team running four automations that each hold their own key already has the problem. One application talking to one provider genuinely does not.

Telling them apart

AI gateway vs Model routing

AI gateway

The single point everything passes through: keys, limits, logging, attribution.

Model routing

One decision made at that point: which model answers this.

Routing usually lives in a gateway. A gateway that never routes is still doing most of its job.

Questions

When is one worth it?: When more than a couple of separate things call a model, or the first time somebody asks a question nobody can answer: what did we spend on which application, what was asked last Tuesday, which model is that automation actually using. The threshold is the number of callers rather than the size of the organisation.
What does it solve that nothing else does?: A view across applications. Individual products each report their own usage, and no amount of reading those separately produces attribution across them, a single record, or the ability to change a model everywhere at once. That cross-cutting view is the thing only a common path can give you.
What does it cost us?: Something else to run and depend on. It has to be available, maintained and secured, and because everything passes through it, it becomes both a single point of failure and a concentration of credentials. Both are manageable and neither is free, which is why it is worth adopting at the point of need rather than in advance.

Key takeaways

One path means spending can be attributed across applications.
Provider keys stop being spread through several codebases.
Model changes become configuration instead of edits in every application.
It is another component to run and depend on, so adopt it at the point of need.

Tools that use this

OpenRouter
The centralising idea offered as a service rather than run internally.
n8n
Several automations each holding their own credentials, which is what it replaces.
Ollama
A central internal address, even without a full gateway in front.