Why do long inputs cost so much more?

Because the design compares every part of the input with every other part, so the work grows faster than the length. That is a property of the architecture rather than a pricing decision, which is why every provider's bill has the same shape and why nobody has competed the cost away.

Do I need to know any of this?

One thing, and it saves money: length is the expensive dimension. Once that is understood, the practical habits follow on their own, including sending only the relevant part of a document, starting a fresh conversation rather than continuing an enormous one, and being suspicious of any design that keeps everything in view by default.

Is something replacing it?

Alternatives are researched continuously and nothing has displaced it in the products people actually buy. The honest position is that the field consolidated hard around one design, that this is unusual, and that a buyer today can reasonably assume anything they are shown is a variation of it.

Why does the same design handle images?

Because it never depended on the parts being words. Anything divisible into pieces that can be compared works, so pictures cut into patches and audio cut into slices go through the same machinery, which is why capabilities arrive across modalities together rather than one field at a time.

Glossary

Transformer

A transformer is the design underneath almost every modern AI system, and its defining property is that every part of the input can be weighed against every other part at the same time.

In plain terms

The design nearly everything is built on. Its trick is that when working out what one word means, it can look at every other word in the whole input at once and decide which ones matter. Older designs read left to right and forgot as they went. This one holds the whole thing in view, which is why long documents became possible and why they became expensive.

Why it matters

Because two things buyers care about come from the same property. Handling a long document coherently and paying more for a long input are not separate facts about pricing and capability; they are the same design decision seen from two sides, which is why no vendor has quietly made length cheap and why the pattern holds across every product in this guide.

How it works

Every part of the input can attend to every other part, and each connection is weighed rather than fixed. Working out what a pronoun refers to, or which clause a condition qualifies, becomes a matter of assigning importance across the whole input at once instead of hoping the relevant word was recent, which is what earlier designs were reduced to.

That comparison happens for all parts simultaneously, which is what made training at scale possible. Reading in sequence forces the work to happen in order and modern hardware is built to do enormous numbers of things at once, so a design that removed the ordering reached a scale of training that had simply been unavailable before.

The cost of comparing everything to everything grows faster than the length does. Doubling an input more than doubles the comparisons, which is the direct reason long inputs are disproportionately expensive and why every product prices by length rather than by request, and it is a property of the design rather than a commercial choice.

It turned out not to care what the parts are, which is why one design now handles pictures and sound as well. Anything that can be cut into pieces and compared can be fed to it, so the same architecture underlies systems that read images, transcribe audio and generate video, and the field consolidated around it rather than diversifying.

One property, two consequences

It is worth holding these two together because they are usually encountered months apart and attributed to different causes. The capability arrives first and is genuinely impressive: a system that can be handed an entire agreement and asked about a clause on page forty, tracking what a term referred to across the whole document. Nobody explains why that became possible. The cost arrives later, usually as a surprise in the second month, when somebody notices that long conversations are dramatically more expensive than short ones and assumes it is a commercial decision to be negotiated or routed around. It is not. The mechanism that lets the system weigh page forty against page one is the same mechanism that makes the work grow faster than the input, and no provider can offer the first without charging for the second. Understanding that turns a puzzling bill into a predictable one, and it makes the sensible habits obvious rather than arbitrary.

Seen in the wild

The hub at the centre of open machine learning, where a very large share of what is published shares this one underlying design.
Hugging Face
An open-weight family downloaded and served on your own hardware, where the design's memory demands become a purchasing decision.
Llama (Meta)
An assistant that holds very long documents in one session, which is this property being sold directly as a capability.
Google Gemini

Common misconceptions

People assume

It is one of several competing designs.

In fact

The field consolidated around it to a degree that is unusual in any technology. Almost everything in this guide, across text, images, audio and video, is built on variations of the same idea, which is why capabilities and limitations rhyme so strongly across products that otherwise share nothing.

People assume

Long inputs are expensive because vendors charge for them.

In fact

They are expensive because comparing everything to everything grows faster than length does. The pricing reflects a real cost that rises steeply, which is why no competitor has undercut it away and why the same shape appears on every provider's bill.

People assume

It only applies to language.

In fact

It applies to anything that can be cut into pieces and compared, which turned out to include images, audio and video. That generality is the reason a single design ended up under systems that look completely unrelated from the outside.

Telling them apart

Transformer vs Neural network

Transformer

One specific arrangement, with attention at its centre.

Neural network

The general shape of units, layers and connections.

The narrower word makes a claim about how the parts are wired, which is where the interesting properties come from.

Questions

Why do long inputs cost so much more?: Because the design compares every part of the input with every other part, so the work grows faster than the length. That is a property of the architecture rather than a pricing decision, which is why every provider's bill has the same shape and why nobody has competed the cost away.
Do I need to know any of this?: One thing, and it saves money: length is the expensive dimension. Once that is understood, the practical habits follow on their own, including sending only the relevant part of a document, starting a fresh conversation rather than continuing an enormous one, and being suspicious of any design that keeps everything in view by default.
Is something replacing it?: Alternatives are researched continuously and nothing has displaced it in the products people actually buy. The honest position is that the field consolidated hard around one design, that this is unusual, and that a buyer today can reasonably assume anything they are shown is a variation of it.
Why does the same design handle images?: Because it never depended on the parts being words. Anything divisible into pieces that can be compared works, so pictures cut into patches and audio cut into slices go through the same machinery, which is why capabilities arrive across modalities together rather than one field at a time.