When is it worth arranging?

When a substantial block of identical material accompanies a high volume of requests: a long instruction set, reference documents or examples sent with every call in an automated process. For occasional interactive use the cache usually expires between requests and the effort is not repaid.

Why is our caching not working?

Most often because something variable sits near the front. A timestamp, a session identifier or a user name early in the prompt breaks the identical prefix and stops reuse from that point on. Moving everything stable to the front and everything changeable to the end is the first thing to check.

Does it change the answers?

No. The reuse is of the work done taking material in, not of any response, so each request is answered afresh. That also means it does not make repeated questions consistent with each other, which is a separate concern needing a different mechanism.

Can it cost more than it saves?

Occasionally, because writing to a cache often carries its own cost against a saving on later reads. Where a long prefix is used once or twice the arrangement can be marginally worse than doing nothing. Sustained repetition is what makes it pay, so volume matters more than the size of the repeated block.

Glossary

Prompt caching

Prompt caching reuses the processing already done on material you send repeatedly, so a long fixed preamble is charged and processed at a reduced rate after the first time.

In plain terms

If you send the same long chunk of material with every request, the provider can keep the work it already did on it and reuse that instead of starting again. You still pay something and you pay less, and the answer starts arriving sooner. The catch is that it only helps where the repeated part is genuinely identical and comes first.

Why it matters

Because it is one of the few cost reductions in this area that is real, sizeable and does not involve using the tool less. Most advice about AI spending amounts to sending less or accepting a weaker model; this is a case where the same work costs materially less because the provider genuinely has less to do. For anything with a long fixed preamble sent at volume, it is usually the largest single lever available.

How it works

What is reused is the processing of a prefix, which is why order decides everything. The repeated material has to sit at the front and be byte-for-byte identical between requests, because the moment anything differs the reuse stops from that point onward. A timestamp or a user name placed early in a prompt will quietly prevent the whole thing from working.

That gives a clear structural rule: stable material first, variable material last. Instructions, reference documents and examples at the front; the user's actual question at the end. Many prompts are assembled in the opposite order for no reason other than that it read naturally when somebody wrote it, and rearranging costs nothing.

Caches expire, and the lifetime is short enough that the pattern of use matters. A steady stream of requests keeps a cache warm and benefits fully; a handful of requests spread across a day may find it gone each time and gain nothing. This is why the mechanism suits sustained automated work far better than occasional interactive use.

There is often a cost to writing to the cache as well as a saving on reading from it, so it is not free in both directions. Where the same prefix is reused many times the saving dominates comfortably; where a long prefix is sent once or twice the arrangement can cost slightly more than doing nothing. Volume is what makes it worthwhile rather than length alone.

The terms vary by provider and are the part to check rather than assume: whether caching happens automatically or must be requested, how long a cache survives, what minimum size qualifies, and how the reduced rate is expressed. The mechanism is broadly consistent across providers and the specifics are not, so this belongs on the provider's own documentation rather than in a general reference.

Two prompts, same content

This is worth showing as a picture because the failure is completely silent. Nothing errors, nothing warns, and the only symptom is a bill that did not fall as expected, which is easy to attribute to almost anything else. The cause is that reuse works forward from the start of the prompt and stops at the first difference, so one changing line near the top disqualifies everything beneath it however stable that material is. The fix costs nothing and is not obvious, because prompts are usually assembled in whatever order made sense to whoever wrote the code, and putting context before the request reads naturally to a person while being exactly wrong for the mechanism. Anybody troubleshooting caching should look at the first few lines before looking anywhere else.

Seen in the wild

An automation sending the same long instruction block with every record, which is the pattern this most directly suits.
n8n
Comparing providers on whether caching is automatic or requested, and on the terms attached to it.
OpenRouter
A coding assistant repeatedly working against the same project files, where the repeated context is substantial.
Claude Code

Common misconceptions

People assume

It caches answers, so we might get a stale response.

In fact

It caches the processing of the input, not the output. Each request still produces a fresh answer; what is reused is the work of taking in material the model has already been given. Two identical questions against a warm cache can still return different responses.

People assume

It applies automatically to anything repeated.

In fact

It applies to an identical prefix, which is a stricter condition than it sounds. Anything variable placed early breaks it for everything after, so a prompt that repeats ninety per cent of its content but starts with a changing line may get no benefit at all. The order is the mechanism.

Telling them apart

Prompt caching vs Context pricing

Prompt caching

Pay less for repeated material by reusing the work already done on it.

Context pricing

Pay for everything you send, every time you send it.

Caching is the specific answer to the problem context pricing describes, and only where the repetition is exact.

Questions

When is it worth arranging?: When a substantial block of identical material accompanies a high volume of requests: a long instruction set, reference documents or examples sent with every call in an automated process. For occasional interactive use the cache usually expires between requests and the effort is not repaid.
Why is our caching not working?: Most often because something variable sits near the front. A timestamp, a session identifier or a user name early in the prompt breaks the identical prefix and stops reuse from that point on. Moving everything stable to the front and everything changeable to the end is the first thing to check.
Does it change the answers?: No. The reuse is of the work done taking material in, not of any response, so each request is answered afresh. That also means it does not make repeated questions consistent with each other, which is a separate concern needing a different mechanism.
Can it cost more than it saves?: Occasionally, because writing to a cache often carries its own cost against a saving on later reads. Where a long prefix is used once or twice the arrangement can be marginally worse than doing nothing. Sustained repetition is what makes it pay, so volume matters more than the size of the repeated block.

Key takeaways

It reuses the processing of an identical prefix, never the answer.
Stable material first, variable material last, or it silently does nothing.
Caches expire, so sustained automated work benefits and occasional use does not.
Writing to the cache has a cost; volume rather than length is what makes it pay.
The terms differ by provider and belong on their documentation, not in a general reference.

Tools that use this

n8n
The same long instruction block sent with every record.
OpenRouter
Comparing whether caching is automatic or requested, and on what terms.
Claude Code
Repeated work against the same project files, where the shared context is large.