A cache miss costs you money, never memory

A question worth stealing, asked in the middle of an explanation of prompt caching: we just said the cache was unreliable — so how would we know if we had lost part of the conversation?

It is the right question asked of the wrong component. You cannot lose the conversation to the cache, because the cache never had custody of it. Once that lands, caching stops being a reliability problem and becomes a line item, which is a much easier kind of problem to have.

What is actually in the request

The Messages API is stateless. Every request carries the entire transcript — the system prompt, every user turn, every assistant reply — because the model has no memory of the previous call. That is not a caching detail. It is the shape of the API, and it is the reason the conversation cannot go missing: the model sees it because you put it in the request.

What the cache holds is a precomputed shortcut for reading that prefix. The prompt renders in a fixed order — tools, then system, then messages — and the server hashes the rendered bytes up to each cache_control breakpoint. On a hit it restores the saved computation. On a miss it processes the same tokens from scratch, at full price, and writes a fresh entry.

The same request, hit and miss

	Cache hitA matching prefix was found	Cache missExpired, evicted, or never written
What you sent	The whole transcript	The whole transcript
What the server does	Restores the saved computation for the prefix	Recomputes it from the tokens you sent, then writes an entry
What you are billed	About a tenth of base input price across the cached span	Full price, plus the write premium on whatever it caches
What the model reads	The full conversation	The full conversation
The response	Identical	Identical

The bottom two rows are the whole argument. Everything the cache changes is in the middle two.

So “unreliable” cashes out as unreliable savings and perfectly reliable correctness. A cache miss costs you money, never memory.

It is worth seeing what the alternative would have cost. If the server did store your conversation and you did not resend it, expiry would mean amnesia — a reliability problem that leaks straight into correctness, and one you would have to design around on every turn. The stateless design puts the state where you control it precisely so that the server-side layer is allowed to be flaky. Nothing depends on it.

The ledger, turn by turn

One correction people usually need first: the assistant’s replies are part of the prefix too, and they are usually the bulk of the tokens. A third turn is not comments one through three. It is the system prompt, comment 1, reply 1, comment 2, reply 2, and then the new comment.

Three turns, automatic caching on

	You sendEvery time, in full	Read from cachecache_read_input_tokens	Written this turncache_creation_input_tokens
Turn 1	sys + c1	nothing yet	sys + c1
Turn 2	sys + c1 + r1 + c2	sys + c1	r1 + c2
Turn 3	sys + c1 + r1 + c2 + r2 + c3	sys + c1 + r1 + c2	r2 + c3

c is your comment, r the reply to it. The first column never shrinks: it is what leaves your machine on every request.

Notice the off-by-one. On turn 3, reply 2 is not read from cache — it could not be, because it did not exist when turn 2’s entry was written. Each request reads everything through the previous request’s breakpoint and pays the cheap write rate on whatever is new since then: the latest assistant reply plus your latest message. The cache point rolls forward one turn behind you, like a zipper closing.

And “read from cache” means retrieved as computation, not as content. Those tokens are still in the request you sent. Retrieval is a discount on processing what you sent, never a substitute for sending it.

Reading the meter

Three fields on usage report what happened, and the one that misleads people is input_tokens: it is the uncached remainder only. The prompt’s real size is the sum of all three. An agent that ran for hours and reports four thousand input_tokens did not have a small prompt.

Signature	What the fields show	What happened
Healthy	Read is large and grows each turn; creation is roughly one reply plus one message	Working as intended
Re-warmed	Read drops to zero; creation balloons to the size of the whole history	The gap between requests outran the TTL — someone took six minutes to reply
Never engaged	Both are zero	The prefix is under the model’s minimum. No error is raised
Something moved	Read covers part of the prefix; creation is far bigger than one turn	Something changed mid-prompt and invalidated everything downstream of it

In all four rows the response is identical. You would never detect any of them from the model’s answer, because the model always received the full conversation. That is what makes this the costliest kind of failure to leave unmonitored: requests keep succeeding and the bill is simply higher.

It also explains the shape it usually arrives in. Caching rarely fails on the day it is written. It fails months later, when someone interpolates a timestamp into a system prompt header, or a serializer stops ordering its keys, or a tool list becomes conditional. Nothing announces it. The standing check is small — an integration test asserting that a second identical request reports cache_read_input_tokens > 0 — and it is worth more than any amount of rereading the prompt-assembly code.

Three misses that are not your prefix

The prefix is under the minimum. Caching engages only above a model-dependent token threshold, and the thresholds are not monotonic across generations: 512 tokens on Claude Opus 5, 1,024 on Opus 4.8 and Sonnet 5, 2,048 on Opus 4.7, and 4,096 on Opus 4.6 and Haiku 4.5. A 3,000-token prefix caches on one and silently does not on another, with no code change and no error — just two zeros in usage.

The entry fell out of the lookback window. A breakpoint walks backward at most twenty positions looking for a prior entry. Runs of consecutive tool_use and tool_result blocks each collapse to a single position, so parallel tool calls are safe; a long sequential tool loop inside one turn is not. Past twenty, the next request’s breakpoint cannot find the previous entry and rewrites the whole conversation — on payloads that are byte-identical, which is what makes it so confusing to debug.

The other requests were still writing. An entry becomes readable only once the first response begins streaming. Fan out twenty identical-prefix requests at once and all twenty pay full price, none of them able to read what the others have not finished writing. Send one, wait for its first token, then fire the rest.

There is a fourth thing worth knowing, less a miss than a miscalculation. The TTL clock starts at the beginning of the request that writes or reads the entry, so generation time counts against it. A four-minute turn leaves about one minute for the next request to start. “An agent loop keeps the five-minute cache warm by itself” is true, right up until a single turn takes four minutes.

Which workloads have the shape

The pattern that pays is always the same: a big stable prefix, many cheap variable suffixes, arriving faster than the TTL expires. Everything else is a detail of where the breakpoints go.

The arithmetic is worth doing once. A five-minute write costs about 1.25× base input price and a read about 0.1×, so two requests break even against 2× uncached. A one-hour write costs 2×, which needs three requests to clear 3×. The longer TTL survives gaps in bursty traffic, but the doubled write means it has to be read more times before it earns anything.

Those multipliers are general rather than universal. Claude Fable 5.1 reads cached tokens at roughly a fortieth of base input price, which moves every break-even below it proportionally.

Agent loops are the highest-leverage case, and not narrowly. One task might take twenty to fifty calls, each carrying the same tool definitions, the same system prompt, and an ever-growing trail of tool calls and results. Almost all of it is prefix-identical between steps. Past that:

• Multi-turn chat, where every turn resends a growing history and reads all but the newest reply back — the ledger above.
• Question-answering over one document. Load the contract or the transcript once, cache it, then ask twenty questions against it. Without caching you reprocess the whole thing per question.
• Many-shot prompting, where quality keeps improving past twenty detailed examples. Caching makes the example block essentially free after the first request, so there is no longer a reason to trim it for cost.
• High-volume fixed prompts — a support bot, a moderation endpoint. Constant traffic refreshes the five-minute entry at no charge, so effectively everything after the first request is a hit.
• Layered retrieval: a breakpoint on tools, one on instructions, one on the retrieved documents, one riding the conversation. Swapping documents invalidates only the last two. Note that this is exactly four breakpoints, which is the per-request ceiling — there is no fifth layer available.
• Rate-limit headroom, the underrated one. On the Claude API cache reads do not count against input-token rate limits on most models, so keeping entries warm raises effective throughput as well as cutting cost.

The inverse is worth stating plainly, because caching is not free to get wrong. One-off prompts, prefixes that mutate on every call, and requests arriving hours apart have none of the shape. There the write premium is paid on bytes that are never read back, and caching is slightly negative.

Why this sits on an exam

Everything above is downstream of a single fact, and the Architect blueprint names that fact as an objective in its own right: “Explain why conversation history must be resent on every request”, inside Context Management & Reliability, which is 15% of the exam.

That is a strikingly specific thing to ask a candidate to explain, and this is why. People who can only recite that the API is stateless tend to treat caching as a separate feature bolted on beside it, which is exactly where the worry about losing half a conversation comes from. People who understand why the history is resent get the cache for free, because once the transcript is in the request there is nothing left for the cache to be except a discount.

The whole thing in four lines

1. You send the entire conversation on every request. The cache does not hold it and cannot lose it.
2. A hit restores computation over tokens you sent anyway; a miss redoes that computation. The model reads the same thing either way.
3. Each turn reads through the previous breakpoint and writes the newest reply plus the newest message. The cache point trails one turn behind you.
4. Watch the usage fields, because nothing else will tell you. Response quality is identical whether caching works or not.

Sources: Claude’s documentation on prompt caching — render order, the four-breakpoint limit, per-model minimums, TTL and pricing multipliers, the twenty-position lookback, and concurrent-write timing — and on rate limits. Blueprint figures are read live from the objectives this site scores against. Checked September 2026.

More on this

• Give the task the shape it already has — Most agent designs go wrong by imposing a structure the work does not have. Draw the dependencies and the pattern names itself.
• A subagent knows only its brief — Whatever the brief leaves out, the subagent has to guess.
• Where a person gets to say no — A plan to approve, a gate, a retry limit, a customer asking for a human. Each is a pause, and each belongs somewhere different.