Prompt caching means the model provider keeps the start of a prompt it has just processed, and when the next request begins with exactly the same text, it reads that part from its cache and bills it at a fraction of the normal input price. For an agent that asks a model for a decision every few seconds, most of every prompt is the same text, so it is close to free money.
In my multi-agent testbed, where Claude, GPT and a local model each drive an agent in a shared Minecraft world, turning it on cut the input cost of Claude calls by two-thirds or more: across 2,774 logged Claude calls, 75-76% of input tokens were read from the cache. It took one commit, on October 17, 2026.
What it doesn't do: it doesn't reduce the number of calls, it doesn't touch output tokens, and it didn't work at all on one model (Claude Haiku 4.5), because the prompt was shorter than that model's minimum. More on each below.
Why an agent loop suits it
Each decision sends the same three things, in the same order, and only the last one changes:
| Part of the prompt | Changes between decisions? | Size here (estimated) |
|---|---|---|
| The tool definition (the shape of a decision) | No | small |
| The system prompt: rules, the goal, and the 25 skills the agent can choose from | No | about 4,300 tokens |
| The observation: position, health, inventory, what's nearby, recent events | Yes, every time | about 1,400 tokens |
The sizes are estimated from the text at about four characters a token. Measured, the median call was 5,200-8,700 input tokens depending on the model (tokenizers differ between providers), and an agent made roughly 40-140 decisions in a three-day game. Without caching, the unchanging part is paid for in full every time.
How it's coded for Claude
With Anthropic's API you mark where a reusable prefix ends with a cache_control breakpoint. The
testbed puts one after the system prompt and one after the conversation history (the turns before
the newest observation), in a single function
(anthropic.ts):
const EPHEMERAL = { type: "ephemeral" } as const;
export function cachedRequest(system: string, turns: ChatTurn[]) {
const messages: Anthropic.MessageParam[] = turns.map((turn, i) =>
i === turns.length - 2
? { role: turn.role, content: [{ type: "text", text: turn.content, cache_control: EPHEMERAL }] }
: { role: turn.role, content: turn.content },
);
return { system: [{ type: "text" as const, text: system, cache_control: EPHEMERAL }], messages };
}
The tool definition sits before the system prompt, so it is part of the cached prefix too.
The usage numbers need adding back up. With caching on, input_tokens counts only what came
after the last breakpoint. The cached part comes back separately, as tokens read from the cache and
tokens written to it, so the real input size is all three:
// input_tokens counts only what came after the last cache breakpoint; the total is all three.
const cacheRead = response.usage.cache_read_input_tokens ?? 0;
const cacheWrite = response.usage.cache_creation_input_tokens ?? 0;
inputTokens: response.usage.input_tokens + cacheRead + cacheWrite,
Miss this and every report undercounts the prompt by three-quarters.
Pricing: a cache read costs 0.1x the input price (0.05x on Opus 5.5 and Sonnet 5.5) and a cache
write 1.25x
(pricing.ts),
so each call is costed in four parts:
const plain = call.inputTokens - read - write;
const readPrice = claude ? inputPrice * (CLAUDE_CACHE_READ_BY_MODEL[base] ?? CLAUDE_CACHE_READ) : (listedCached ?? inputPrice);
const writePrice = claude ? inputPrice * CLAUDE_CACHE_WRITE : inputPrice;
return (plain * inputPrice + read * readPrice + write * writePrice + call.outputTokens * outputPrice) / 1_000_000;
OpenAI: nothing to mark
OpenAI caches long prompt prefixes by itself and reports the hits in
usage.prompt_tokens_details.cached_tokens, which the adapter records
(openai.ts).
The catch is on the billing side: for the GPT model used here, cache writes cost more than plain
input and the API doesn't report them, so the testbed bills cached input at the full price. Its GPT
costs are overstated, never understated.
Keeping the start of the prompt stable
A cache only hits when the new prompt starts with exactly the same text. That rules out one common way of keeping a conversation inside a token budget: dropping the oldest exchange every time a new one is added. Each drop changes the start of the history, so every call would miss.
The testbed's "full memory" mode, which sends the conversation so far, cuts the history in one go
when it outgrows its budget, down to half of it (CLOUD_HISTORY_REFILL = 0.5 in
llm.ts).
The start then holds until the history fills up again. The local model, which has no cache to
protect, still trims one exchange at a time.
The runs measured below used the "plan" memory mode instead, where each decision gets the system prompt plus one observation. So what they cached is the system prompt and the tool definition; the history breakpoint only comes into play with full memory.
What it saved
Every model call is logged with its usage, so these come straight from the logs: all runs on one machine between October 23 and October 30, 2026, with input prices from the testbed's price table.
| Model | Calls | Input tokens | Read from cache | Written to cache | Input cost, cached | Same input, uncached |
|---|---|---|---|---|---|---|
| Claude Opus 5.5 ($4 per million input) | 613 | 5.18M | 76.4% | 1.0% | $5.74 | $20.74 |
| Claude Sonnet 5.5 ($2 per million input) | 918 | 7.99M | 74.8% | 0.8% | $4.65 | $15.99 |
| Claude Haiku 5.5 ($0.10 per million input) | 1,243 | 11.09M | 75.1% | 1.0% | $0.36 | $1.11 |
| GPT-6 Luna | 16,231 | 92.38M | 75.2% | not reported | not costed (see above) |
That is 72% less input cost on Opus, 71% less on Sonnet and 67% less on Haiku 5.5. Opus and Sonnet save more because their cache reads cost 0.05x the input price, not 0.1x. Earlier batches in the project's write-ups read 70-88% of their input from the cache, depending on the batch.
Where it didn't work
Claude Haiku 4.5 never cached. At the time the prompt was about 2,150 tokens, under that model's minimum cacheable prompt length of 4,096 tokens, so every call paid full price. Nothing failed: the only sign was a cache-read count of zero in every response's usage. The prompts have grown since (the runs above average about 8,500 input tokens a call), so Haiku 4.5 would likely clear its minimum now, though it hasn't been run again to show it; Haiku 5.5, at those lengths, reads 75% from the cache. The testbed's report has a "Cached" column (the share of input read from the cache) for exactly this, and its docs say to check that it rises above 0% after the first few decisions: if it stays at 0% for a model, that model's minimum is above your prompt.
What this doesn't fix
- Output tokens. Only input is cached. Decisions here are short, but a model that writes long reasoning pays for every word of it.
- The part that changes. The observation (about a quarter of each call here) is new every time and always full price.
- The number of calls. Caching makes each call cheaper, not fewer. Asking the model less often (deciding on events rather than on a timer) is a separate saving.
- Small prompts. Below a model's minimum length nothing is cached, silently.
- Longer gaps. Claude's cache lasts five minutes by default, and every hit restarts the clock at
no cost. An agent that pauses longer than that between decisions pays the write again. Anthropic
offers a one-hour cache (
"ttl": "1h"), but its writes cost 2x the input price instead of 1.25x; the testbed decides every few seconds, so it uses the default.
The code, the run logs' format and the report that measures all this are in the multi-agent-experimentation repository.