Prompt Caching Explained: The 90% Discount Most Teams Leave on the Table
Every chat request you send to an LLM API resends everything the model needs to know: the system prompt, your instructions, the tools schema, the conversation so far. On a busy application, the same thousands of tokens travel down the wire — and get billed — hundreds of thousands of times a day. Prompt caching is the providers' answer: input tokens the API has recently seen are billed at a fraction of the normal rate, typically a 90% discount.
How it works, mechanically
When your request shares a long identical prefix with a recent request, the provider can skip recomputing that prefix and serve it from cache. The key word is prefix: the cache matches from the first token forward and breaks at the first difference. That single fact dictates how you should structure prompts:
- Static content first: system prompt, instructions, tool definitions, few-shot examples.
- Semi-static next: retrieved documents, user profile.
- Dynamic content last: the newest user message.
Put a timestamp or a random request ID at the top of your prompt and you've silently disabled caching for every request — one of the most common (and expensive) integration mistakes we see.
Provider differences that matter
- OpenAI: caching is automatic for prompts over ~1K tokens; cached input is billed at ~10% of the fresh rate. No code changes needed — but prefix discipline still decides your hit rate.
- Anthropic (Claude): caching is explicit — you mark cache breakpoints in the request. Cache writes cost ~25% extra, reads cost ~10% of fresh input, and cache lifetime is minutes (extendable). Explicit control means you can pin exactly the expensive prefix.
- Google (Gemini): offers both implicit caching and explicit cached-content objects with per-hour storage pricing — the right tool when you keep a huge context (like a whole codebase) hot for many requests.
Live prices: fresh vs cached input
From our live pricing data (updated 2026-08-02), per million input tokens:
| Model | Fresh input | Cached input | Discount |
|---|---|---|---|
| Anthropic Claude Opus 4.7 (Fast) | $30.00 | $3.00 | 90% |
| Anthropic Claude Opus 4.1 | $15.00 | $1.50 | 90% |
| Anthropic Claude Opus 4 | $15.00 | $1.50 | 90% |
| OpenAI o1 | $15.00 | $7.50 | 50% |
| Anthropic Claude Opus 5 (Fast) | $10.00 | $1.00 | 90% |
| Anthropic Claude Fable 5 | $10.00 | $1.00 | 90% |
| Anthropic Claude Opus 4.8 (Fast) | $10.00 | $1.00 | 90% |
| OpenAI GPT-5 Image | $10.00 | $1.25 | 88% |
| OpenAI GPT-5.4 Image 2 | $8.00 | $2.00 | 75% |
| Anthropic Claude Opus 5 | $5.00 | $0.5 | 90% |
| Anthropic Claude Opus 4.8 | $5.00 | $0.5 | 90% |
| Anthropic Claude Opus 4.7 | $5.00 | $0.5 | 90% |
What caching actually saves: three scenarios
- Support chatbot (the best case). 8-turn conversations resend a growing history every turn — roughly 80% of all input tokens are cacheable. Real-world bills drop 35–50%. Model it precisely in our chatbot cost calculator.
- Agent with a large tool schema. A 4K-token system-plus-tools prefix, reused across every step of every task: cache hit rates above 90% are normal, and the prefix effectively becomes free.
- One-shot document processing (the worst case). Every request has unique content, nothing repeats, caching saves ~0%. Here the lever is the Batch API instead.
A 5-minute checklist to raise your hit rate
- Move all static content to the top of the prompt; dynamic content to the bottom.
- Remove timestamps, request IDs, and user names from the system prompt.
- Keep the static prefix byte-identical across requests — even whitespace breaks matching.
- On Claude, place a cache breakpoint right after your tools/instructions block.
- Watch the
cached_tokensfield in API responses — it's the ground truth for your hit rate.
Then plug your real numbers into the API cost calculator — the caching toggle shows exactly what the discount is worth on your workload.