Compendium

Prompt caching

9 min read

Prompt caching

Prompt caching is the single highest-leverage cost optimization for any non-trivial Claude application. The mechanism is server-side prefix caching — Anthropic stores the KV-cache state at marked breakpoints and reuses it on subsequent requests that hit the same prefix, charging 0.1× the base input price for the cached portion (a 90 % discount on input tokens). The write cost is 1.25× base for 5-minute TTL and 2× for 1-hour TTL. This note covers the full mechanics: breakpoint placement, key composition, invalidation rules, multi-turn strategies, and the gotchas that make the difference between a 50 % cost reduction and a 90 % one.

See also

1. The two implementation paths

Per platform.claude.com/docs/en/build-with-claude/prompt-caching.

1.1 Automatic caching (the easy mode)

Add cache_control at the top level of the request:

response = client.messages.create(
    model="claude-opus-4-7",
    max_tokens=1024,
    cache_control={"type": "ephemeral"},   # the magic
    system="You are a helpful assistant.",
    messages=[...],
)

The system manages breakpoints for you. Best for multi-turn conversations where the prefix grows naturally. Costs one of your four explicit breakpoints if you also use explicit ones (will 400-error if you’ve already used all 4).

Available on Claude API, Claude Platform on AWS, and Microsoft Foundry (beta). Not on Bedrock or Vertex.

1.2 Explicit breakpoints (the precise mode)

Place cache_control on individual content blocks:

system=[
    {"type": "text", "text": "You are a legal document analyzer."},
    {
        "type": "text",
        "text": "[Full 50-page legal agreement text here]",
        "cache_control": {"type": "ephemeral"},   # cache the document
    },
],

Best for fixed long prefixes (long system prompts, RAG context, tool definitions). Required when sections of your prompt change at different frequencies.

2. TTL and pricing

Per the prompt-caching docs:

TTLHeader required?Write cost (× base input)Read cost (× base input)
5 minutes (default)No1.25×0.1×
1 houranthropic-beta: extended-cache-ttl-2025-04-112×0.1×

Cache hits refresh the TTL at no additional cost. So a continuously-hit 5-minute cache effectively lives forever.

Worked example: Opus 4.7 (\$5 in, \$25 out per MTok)

OperationPrice per MTok
Base input$5.00
5m cache write$6.25
1h cache write$10.00
Cache hit / refresh (either TTL)$0.50
Output$25.00

Break-even: A 5m cache write pays for itself after one cache hit ($6.25 to write vs $0.50 to read; the saved $4.50 already covers the 25 % premium since ($5-$0.50) = $4.50 > $1.25 write premium). A 1h cache write breaks even after two hits.

In practice, for any prefix that will be reused more than once in 5 minutes, always cache. The decision tree:

  • Reused 2+ times in 5 min → 5m cache (default)
  • Reused 2+ times but gaps > 5 min and < 1 h → 1h cache
  • Reused only once → don’t cache (write premium not earned back)

3. Breakpoint mechanics

3.1 Maximum 4 explicit breakpoints per request

A 5th explicit cache_control returns 400. Automatic caching uses one of the slots. So:

  • 4 explicit + no automatic → fine
  • 3 explicit + automatic → fine
  • 4 explicit + automatic → 400 error

3.2 20-block lookback window

On read, the system scans backward 20 blocks from the breakpoint to find a matching prior cache entry. If none is found within 20 blocks, the lookup stops and the prefix is written fresh.

For long-growing conversations: if the conversation grows by more than 20 blocks between requests (rare in practice), you need additional breakpoints. The standard pattern of placing the breakpoint on the most recent turn handles this implicitly.

3.3 Cache key composition

Order matters. The cache key is built as:

  1. Tools — all tools[] definitions up to (and including) any cache_control on a tool
  2. System — system message blocks up to any cache_control there
  3. Messages — conversation messages up to any cache_control there

Changes invalidate that level and all levels below. So changing a single tool definition invalidates the entire tool prefix, plus all dependent system and message caches.

3.4 Invalidation matrix

Per the official docs:

ChangeInvalidates tools?Invalidates system?Invalidates messages?
Tool definitions changed✘✘✘
Web search toggled✓✘✘
Citations toggled✓✘✘
Speed setting✓✘✘
tool_choice✓✓✘
Images added✓✓✘
Thinking params changed✓✓✘

✓ = preserved, ✘ = invalidated. So: changing thinking budget preserves the system-prompt cache but invalidates the message cache. Toggling thinking mode mid-conversation always invalidates message caches — plan thinking strategy per session.

4. Minimum cacheable tokens (per model)

Below the threshold, the cache silently no-ops (no error, just no cache effect):

ModelMinimum tokens at breakpoint
Mythos Preview, Opus 4.7, Opus 4.6, Opus 4.54,096
Sonnet 4.6, Sonnet 4.5, Opus 4.1, Haiku 4.51,024
Haiku 3.5 (Bedrock / Vertex only)2,048

Always verify cache write succeeded by checking response.usage.cache_creation_input_tokens > 0. If it’s 0, you’re below threshold or your breakpoint placement is wrong.

5. Tracking cache performance

The usage block reports four cache-relevant fields per request:

{
  "usage": {
    "input_tokens": 50,                   // tokens after last breakpoint (uncached)
    "cache_creation_input_tokens": 5120,  // newly cached this request
    "cache_read_input_tokens": 1800,      // read from prior cache
    "cache_creation": {                   // split by TTL
      "ephemeral_5m_input_tokens": 5120,
      "ephemeral_1h_input_tokens": 0
    },
    "output_tokens": 503
  }
}

Total billed input tokens = cache_creation_input_tokens × ttl_multiplier + cache_read_input_tokens × 0.1 + input_tokens. Verify your hits like this:

def cache_stats(response):
    u = response.usage
    total_input = (
        u.input_tokens
        + (u.cache_creation_input_tokens or 0)
        + (u.cache_read_input_tokens or 0)
    )
    hit_pct = (u.cache_read_input_tokens or 0) / total_input * 100 if total_input else 0
    return f"hit: {hit_pct:.1f}% ({u.cache_read_input_tokens or 0}/{total_input} tokens cached)"

In a well-tuned long-context agent, sustained cache-hit percentages of 70-95 % are typical.

6. Cacheable content types

Can cache:

  • Tool definitions (tools[])
  • System messages (text blocks in system)
  • Text messages (user and assistant)
  • Images, PDFs, documents (user messages)
  • tool_use and tool_result blocks
  • Thinking blocks from prior turns (count as input tokens when present)

Cannot cache:

  • Thinking blocks with explicit cache_control (silently fails)
  • Sub-content like individual citations
  • Empty text blocks

7. Pre-warming the cache (max_tokens: 0)

A request with max_tokens: 0 performs cache write without generating output. Useful at session start:

client.messages.create(
    model="claude-opus-4-7",
    max_tokens=0,                     # no generation
    system=[
        {
            "type": "text",
            "text": large_system_prompt,
            "cache_control": {"type": "ephemeral"},
        }
    ],
    messages=[{"role": "user", "content": "warmup"}],
)
# Returns stop_reason="max_tokens", content=[], usage.cache_creation_input_tokens > 0

Then real requests hit the warm cache immediately. Limitation: cannot combine max_tokens: 0 with extended thinking — Anthropic explicitly disallows it.

8. Mixing TTLs

You can place multiple cache_control blocks with different TTLs. Longer TTLs must come first in the request:

system=[
    {
        "type": "text",
        "text": "Static instructions (long-lived)...",
        "cache_control": {"type": "ephemeral", "ttl": "1h"},   # first
    },
    {
        "type": "text",
        "text": "Daily-rotating context...",
        "cache_control": {"type": "ephemeral", "ttl": "5m"},   # then
    },
],

Billing math (per docs):

  • Position A = highest cache hit
  • Position B = highest 1h breakpoint after A
  • Position C = last breakpoint
  • Charge = reads(A) + 1h_writes(B − A) + 5m_writes(C − B)

9. Workspace / organization isolation

Per the docs (as of 2026-02-05):

SurfaceCache isolation
Claude APIper-workspace
Claude Platform on AWSper-workspace
Microsoft Foundry (beta)per-workspace
Bedrockper-organization (no workspace-level)
Vertex AIper-organization

Cache hits require 100 % identical prompts (including all text, all images, all tool definitions). A single different byte invalidates. Different orgs / workspaces never share caches.

10. Multi-turn conversation strategy

The canonical pattern: place the breakpoint on the last identical block across requests.

For a growing conversation:

messages = [
    {"role": "user", "content": "..."},          # turn 1
    {"role": "assistant", "content": "..."},     # turn 1
    {"role": "user", "content": "..."},          # turn 2
    {"role": "assistant", "content": "..."},     # turn 2
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": new_user_question,
                "cache_control": {"type": "ephemeral"},   # breakpoint on the last turn
            }
        ],
    },
]

On the next turn, the breakpoint moves forward; the prior cache (now within the 20-block lookback) is read.

For a fixed long system prompt + rotating user queries, breakpoint on the system prompt:

system=[
    {
        "type": "text",
        "text": fixed_30k_token_prompt,
        "cache_control": {"type": "ephemeral", "ttl": "1h"},
    }
],
messages=[{"role": "user", "content": query}],

11. Tool-definition caching

Tool definitions can be cached. Place cache_control on the last tool that’s stable across requests:

tools=[
    {"name": "tool_a", "description": "...", "input_schema": {...}},
    {"name": "tool_b", "description": "...", "input_schema": {...}},
    {
        "name": "tool_c",
        "description": "...",
        "input_schema": {...},
        "cache_control": {"type": "ephemeral"},  # caches all 3 tools above
    },
],

Particularly valuable for agentic loops with dozens of tools — tool descriptions alone can be tens of thousands of tokens. Tool-use system-prompt overhead (346 tokens for Claude 4) is also cached as part of the tools prefix.

12. Caching with extended thinking

Per the extended thinking docs:

  • System prompt caches are preserved when thinking parameters change.
  • Message caches are invalidated when budget_tokens changes or thinking enabled/disabled flips.
  • Cached thinking blocks count as input tokens when read.
  • On Opus 4.5+ / Sonnet 4.6+, thinking blocks are kept in context by default; on older models / Haiku, thinking blocks are stripped if a non-tool-result user block follows.

Practical: pin a single thinking config per session. Use 1-hour cache when thinking is enabled (thinking tasks often exceed 5 minutes).

13. Caching with citations / web search

  • Citations + caching: place cache_control on the document block, citations work normally on cached docs.
  • Web search + caching: web search invalidates the tools cache level only (per the matrix above) — system and message caches survive. The encrypted_content and encrypted_index in web-search results must be passed back unmodified in subsequent turns (don’t strip them).

14. Common pitfalls

  1. Breakpoint on changing content: If the breakpoint block contains a timestamp, per-request ID, or any varying content, the hash never matches → no cache hit. Move the breakpoint up to the last stable block.
  2. Below minimum: Caching a 500-token system prompt on Opus silently no-ops (threshold 4,096).
  3. Tool changes nuking everything: Add or remove a tool, and the entire tool + system + message cache invalidates. Either keep tools stable or accept the cost.
  4. Toggling thinking mid-session: Invalidates message caches even with same budget_tokens (since thinking encoding changes the prefix). Decide thinking strategy at session start.
  5. Forgetting to verify: cache_creation_input_tokens == 0 and cache_read_input_tokens == 0 for many requests means caching is silently doing nothing.
  6. Cache miss after exceeding lookback: Growing conversations beyond 20 blocks need additional breakpoints — though this is rare since the breakpoint moves with the conversation.
  7. Cross-workspace isolation: A “shared” prompt across two workspaces doesn’t share a cache.

15. Debugging cache hits

For deeper introspection (per the cache-diagnosis-2026-04-07 beta header), the API returns enhanced cache attribution. Otherwise, the standard usage fields plus this checklist:

  1. Is cache_creation_input_tokens > 0 on the first request after a quiet period? Yes → cache is being written.
  2. Is cache_read_input_tokens > 0 on the second request within TTL? Yes → cache is being hit.
  3. If 1 ✓ and 2 ✘ → check that (a) prompt is byte-identical, (b) same workspace, (c) within TTL, (d) at least the minimum tokens.

16. Further reading

Graph View

Backlinks