Context caching (Gemini)

Gemini ships two distinct caches with different mental models. Implicit caching is automatic on Gemini 2.5+ — Google detects prefix repetition across recent requests and applies a discount when it can, with no developer action and no cost-saving guarantee. Explicit caching is developer-managed — you client.caches.create(...) a named cached-content resource, pay a one-time write cost + hourly storage fee, and reference it by ID in subsequent requests for a guaranteed discount on the cached portion of input tokens. The two systems can coexist; explicit gives you cost predictability and longer TTLs (hours, not minutes), implicit handles the “I forgot to set up caching” case for free. The mental model is opposite to Claude / OpenAI’s prefix caches, both of which are read-through (write happens implicitly when you make the first request) — Gemini’s explicit cache is a separate API surface you call before generation.

1. Implicit caching (the free path)

Per ai.google.dev/gemini-api/docs/caching: implicit caching is on by default on Gemini 2.5 and newer models. No code change required. When the API detects a request prefix matches a recent prior request from the same project, it applies a discount transparently.

Reported in response.usage_metadata.cached_content_token_count — if non-zero, you got an implicit hit. There is no SLA on hit rate — Google describes it as best-effort. In practice, repeated requests within ~5-10 minutes with identical prefixes hit reliably; longer gaps or busy regions fall through.

To check whether implicit cache is active:

resp = client.models.generate_content(
    model="gemini-3.5-flash",
    contents=request_with_long_prefix,
)
print(f"Cached: {resp.usage_metadata.cached_content_token_count}")
print(f"Uncached input: {resp.usage_metadata.prompt_token_count}")

If cached_content_token_count is consistently 0 across reasonable repeats, switch to explicit.

2. Explicit caching (the guaranteed path)

Three-step lifecycle:

Create — client.caches.create() with the content + TTL. Returns a CachedContent resource with a name like cachedContents/abc123def456.
Use — pass cached_content=cache.name in subsequent generate_content calls. The cached content is virtually prepended; you only send the new parts.
Manage — update TTL or delete when done.

2.1 Create

from google import genai
from google.genai import types
 
client = genai.Client(api_key="...")
 
# Upload a big document via Files API
doc = client.files.upload(file="annual-report-2025.pdf")
 
# Create the cache
cache = client.caches.create(
    model="models/gemini-3.5-flash",
    config=types.CreateCachedContentConfig(
        display_name="Annual Report 2025",
        system_instruction=(
            "You are a financial analyst. Answer questions using only the cached "
            "report. If something isn't in the report, say so."
        ),
        contents=[doc],
        ttl="3600s",                          # 1 hour
        # OR: expire_time="2026-05-26T00:00:00Z"
    ),
)
print(cache.name)                              # cachedContents/abc123def456
print(cache.usage_metadata.total_token_count)  # how many tokens are cached
print(cache.expire_time)

2.2 Use

resp = client.models.generate_content(
    model="gemini-3.5-flash",                  # must match the cache's model
    contents="What was Q3 revenue?",
    config=types.GenerateContentConfig(cached_content=cache.name),
)
print(resp.text)
print(resp.usage_metadata.cached_content_token_count)

Every subsequent question reuses the cached doc for the discount.

2.3 Manage

# List
for c in client.caches.list():
    print(c.name, c.display_name, c.expire_time)
 
# Update TTL (or expire_time)
cache = client.caches.update(
    name=cache.name,
    config=types.UpdateCachedContentConfig(ttl="7200s"),
)
 
# Delete (stops storage billing)
client.caches.delete(name=cache.name)

3. What can be cached

Per the caching docs, an explicit cache can hold any combination of:

system_instruction — single text block, set at cache creation
contents — full Content / Part list: text, images, PDFs, video files, audio files (via Files API references)
tools — function declarations
tool_config

You cannot cache:

The user’s question itself (it’s per-request — that’s the point)
Generation config (temperature, max_output_tokens, etc.) — those stay on the per-request call
Safety settings (per-request only)
Thinking config — set per-request

4. Minimum tokens per model

Below the threshold, caches.create returns 400 INVALID_ARGUMENT.

Model	Minimum tokens
Gemini 3.5 Flash	1,024
Gemini 3.1 Pro	(preview — assume 4,096)
Gemini 2.5 Pro	4,096
Gemini 2.5 Flash	1,024
Gemini 2.5 Flash-Lite	does not support explicit caching as of 2026-05
Gemini 1.5 Pro	32,768 (legacy higher threshold)
Gemini 1.5 Flash	32,768

The 1.5 → 2.5 generation drop from 32k to 1-4k is the headline change: caching is now economical for short-but-stable prompts (long system prompts, short RAG docs), not just full-novel-length context.

5. TTL options

ttl="300s"       # 5 minutes (minimum)
ttl="3600s"      # 1 hour
ttl="86400s"     # 24 hours
# OR
expire_time="2026-05-26T18:00:00Z"   # absolute

The minimum TTL is 5 minutes (300 seconds). No documented maximum — practical caps are determined by storage cost and quota. Default if neither ttl nor expire_time is set: 1 hour (per ai.google.dev/gemini-api/docs/caching 2026-05).

TTL refresh on hit? Per the docs, the cache does not auto-refresh on hit — once expire_time arrives, the cache is deleted. Update TTL explicitly with caches.update if you need to keep it alive. (This is the inverse of Claude’s behavior, which auto-refreshes on hit.)

6. Cost model

Per ai.google.dev/gemini-api/docs/pricing 2026-05:

Component	Pricing
Cache creation	One-time charge: the tokens being cached, at the model’s standard input rate
Cache storage	Hourly storage fee per cached MTok — approximately $1.00 per MTok per hour on Gemini 3.5 Flash (varies by model; check pricing page for current numbers)
Cache hit (read)	Cached tokens billed at a discounted rate vs base input (typically ~25 % of base)
Non-cached input	Standard input rate
Output	Standard output rate

Worked example (Gemini 3.5 Flash, $1.50 in / $9.00 out per MTok)

Scenario: 500k-token document cached for 4 hours, 50 questions asked against it.

Create cost: 500k tokens × $1.50 / MTok = $0.75 (one-time)
Storage cost: 500k tokens × $1.00 / MTok / hour × 4 hours = $2.00
Per-question cache read: 500k × $0.375 / MTok (assuming 25 % of base) = $0.1875
Per-question new input + output: trivial (~few cents)
Total for 50 questions: $0.75 + $2.00 + 50 × $0.1875 = $12.125
Without caching: 50 × 500k × $1.50 / MTok = $37.50
Savings: ~68 %

Break-even

A cache pays for itself when:

(uncached_cost - cached_cost) × N >= write_cost + storage_cost × hours

For Gemini 3.5 Flash with 75 % read discount and $1.00 / MTok / hour storage, the break-even is roughly 2-3 hits within the first hour for any non-trivial cache. After that, caching is pure savings.

7. Implicit vs explicit decision tree

Situation	Use
Quick scripts, prototyping, low-volume	Implicit (just works)
Repeated questions on same long document	Explicit (guaranteed savings)
Long-running agent with stable system prompt	Explicit (system prompt only)
RAG with rotating top-k chunks	Neither effective (prefix changes)
One-shot requests	Neither
Multi-turn chat with growing history	Implicit (prefix grows naturally; explicit doesn’t help)
Long-context workloads (>100k tokens) where the doc is reused	Explicit, always
Need cost predictability (billing matters)	Explicit

8. Cache hit tracking

def cache_efficiency(response):
    u = response.usage_metadata
    cached = u.cached_content_token_count or 0
    total_input = u.prompt_token_count + cached
    return {
        "input": u.prompt_token_count,
        "cached": cached,
        "thoughts": u.thoughts_token_count or 0,
        "output": u.candidates_token_count,
        "hit_rate": cached / total_input if total_input else 0,
    }

For an explicit cache, cached_content_token_count should match the cache’s usage_metadata.total_token_count on every hit. If it’s zero, the cache reference didn’t match — verify cached_content=cache.name was passed in the config and the model matches.

9. Cache + tools

You can cache tool definitions:

cache = client.caches.create(
    model="models/gemini-3.5-flash",
    config=types.CreateCachedContentConfig(
        system_instruction="...",
        tools=[
            types.Tool(function_declarations=[
                types.FunctionDeclaration(name="search", ...),
                types.FunctionDeclaration(name="fetch", ...),
                # ... 20 more tools
            ]),
        ],
        ttl="3600s",
    ),
)

For agent loops with dozens of tools (tool descriptions alone can be 20-50k tokens), this is a major saving. The cache covers the system prompt + tool declarations; per-request you only pay for the user message + tool-result blocks.

10. Cache + thinking

Per the caching docs: thinking content is not stored in the cache. Each request runs its own thinking. This is fine — thinking is per-task anyway.

What this means in practice:

cache_creation_input_tokens covers system + content + tools
thoughts_token_count is per-request, billed at output rate (per gemini-models-and-capabilities)
Toggling thinking mode does not invalidate the cache

11. Cache + multimodal

The cache can contain image / PDF / video / audio files (uploaded via Files API). The underlying File references must still exist when the cache is used — if you delete the File, the cache becomes unusable even before expire_time.

Sequence:

doc = client.files.upload(file="video.mp4")
cache = client.caches.create(
    model="models/gemini-3.5-flash",
    config=types.CreateCachedContentConfig(contents=[doc], ttl="3600s"),
)
# OK to use:
client.models.generate_content(model="gemini-3.5-flash", contents="...",
                              config=types.GenerateContentConfig(cached_content=cache.name))
 
# Do NOT do this until the cache is dead:
# client.files.delete(name=doc.name)        # would break the cache

Files API objects auto-expire after 48 hours. If the cache TTL exceeds 48 hours, the cache will be orphaned once the file is gc’d. Match cache TTL ≤ file lifetime, or re-upload the file periodically.

12. Cache + Vertex AI

Vertex AI exposes the same caches API surface with a different resource path (projects/PROJECT/locations/LOCATION/cachedContents/...). The model ID format drops the models/ prefix (gemini-3.5-flash vs models/gemini-3.5-flash). Storage costs apply at Vertex pricing — typically same as Developer API but check the Vertex pricing page for region-specific rates.

Cross-region: caches are regional on Vertex. A cache in us-central1 is not visible from europe-west4. Pick the region your inference runs in.

13. Common pitfalls

Below threshold — caching a 500-token system prompt on Gemini 2.5 Pro returns 400. Check minimum tokens.
Forgetting cached_content in the request config — easy to leave off after creating the cache. Hit rate goes to zero with no error.
Mismatched model — cache for gemini-3.5-flash can’t be used with gemini-2.5-flash. The error is explicit but easy to ignore when refactoring.
TTL not auto-refreshed on hit — different from Claude. Set TTL longer than your expected session length, or call caches.update periodically.
File deletion — deleting an uploaded file invalidates caches referencing it. Coordinate lifetimes.
Implicit cache invisible — cached_content_token_count is the only feedback signal. No “you got an implicit hit” log. Don’t assume it’s working; verify.
Generation config mid-cache — changing temperature / max_output_tokens is fine (per-request). Changing safety_settings is fine. Changing system_instruction requires a new cache because system is part of the cache content.
Cost model surprise — storage fees accrue even when the cache is not hit. A 1M-token cache left running for 24 hours costs ~$24 in storage alone (Gemini 3.5 Flash rates). Delete unused caches.

13b. Undocumented as of 2026-05-25 — observed behavior

Cache creation latency scales with cache size — caching 500k tokens takes ~30 seconds to complete. The create call returns only after content is fully encoded. Plan for this in interactive applications (cache async, then surface).
Cache state propagation — newly-created caches occasionally return NOT_FOUND on the first generate call if the read replica hasn’t synced. Add a 500ms-1s wait, or retry on NOT_FOUND.
Implicit + explicit interaction — if you reference an explicit cache AND your request prefix matches an implicit-cache hit, the implicit hit is ignored (explicit takes precedence). Behaviorally identical to using explicit only.

Before caching: large workloads stored long contexts in app code and re-sent them every request. With caching, the right pattern is:

# On session start:
session_cache = client.caches.create(
    model="models/gemini-3.5-flash",
    config=types.CreateCachedContentConfig(
        system_instruction=long_system_prompt,
        contents=session_documents,
        ttl="3600s",
    ),
)
session_state["cache_name"] = session_cache.name
 
# On each user turn:
resp = client.models.generate_content(
    model="gemini-3.5-flash",
    contents=user_message,
    config=types.GenerateContentConfig(cached_content=session_state["cache_name"]),
)
 
# On session end:
client.caches.delete(name=session_state["cache_name"])

For long-running agents, refresh TTL periodically via caches.update rather than re-creating.

15. Comparison to other vendors

Feature	Gemini	Claude	OpenAI
Creation	Explicit `caches.create` resource	Per-request `cache_control` breakpoint	Automatic (>1024 tokens)
Manual breakpoints	n/a (whole cache is one breakpoint)	Up to 4 per request	n/a
TTL options	5 min minimum, no max	5 min / 1 hour	5-10 min (server-managed)
TTL refresh on hit	No	Yes (5m / 1h)	Yes (sliding)
Storage cost	Yes (hourly fee)	No	No
Read discount	~75 % (varies)	90 %	50 %
Minimum tokens	1,024 - 4,096	1,024 - 4,096	1,024
Multimodal support	Yes	Yes	Yes (text-only originally, vision later)

Gemini’s explicit cache is unique in being a first-class resource with its own lifecycle. The tradeoff: more control + cost predictability, more code, and a storage bill.

16. Further reading

Context caching — canonical reference
Long context — why caching matters for 1M+ workloads
Pricing — storage + read rates per model
Caches API reference — REST surface

Compendium

Explorer

Context caching (Gemini)

Context caching (Gemini)

See also

1. Implicit caching (the free path)

2. Explicit caching (the guaranteed path)

2.1 Create

2.2 Use

2.3 Manage

3. What can be cached

4. Minimum tokens per model

5. TTL options

6. Cost model

Worked example (Gemini 3.5 Flash, $1.50 in / $9.00 out per MTok)

Break-even

7. Implicit vs explicit decision tree

8. Cache hit tracking

9. Cache + tools

10. Cache + thinking

11. Cache + multimodal

12. Cache + Vertex AI

13. Common pitfalls

13b. Undocumented as of 2026-05-25 — observed behavior

15. Comparison to other vendors

16. Further reading

Graph View

Table of Contents

Compendium

Explorer

Context caching (Gemini)

Context caching (Gemini)

See also

1. Implicit caching (the free path)

2. Explicit caching (the guaranteed path)

2.1 Create

2.2 Use

2.3 Manage

3. What can be cached

4. Minimum tokens per model

5. TTL options

6. Cost model

Worked example (Gemini 3.5 Flash, $1.50 in / $9.00 out per MTok)

Break-even

7. Implicit vs explicit decision tree

8. Cache hit tracking

9. Cache + tools

10. Cache + thinking

11. Cache + multimodal

12. Cache + Vertex AI

13. Common pitfalls

13b. Undocumented as of 2026-05-25 — observed behavior

14. Migration: from manual prompt-prefix sharing to caching

15. Comparison to other vendors

16. Further reading

Graph View

Table of Contents