Compendium

Hidden tricks and gotchas (Gemini)

10 min read

Hidden tricks and gotchas (Gemini)

The things that don’t make the front page of the docs but consistently bite or empower. Sources: ai.google.dev doc-page footnotes, google-gemini/cookbook issue threads, python-genai SDK source, and observed behavior across production deployments. Each entry is tagged with its source — [official] for documented but easily missed, [observed] for undocumented-but-stable as of 2026-05-25.

See also

1. Thinking-mode semantics changed between 2.5 and 3.x [official]

The two generations use different parameter names:

GenerationParameterRange
Gemini 2.5thinking_budget0 to 24576 tokens. 0 disables (only on 2.5 Flash; 2.5 Pro cannot disable). -1 for dynamic.
Gemini 3.xthinking_level"minimal" / "low" / "medium" / "high". Defaults vary (Pro defaults to "high").

Code that ports between generations must adapt. The SDK does not auto-translate; passing thinking_budget to a 3.x model is ignored. Passing thinking_level to 2.5 is ignored.

Pro tip — disable thinking on 2.5 Flash for low-latency tasks: setting thinking_budget=0 removes the reasoning step entirely, dropping TTFT from seconds to sub-second. Use for chat-completion-style workloads where you’d skip Claude’s extended-thinking block.

Pro tip — include_thoughts=True: surfaces a summarized version of the model’s reasoning in response.candidates[0].content.parts[].thought. Useful for debugging and UI explanations. You’re still billed for full thinking tokens, not just the summary — usage_metadata.thoughts_token_count is the truth.

2. The Gemini 3 unique id requirement on function calls [official]

Gemini 3 emits a unique id on every function_call. The matching function_response must echo it:

# Wrong (works on 2.x, breaks on 3.x):
Part(function_response=FunctionResponse(name="get_weather", response={"result": ...}))
 
# Right (works on both):
Part(function_response=FunctionResponse(id=fc.id, name="get_weather", response={"result": ...}))

If you build your own tool loop, propagate the id from function_call to function_response. The Python SDK’s automatic-function-calling handles this for you; manual loops do not.

3. Implicit cache is invisible without cached_content_token_count [official]

Implicit caching is on by default on Gemini 2.5+. There is no log, no header, no event that says “you got a cache hit.” The only feedback channel is response.usage_metadata.cached_content_token_count. If you don’t check it, you don’t know.

Sustained implicit hit rates are ~30-70 % on typical workloads with repeated prefixes. If you see 0 consistently, switch to explicit caching.

4. response_mime_type + response_schema is silently optional [observed]

You can pass response_schema without response_mime_type="application/json" and get plain-text JSON output that’s not constrained:

# Silently DOESN'T constrain decoding:
config=types.GenerateContentConfig(response_schema=MyModel)
 
# Correctly constrains:
config=types.GenerateContentConfig(
    response_mime_type="application/json",
    response_schema=MyModel,
)

If your structured output isn’t matching the schema consistently, this is the first thing to check.

5. Files API objects expire after 48 hours [official]

Uploaded files (PDF, video, audio, image) auto-delete 48 hours after upload. No warning, no grace period — they just return 404 NOT_FOUND on the 49th hour.

This breaks:

  • Caches referencing the file (cache TTL > 48h is wasted)
  • Long-running agent sessions
  • Daily-batch workloads that upload once and query throughout the day (fine) but not weekly batches

Workarounds:

  • Re-upload the file on schedule (cron / scheduler)
  • Use Vertex AI (different lifetime semantics — files live in GCS)
  • Pass bytes inline per request (capped at 20 MB)

6. Cache TTL does NOT auto-refresh on hit [official]

Unlike Claude’s prompt caching, Gemini’s explicit cache expires exactly at expire_time regardless of how recently it was hit. You must caches.update(name, ttl=...) periodically to keep it alive.

For long-running agents, set up a refresh schedule:

import time
def refresh_cache(name, every=1800):    # every 30 min, for a 1h TTL cache
    while True:
        client.caches.update(name=name, config=types.UpdateCachedContentConfig(ttl="3600s"))
        time.sleep(every)

Or set a long initial TTL (12-24h) and accept the storage cost.

7. Gemini 2.5 Pro has a >200k surcharge [official]

Gemini 2.5 Pro charges $1.25 / MTok input for the first 200k input tokens, then $2.50 / MTok for everything above. Similarly $10 → $15 on output. The watermark is per-request — sending 250k tokens doesn’t get the cheap rate on the first 200k, it triggers the >200k rate for the whole request.

Workarounds:

  • Chunk: split into requests < 200k each
  • Cache: cached tokens count differently (check current pricing for cached-tier behavior)
  • Use 2.5 Flash: no surcharge, lower base rate

8. Grounding cost model changed at Gemini 3 [official]

Gemini 2.5 grounding was per-prompt — any request that triggered any number of searches was one billable use. Gemini 3 is per-query — each search the model decides to execute is a separate billable unit.

For agentic workflows with multiple search rounds per turn, this can 5-10× the grounding bill. Mitigations:

  • Free tier: 5,000 grounded queries / month on Gemini 3 models — useful for development
  • Be explicit in system prompts about when to search (“Only search for events after 2025”)
  • Cache aggressively — cached requests don’t re-trigger searches

9. Vertex AI Express mode is the secret API-key fast path [official]

You can use Vertex AI without setting up service accounts and IAM via Express Mode:

client = genai.Client(vertexai=True, api_key="VERTEX_EXPRESS_KEY")

This gets you:

  • Regional endpoints (data residency!)
  • Vertex pricing tier
  • No GCP IAM complexity

But it loses:

  • Private VPC routing
  • Audit logs
  • Full Model Garden access (some partner models gated by IAM)

Great middle ground for indie devs wanting regional control without enterprise plumbing.

10. SDK defaults to v1beta API — pinning to v1 loses features [observed]

The google-genai SDK defaults to v1beta. Pinning to v1 is “stable” but lacks:

  • Thinking config (no thinking_budget / thinking_level)
  • Caches API (no explicit caching)
  • Code execution tool
  • Some batch modes
  • Some structured-output features

Use v1beta for any modern feature. The “beta” label is misleading; v1beta is the de-facto stable API.

11. YouTube URLs are first-class [official]

contents=[
    "Summarize this video.",
    types.Part(file_data=types.FileData(file_uri="https://www.youtube.com/watch?v=...")),
]

No upload required. Only works on public videos. Subject to YouTube usage limits (Google rate-limits by IP / project for heavy YouTube ingestion).

12. count_tokens is free [official]

client.models.count_tokens(model="gemini-3.5-flash", contents=...)

Does not invoke the model. Zero cost. Use liberally:

  • Pre-flight cost estimation
  • Cache-create cost vs no-cache decision
  • Decide chunk boundaries
  • Validate schemas (a complex response_schema failing here flags before you spend tokens)

13. Function-call ordering in parallel mode is non-deterministic [observed]

When the model returns multiple function_call parts in one turn, the order is not semantically meaningful — they’re presumed independent. The SDK auto-flow executes them sequentially by default, but for true parallelism, run them concurrently yourself. Side-effects in functions: if A and B are returned together but only one should run depending on A’s result, the model is signaling they’re independent — which is wrong.

Mitigation: declare dependent functions with explicit signaling in description (“Do not call book_flight without first calling check_seat_availability”). Or implement guards in the function bodies.

14. Tile sizing for images affects cost dramatically [official]

A 1024×1024 image = 4 tiles = 1,032 tokens. A 4096×4096 image = 36 tiles = 9,288 tokens. 9× the cost for an image you might not have intended to be that high resolution. Common surprise when users drag-and-drop original photos from phones.

Solutions:

  • Pre-resize: PIL / Sharp / equivalent to 768×768 or lower before sending
  • Set media_resolution=MediaResolution.MEDIA_RESOLUTION_LOW — forces 258 tokens flat regardless of image size

15. RECITATION finish reason [observed]

finish_reason: "RECITATION" means the output matched training data too closely (verbatim quotation likely). The model halts mid-generation. Causes:

  • User asked for verbatim copyrighted text
  • Model decided to regurgitate something it memorized

Re-prompt with paraphrasing instruction:

"Summarize in your own words, do not quote directly:"

16. Safety filter can block silently on some categories [observed]

finish_reason: "SAFETY" is the documented block path. But BLOCKLIST, PROHIBITED_CONTENT, and SPII (sensitive PII) are separate finish reasons that also indicate filtered output. Check all four. Per the docs, BLOCK_NONE on most categories is restricted to higher tiers — free tier defaults to BLOCK_MEDIUM_AND_ABOVE.

17. Embedding model upgrades require full re-indexing [official]

gemini-embedding-001 and gemini-embedding-2 vectors are not compatible — different vector spaces, no projection. Upgrading requires re-embedding everything in the index. Budget for this when planning migrations: a 100M-document index at 500 tokens / doc = 50B tokens × $0.15 / MTok = $7,500 to re-embed.

18. Cache creation latency is non-trivial [observed]

Creating a cache with 500k tokens of content takes ~30 seconds end-to-end. The caches.create call doesn’t return until encoding completes. Plan for this:

  • Async create at session start; surface progress
  • Don’t create → immediately use in tight UI loops
  • New cache occasionally returns NOT_FOUND on first use (read-replica lag) — retry once with a 1s sleep

19. Vertex AI is regional in surprising ways [official]

A cache, file, batch job, or tuned model created in us-central1 is invisible from europe-west4. The same project, the same credentials, but the resources are silo’d by region. Don’t mix; pin all related work to one region.

20. The deprecated google-generativeai SDK still works but won’t get new features [official]

The pre-2025 Python SDK (pip install google-generativeai) is in maintenance. New features (caching v2, structured caches, batch v2, multimodal embedding-2) ship only on google-genai. Migrate when you encounter a feature gap; otherwise both work.

21. Token-count differs slightly across modalities at boundaries [observed]

Per-second audio (32 tokens) and per-frame video (~263 tokens) are nominal — the actual count varies slightly based on audio compression and frame sampling. Don’t rely on exact-token math for capacity planning; use count_tokens for the truth.

22. Streaming and structured output are compatible [observed]

You can stream with response_mime_type="application/json" — partial JSON arrives incrementally. Parsing partial JSON is the consumer’s problem. Practical: stream for UX, parse the final accumulated payload, not intermediate.

buffer = ""
for chunk in client.models.generate_content_stream(...):
    buffer += chunk.text
# Parse buffer as JSON after stream completes
final = json.loads(buffer)

23. tool_config with mode=ANY + zero matching tools is undefined [observed]

If you set function_calling_config.mode = "ANY" and allowed_function_names = ["nonexistent_tool"], the model’s behavior is undefined — usually errors with MALFORMED_FUNCTION_CALL or generates plain text against the mode constraint. Validate that allowed_function_names is a subset of declared tools.

24. Live API is a different surface entirely [official]

The Live API (bidirectional voice + video streaming) uses a WebSocket rather than the REST generateContent shape, with its own session-config / streaming-input / streaming-output model. Code that works on client.models.generate_content does not port to Live — switch to client.live.connect(...) (Python SDK) or the equivalent WebSocket connection.

25. Top 5 hidden tricks summary

  1. thinking_budget=0 on 2.5 Flash — cuts latency by removing the reasoning step; production hack for chat workloads.
  2. Matryoshka two-stage retrieval — 128-dim first pass, 3072-dim rerank → 8× cheaper retrieval at scale (see embeddings-and-vertex-search-gemini).
  3. Explicit cache + Files API for long docs — first request creates cache (one-time cost), every subsequent query gets ~75 % off on the cached portion; pays for itself in 2-3 hits.
  4. media_resolution: LOW for image classification — flat 258 tokens regardless of image size; 10× cost reduction on phone-camera uploads.
  5. Free count_tokens for cost pre-flight — never guess; the API tells you the exact cost before you spend a single billed token.

26. Further reading

Graph View