Compendium

Embeddings and Vertex AI Search (Gemini)

8 min read

Embeddings and Vertex AI Search (Gemini)

Google’s embeddings story in 2026 splits between gemini-embedding-2 (the current multimodal embedding model — text, image, video, audio, documents all mapped into a single unified embedding space for cross-modal search) and gemini-embedding-001 (text-only legacy, still supported but on a deprecation glide-path). Both support Matryoshka representations via the output_dimensionality parameter — you can ask for 128, 256, 768, 1536, or 3072 dimensions and the model returns the appropriately-truncated head of the full vector, which means you can store cheap 128-dim vectors for first-pass retrieval and re-rank with the full 3072-dim later. For managed RAG infrastructure, Vertex AI Search (formerly Discovery Engine) is Google’s hosted product — upload docs, get an indexed semantic search endpoint without running your own vector DB.

See also

1. Model catalog

Per ai.google.dev/gemini-api/docs/embeddings 2026-05.

ModelModalitiesMax input tokensOutput dims (Matryoshka)Status
gemini-embedding-2Text, image, video, audio, documents8,192 (per item)128 / 256 / 768 / 1536 / 3072Stable / GA
gemini-embedding-001Text only2,048128 / 256 / 768 / 1536 / 3072Stable, deprecating
text-embedding-004Text only2,048768 (fixed)Legacy

Embedding spaces are incompatible between models. Upgrading from gemini-embedding-001 to gemini-embedding-2 requires re-embedding all existing data. There’s no projection between them. Plan migrations as a re-index.

Why gemini-embedding-2 is a big deal

Cross-modal search in a single index: you can query “show me clips where someone laughs” and match against an embedding index that mixes text descriptions, image frames, and audio clips. Before gemini-embedding-2, this required separate per-modality indexes + a reranking layer.

2. Basic embedding call

Python (unified SDK)

from google import genai
from google.genai import types
 
client = genai.Client(api_key="...")
 
result = client.models.embed_content(
    model="gemini-embedding-2",
    contents=[
        "First chunk of text to embed.",
        "Second chunk.",
        "Third chunk.",
    ],
    config=types.EmbedContentConfig(
        output_dimensionality=768,
    ),
)
 
for emb in result.embeddings:
    print(len(emb.values))      # 768
    print(emb.values[:5])       # first 5 floats

Multimodal

result = client.models.embed_content(
    model="gemini-embedding-2",
    contents=[
        "A photo of a cat sleeping in a sunbeam.",          # text
        types.Part.from_bytes(data=img_bytes, mime_type="image/jpeg"),  # image
        client.files.upload(file="clip.mp4"),                # video via Files API
    ],
    config=types.EmbedContentConfig(output_dimensionality=1536),
)

All three embeddings sit in the same vector space — cosine similarity is meaningful across modalities.

REST

curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-embedding-2:embedContent" \
  -H "x-goog-api-key: $GEMINI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "content": {"parts": [{"text": "Hello world"}]},
    "outputDimensionality": 768
  }'

For batches, use :batchEmbedContents:

POST .../models/gemini-embedding-2:batchEmbedContents
{
  "requests": [
    {"content": {"parts": [{"text": "First"}]}, "outputDimensionality": 768},
    {"content": {"parts": [{"text": "Second"}]}, "outputDimensionality": 768}
  ]
}

3. Matryoshka dimensions

The output is truncatable: a 3072-dim vector contains the same information as a 1536-dim vector in its first 1536 components. The model is trained so that the truncated vectors are still useful (not random subsets).

Allowed values

128, 256, 768, 1536, 3072. Not arbitrary — these are the supported truncation points.

Tradeoffs

DimQualityStorage / index sizeSearch speed
128Lossy — first-pass coarse8× smaller than 10248× faster
256Good for high-recall first pass4× smaller4× faster
768Strong default — most RAGBaselineBaseline
1536Higher precision2× larger2× slower
3072Best quality, near-saturation4× larger4× slower

Two-stage retrieval pattern

# Index: store both 128-dim (cheap) and 3072-dim (precise)
emb_full = embed(text, dim=3072)
emb_short = emb_full[:128]   # truncation works in NumPy too if normalized
 
# Search:
# Stage 1: ANN search on 128-dim index, k=200 candidates
# Stage 2: rerank top 200 with 3072-dim cosine

Saves 96 % of search compute with minimal quality loss. Standard pattern for high-QPS retrieval at scale.

4. Task type (legacy on 001, replaced on 2)

gemini-embedding-001 accepts a task_type parameter that biases the embedding:

config=types.EmbedContentConfig(
    output_dimensionality=768,
    task_type="RETRIEVAL_DOCUMENT",      # for documents being indexed
    # or "RETRIEVAL_QUERY" for queries
    # or "SEMANTIC_SIMILARITY", "CLASSIFICATION", "CLUSTERING", "QUESTION_ANSWERING", "FACT_VERIFICATION"
)

Important asymmetry: documents indexed with RETRIEVAL_DOCUMENT should be queried with RETRIEVAL_QUERY (and vice versa). Mixing causes ~10-20 % retrieval-quality loss in observed benchmarks. The two task types produce embeddings biased for their role in retrieval, even though they share the same vector space.

On gemini-embedding-2

Per ai.google.dev/gemini-api/docs/embeddings 2026-05: “include task instructions directly in the prompt for text-only tasks” rather than via a task_type parameter. The newer model handles task biasing via natural-language instruction embedded in the input.

result = client.models.embed_content(
    model="gemini-embedding-2",
    contents=["Retrieve documents matching: What is photosynthesis?"],
)

For document indexing, just embed the document text without instruction prefixes. For querying, prefix the query with intent: “Retrieve documents matching: “.

5. Normalization

Embeddings are L2-normalized by default — cosine_similarity and dot_product give equivalent results. Most vector DBs (Pinecone, Qdrant, pgvector, Weaviate) expect normalized inputs; just use them directly.

If you truncate a Matryoshka vector below the model’s output, re-normalize:

import numpy as np
v = np.array(emb.values)
v_short = v[:128]
v_short = v_short / np.linalg.norm(v_short)

6. Pricing

Per ai.google.dev/gemini-api/docs/pricing:

ModelCost per 1M input tokens
gemini-embedding-2(check pricing page — typically $0.15-$0.30)
gemini-embedding-001Lower legacy rate

Free tier: limited daily quota, suitable for prototyping.

Batch embeddings: 50 % discount via client.batches.create_embeddings(...).

7. Token counting + cost estimation

count = client.models.count_tokens(
    model="gemini-embedding-2",
    contents=texts,
)
print(f"{count.total_tokens} tokens × \$0.15/MTok = \${count.total_tokens / 1_000_000 * 0.15:.4f}")

For 1M chunks of average 500 tokens each = 500M tokens = $75 to embed. Compare to OpenAI text-embedding-3-large at $0.13 / MTok for same workload.

8. Vertex AI Search (managed RAG)

Per cloud.google.com/vertex-ai-search 2026-05. Google’s hosted RAG product (formerly Discovery Engine).

What it is

  • Upload documents (PDF, text, structured records) to a “datastore”
  • Vertex AI Search indexes them automatically (embedding + keyword)
  • Query endpoint returns ranked results + LLM-synthesized answer (powered by Gemini)
  • No code to run embeddings, no vector DB to operate

When to use

  • Production RAG without infra work
  • Multi-source search (mix docs + structured data + cloud storage)
  • Compliance requirements (HIPAA / PCI / GDPR — Google manages security boundary)
  • Customer-facing search UIs (Vertex provides ready widgets)

When to skip

  • Highly custom retrieval (BM25 + rerank pipelines, hybrid sparse-dense, etc.) — run your own
  • Tight cost control on a known workload — managed services charge premiums
  • Need cross-modal embedding flexibility — use gemini-embedding-2 directly

Tiers

  • Search — basic semantic + keyword search
  • Search + Generative AI — adds LLM answer synthesis
  • Vertex AI Agent Builder — adds full conversational agent on top

Per-tier pricing varies; see Vertex pricing page.

9. Vector DB choices (when running your own)

Common pairings with Gemini embeddings:

DBStrengthsNotes
pgvector (Postgres)Cheap, transactional, joins with relational dataStrong for hybrid (semantic + filter) queries
PineconeManaged, fast ANN, simple APIPer-vector pricing; check at scale
QdrantSelf-hosted or cloud, rich filters, Rust speedStrong open-source choice
WeaviateMulti-modal native, hybrid search built-inGood fit for gemini-embedding-2 multimodal
MilvusScales to billions of vectorsHeavier ops
Vertex AI Vector SearchNative GCP, scales to billions, low-latencyBest for GCP-resident workloads

For multimodal indexes with gemini-embedding-2, Weaviate and Qdrant have first-class multimodal support; pgvector treats all vectors as opaque (works fine, just not optimized).

10. Worked RAG example

from google import genai
from google.genai import types
 
client = genai.Client(api_key="...")
 
# Index docs
docs = ["Document chunk 1...", "Document chunk 2...", ...]
result = client.models.embed_content(
    model="gemini-embedding-2",
    contents=docs,
    config=types.EmbedContentConfig(output_dimensionality=768),
)
vectors = [e.values for e in result.embeddings]
# Store (doc, vector) pairs in your vector DB...
 
# Query
query = "What is the company's retention policy?"
q_result = client.models.embed_content(
    model="gemini-embedding-2",
    contents=[f"Retrieve documents matching: {query}"],
    config=types.EmbedContentConfig(output_dimensionality=768),
)
q_vec = q_result.embeddings[0].values
 
# (In your vector DB) cosine-similarity top-k
top_chunks = vector_db.search(q_vec, k=5)
 
# Synthesize
resp = client.models.generate_content(
    model="gemini-3.5-flash",
    contents=[
        "Answer the question using only the provided context. Cite chunk numbers.",
        *[f"[Chunk {i+1}] {c.text}" for i, c in enumerate(top_chunks)],
        f"Question: {query}",
    ],
)
print(resp.text)

11. Embedding for non-RAG use cases

  • Classification — embed examples + query, k-NN to class
  • Clustering — embed corpus, k-means / HDBSCAN over vectors
  • Semantic deduplication — embed records, find near-duplicates by cosine threshold
  • Recommendation — embed items + user history, find nearest items
  • Cross-lingual search — gemini-embedding-2 is multilingual; English query can match Japanese documents

12. Common pitfalls

  1. Mixing task types — indexing with RETRIEVAL_DOCUMENT then querying with the default mode loses 10-20 % retrieval quality. Be consistent.
  2. Re-indexing on model upgrade — gemini-embedding-001 → gemini-embedding-2 requires re-embedding all data. Plan the migration cost.
  3. Truncation without normalization — Matryoshka truncation produces an unnormalized vector; re-normalize before similarity ops.
  4. 3072 dim for everything — wastes storage + search time. 768 is enough for most RAG; 1536 / 3072 for marginal precision gains.
  5. Embedding long documents whole — gemini-embedding-2 max input is 8,192 tokens (down from full-doc fantasies); chunk first.
  6. Forgetting batch endpoint — batchEmbedContents is rate-limit-friendly and 50 % cheaper. Use it.
  7. Using sparse embeddings as cosine inputs — embeddings are dense L2-normalized floats. Don’t binarize or sparsify without a re-training step.
  8. Skipping the query intent prefix on gemini-embedding-2 — embeddings are quality-sensitive to task framing. “Retrieve documents matching:” + query is meaningfully better than raw query.

13. Comparison to other vendors

AspectGeminiOpenAIClaude
Current modelgemini-embedding-2 (multimodal)text-embedding-3-large (text-only)n/a — Anthropic doesn’t ship embedding models
MultimodalYesNo (text only)n/a
MatryoshkaYes (5 dim options)Yes (dimensions param, truncatable)n/a
Max input tokens8,1928,191n/a
Cross-modal queriesYes (text↔image↔audio↔video)Non/a
Task-type biasinggemini-embedding-001 only; embed-2 uses prompt instructionsNone (single embedding)n/a
Managed RAG serviceVertex AI SearchOpenAI File Search (Assistants/Responses)Claude has no equivalent

14. Further reading

Graph View