Vision and multimodal

Every current Claude model is multimodal in: it accepts images and PDFs in the user message, produces text out. There is no image generation or audio I/O from Claude itself. This note covers image input (base64, URL, Files API), PDF support (text extraction + page-level citations), document blocks, all size and format limits, the image token-counting formula, and the practical patterns for getting good multimodal performance — including the ones the docs don’t explicitly call out.

1. Supported image formats

Per platform.claude.com/docs/en/build-with-claude/vision:

image/jpeg
image/png
image/gif
image/webp

Animated GIFs: only the first frame is processed. Other formats (tiff, heic, svg, etc.) → convert to one of the above first.

2. Image sourcing

Three ways to attach an image:

2.1 Base64

import base64
with open("photo.jpg", "rb") as f:
    image_data = base64.standard_b64encode(f.read()).decode("utf-8")
 
message = client.messages.create(
    model="claude-opus-4-7",
    max_tokens=1024,
    messages=[{
        "role": "user",
        "content": [
            {
                "type": "image",
                "source": {
                    "type": "base64",
                    "media_type": "image/jpeg",
                    "data": image_data,
                },
            },
            {"type": "text", "text": "Describe this photo."},
        ],
    }],
)

2.2 URL

{
    "type": "image",
    "source": {
        "type": "url",
        "url": "https://example.com/photo.jpg",
    },
}

Anthropic fetches the URL server-side. Must be publicly accessible. Useful for not bloating your request size.

2.3 Files API (preferred for reuse)

Upload once with the Files API (anthropic-beta: files-api-2025-04-14), reference by file_id:

uploaded = client.beta.files.upload(file=open("photo.jpg", "rb"), purpose="vision")
print(uploaded.id)  # file_011CNvxoj286tYUAZFiZMf1U
 
# Later, reference it
{
    "type": "image",
    "source": {
        "type": "file",
        "file_id": uploaded.id,
    },
}

Files persist for the retention period (per Files API docs, 30 days standard) and are billed only for storage (negligible) — your message payload stays tiny.

3. Size and count limits

Per platform.claude.com/docs/en/build-with-claude/vision:

Limit	Value
Max image dimensions	8000 × 8000 px (drops to 2000 × 2000 px if you send >20 images in one request)
Max images per message on claude.ai	20
Max images per API request — 200k-context models	100
Max images per API request — 1M-context models (Opus 4.7, 4.6, Sonnet 4.6)	600
Request body size	32 MB standard endpoints; lower on Bedrock / Vertex

For many large images: use the Files API by file_id. Even Files-API references can hit the request-size ceiling before 600 images if each is many MB at base64 — downsample first.

4. Image token costs

Per the vision docs, the token cost formula is:

tokens ≈ (width × height) / 750

Worked examples:

200×200 = 53 tokens
1000×1000 = 1,334 tokens
1200×1568 (about a US-letter scan) = 2,509 tokens
8000×8000 (the max) = 85,334 tokens

Practical: a typical screenshot at 1920×1080 ≈ 2,765 tokens. A photo at 4032×3024 (iPhone) ≈ 16,256 tokens — large enough that downsizing to ~1280px max edge saves real money.

Image-evaluation tip

Use count_tokens endpoint to pre-flight expensive multi-image requests:

count = client.messages.count_tokens(
    model="claude-opus-4-7",
    messages=[{"role": "user", "content": [{"type": "image", "source": {...}}]}],
)
print(count.input_tokens)

5. Multi-image patterns

Multiple images get analyzed jointly. Pattern:

{
    "role": "user",
    "content": [
        {"type": "image", "source": {...}},
        {"type": "text", "text": "Image 1:"},
        {"type": "image", "source": {...}},
        {"type": "text", "text": "Image 2:"},
        {"type": "text", "text": "Which has more flowers?"},
    ],
}

Interleaving text labels around images is a major prompt-engineering improvement — without labels Claude often confuses which image is which. Per Anthropic’s prompting docs, always caption images explicitly when using more than two.

6. PDF support

Per platform.claude.com/docs/en/build-with-claude/pdf-support. Beta header: pdfs-2024-09-25 (now GA on Claude 4 models — header still accepted, no-op).

6.1 As a document block

import base64
with open("paper.pdf", "rb") as f:
    pdf_data = base64.standard_b64encode(f.read()).decode("utf-8")
 
message = client.messages.create(
    model="claude-opus-4-7",
    max_tokens=2048,
    messages=[{
        "role": "user",
        "content": [
            {
                "type": "document",
                "source": {
                    "type": "base64",
                    "media_type": "application/pdf",
                    "data": pdf_data,
                },
                "title": "Research paper",
                "context": "ICML 2025 paper on attention",
                "citations": {"enabled": True},
            },
            {"type": "text", "text": "Summarize the key findings."},
        ],
    }],
)

6.2 As an image (one image per page)

The PDF block extracts text and sends an image of each page. Both modalities are available to the model:

The text is what citations reference (page-level).
The image is what the model sees for figures, tables, diagrams, handwriting.

Scanned PDFs that have no extractable text → only image perception works → citations not possible (or fall back to nothing). Run OCR first.

6.3 PDF limits

Max size: 32 MB per file (request body limit)
Max pages: 100 pages per request standard; up to 1,000 with the pdfs-2024-09-25 beta header on supported models
Token cost: text tokens + image tokens per page (each page is a separate image at ~1,500-3,000 tokens depending on resolution)
Format: PDF only — .docx, .xlsx, .csv, .md, .txt are not supported as document blocks; convert to text and include inline (or as plain-text document for citations)

7. Document blocks (broader than PDF)

The document content block accepts four source types:

`source.type`	Use for
`base64` (with `media_type: "application/pdf"`)	PDFs
`text` (with `media_type: "text/plain"`, `data: "..."`)	Plain text RAG chunks → sentence-chunked citations with `char_location` indices
`url` (with `media_type: "application/pdf"`)	PDFs from a public URL
`content` (with `content: [{type: "text", text: "..."}, ...]`)	Custom content; user-defined chunks → `content_block_location` citations
`file` (with `file_id: "file_…"`)	Files API references

The content source type gives the most control over citation granularity. Use it for transcripts, bullet lists, code snippets, or anything where sentence-chunking would split awkwardly.

{
    "type": "document",
    "source": {
        "type": "content",
        "content": [
            {"type": "text", "text": "Q3 revenue grew 12% YoY."},
            {"type": "text", "text": "Operating margin expanded to 23%."},
        ],
    },
    "title": "Q3 earnings call summary",
    "citations": {"enabled": True},
}

8. `title` and `context` fields

Both are optional, both go to the model, neither is citable:

title — short label, shown in citation references
context — longer free-text “this is a trustworthy primary source”, “this is summarized notes from a meeting”, etc. Use to tell Claude how to weight the document.

context is particularly useful for RAG with mixed-quality sources — annotate which docs are authoritative.

9. Vision tips for production

These come from Anthropic’s prompting docs plus community-discovered patterns:

Downsize before sending. A 1280×720 image is plenty for nearly every Claude task. Larger images cost more without gaining accuracy beyond that resolution. Use ImageMagick / Sharp / Pillow.
Use the Files API for repeat images. Logos, watermarks, template documents that appear in 1000s of requests — upload once, reference forever.
For tables / forms / receipts: pass both the image and a transcription of the text content if you have one — vastly improves accuracy.
Don’t ask Claude to “read text from this image” if there’s a lot of text. Run OCR first, send OCR output + image.
Coordinate-system math (“what’s at pixel 200,400?”) works poorly. Use bounding-box descriptions (“top-right quadrant”, “the second row”).
For computer use (per computer-use-and-code-execution): screenshots up to ~1280×800 work well; larger ones can confuse the model and burn tokens.
PII / sensitive content: Claude will refuse some image categories outright (CSAM, deeply identifying personal info in some contexts). Plan for refusals in your error handling.
No image generation. Claude does not output images. Pair with a separate generation model (DALL-E, Stable Diffusion, etc.) if needed.
No video, no audio. Multimodal in 2026 still means images + PDFs + text.

10. Files API quick reference

Per platform.claude.com/docs/en/build-with-claude/files. Beta header: files-api-2025-04-14.

Upload

uploaded = client.beta.files.upload(file=open("photo.jpg", "rb"), purpose="vision")
# purpose values: "vision", "code_execution", "assistants" (legacy), "fine_tune" (if available)

Reference by `file_id`

{"type": "image", "source": {"type": "file", "file_id": uploaded.id}}
{"type": "document", "source": {"type": "file", "file_id": uploaded.id}}

List / get / delete

files = client.beta.files.list()
metadata = client.beta.files.retrieve(uploaded.id)
content = client.beta.files.download(uploaded.id)  # bytes
client.beta.files.delete(uploaded.id)

Limits

Max file size: 32 MB (matches request-size limit)
Retention: 30 days (per docs, per file)
Storage cost: free up to some threshold then nominal — see pricing

11. Streaming with images

Streaming works identically with image content. Images count as input tokens, so no content_block_delta events for them — they’re part of the prefix, not the response. Output streams as usual.

12. Vision + tool use

Tools can return images in their tool_result.content:

{
    "type": "tool_result",
    "tool_use_id": "toolu_01",
    "content": [
        {"type": "text", "text": "Screenshot of current page:"},
        {"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": "..."}},
    ],
}

This is how computer use, browser-automation MCP servers, and screenshot-based UI testing tools work. Claude sees the image, reasons about UI state, decides next action.

13. Common gotchas

media_type must match the file. Sending PNG bytes with media_type: "image/jpeg" → silent corruption / 400 error.
Base64 must be standard, not URL-safe. URL-safe base64 (-, _) breaks the parser.
Animated GIFs: first frame only. Convert to PNG / JPEG explicitly to avoid surprises.
HEIC / HEIF (iPhone): not supported. Convert to JPEG.
WebP transparency: preserved, but Claude doesn’t gain anything from alpha — flatten if size matters.
PDF citations require text. Scanned PDFs without OCR have no citation source.
Image-only PDF pages (just a scan, no text layer): page acts as an image, not citable.
Image tokens count for caching minimums. A 4,000-token image hits the Opus 4,096-token cache threshold by itself.

14. Further reading

Vision overview
PDF support
Files API
Working with other file formats — convert docx/xlsx/etc. first
Citations — pairs natively with document blocks
Computer use — screenshot pipeline

Compendium

Explorer

Vision and multimodal

Vision and multimodal

See also

1. Supported image formats

2. Image sourcing

2.1 Base64

2.2 URL

2.3 Files API (preferred for reuse)

3. Size and count limits

4. Image token costs

Image-evaluation tip

5. Multi-image patterns

6. PDF support

6.1 As a document block

6.2 As an image (one image per page)

6.3 PDF limits

7. Document blocks (broader than PDF)

8. `title` and `context` fields

9. Vision tips for production

10. Files API quick reference

Upload

Reference by `file_id`

List / get / delete

Limits

11. Streaming with images

12. Vision + tool use

13. Common gotchas

14. Further reading

Graph View

Table of Contents

Backlinks

Compendium

Explorer

Vision and multimodal

Vision and multimodal

See also

1. Supported image formats

2. Image sourcing

2.1 Base64

2.2 URL

2.3 Files API (preferred for reuse)

3. Size and count limits

4. Image token costs

Image-evaluation tip

5. Multi-image patterns

6. PDF support

6.1 As a document block

6.2 As an image (one image per page)

6.3 PDF limits

7. Document blocks (broader than PDF)

8. title and context fields

9. Vision tips for production

10. Files API quick reference

Upload

Reference by file_id

List / get / delete

Limits

11. Streaming with images

12. Vision + tool use

13. Common gotchas

14. Further reading

Graph View

Table of Contents

Backlinks

8. `title` and `context` fields

Reference by `file_id`