Bedrock API and Converse

Bedrock exposes two API surfaces for inference. The legacy InvokeModel / InvokeModelWithResponseStream APIs take a model-specific request body (Anthropic’s Messages format, Meta’s chat template JSON, Cohere’s prompt object, etc.) — useful when you need a model-specific parameter that the unified API doesn’t expose, but a maintenance burden because every provider’s body is different. The modern Converse / ConverseStream APIs (GA since June 2024) accept a unified message + tool format that works identically across all supported models — write once, run on Claude or Llama or Nova. Use Converse unless you have a specific reason not to.

1. The two endpoints and clients

Per docs.aws.amazon.com/bedrock/latest/APIReference/:

bedrock-runtime — inference endpoint. Hosts InvokeModel, InvokeModelWithResponseStream, Converse, ConverseStream, and ApplyGuardrail.
bedrock — control-plane endpoint. Manages models, provisioned throughput, inference profiles, guardrails, custom model imports, model evaluation jobs.
bedrock-agent — control-plane for Agents, Knowledge Bases, Prompts, Flows.
bedrock-agent-runtime — invocation endpoint for Agents, Knowledge Bases (RetrieveAndGenerate, Retrieve), Flows (InvokeFlow).

import boto3
 
runtime = boto3.client("bedrock-runtime", region_name="us-east-1")
control = boto3.client("bedrock", region_name="us-east-1")
agent_control = boto3.client("bedrock-agent", region_name="us-east-1")
agent_runtime = boto3.client("bedrock-agent-runtime", region_name="us-east-1")

import { BedrockRuntimeClient } from "@aws-sdk/client-bedrock-runtime";
import { BedrockClient } from "@aws-sdk/client-bedrock";
import { BedrockAgentClient } from "@aws-sdk/client-bedrock-agent";
import { BedrockAgentRuntimeClient } from "@aws-sdk/client-bedrock-agent-runtime";
 
const runtime = new BedrockRuntimeClient({ region: "us-east-1" });

IAM permissions needed:

bedrock:InvokeModel for Converse and InvokeModel.
bedrock:InvokeModelWithResponseStream for ConverseStream and InvokeModelWithResponseStream.
bedrock:ApplyGuardrail for ApplyGuardrail.

2. Converse — the unified API

Per docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html.

Request fields

Field	Purpose
`modelId`	Foundation model ID, inference profile ID, application inference profile ARN, or prompt-management prompt ARN
`messages`	Array of `Message` (role: `user` / `assistant`, content blocks)
`system`	Array of `SystemContentBlock` — system prompts
`inferenceConfig`	Base inference params: `maxTokens`, `stopSequences`, `temperature`, `topP`
`additionalModelRequestFields`	Model-specific params (e.g. Claude’s `top_k`)
`additionalModelResponseFieldPaths`	JSON-pointer paths for extra response fields
`toolConfig`	Tool definitions + `toolChoice`
`guardrailConfig`	Apply a guardrail by `guardrailIdentifier` + `guardrailVersion`
`promptVariables`	When `modelId` is a Prompt Management ARN, fills in `{{var}}` placeholders
`serviceTier`	`default` / `flex` / `priority` / `reserved`
`requestMetadata`	Key-value tags written to invocation logs (CloudWatch + S3)

Content blocks (`ContentBlock`)

The content array of any Message can contain a mix of:

text — plain string
image — {format: "png"|"jpeg"|"gif"|"webp", source: {bytes: ...}} or {source: {s3Location: {uri, bucketOwner}}}
document — {format: "pdf"|"csv"|"docx"|"xls"|"html"|"txt"|"md", name, source} — must be accompanied by a text block describing the task
video — {format: "mp4"|"mov"|"webm"|"mkv"|"flv", source} (Nova Pro/Premier only)
toolUse — model-generated tool call: {toolUseId, name, input}
toolResult — your reply to a tool call: {toolUseId, content: [...], status: "success"|"error"}
cachePoint — {type: "default"} — marks a prompt-cache breakpoint
guardContent — wraps content for selective guardrail evaluation
reasoningContent — model’s reasoning trace (Claude extended thinking, Nova reasoning)

Minimal Python example

import boto3
 
runtime = boto3.client("bedrock-runtime", region_name="us-east-1")
 
response = runtime.converse(
    modelId="anthropic.claude-sonnet-4-6-20260119-v1:0",
    messages=[
        {"role": "user", "content": [{"text": "Explain CAP theorem in 50 words."}]}
    ],
    system=[{"text": "You are a distributed systems tutor."}],
    inferenceConfig={"maxTokens": 200, "temperature": 0.3},
)
 
print(response["output"]["message"]["content"][0]["text"])
print(f"Tokens: in={response['usage']['inputTokens']} "
      f"out={response['usage']['outputTokens']} "
      f"latency={response['metrics']['latencyMs']}ms")

Minimal TypeScript example

import {
  BedrockRuntimeClient,
  ConverseCommand,
} from "@aws-sdk/client-bedrock-runtime";
 
const client = new BedrockRuntimeClient({ region: "us-east-1" });
 
const response = await client.send(
  new ConverseCommand({
    modelId: "anthropic.claude-sonnet-4-6-20260119-v1:0",
    messages: [
      { role: "user", content: [{ text: "Explain CAP theorem in 50 words." }] },
    ],
    system: [{ text: "You are a distributed systems tutor." }],
    inferenceConfig: { maxTokens: 200, temperature: 0.3 },
  }),
);
 
console.log(response.output?.message?.content?.[0]?.text);

Multi-turn conversation pattern

Maintain messages as an append-only list. Add the assistant’s reply to the list before sending the next user message:

messages = []
 
def send(user_text):
    messages.append({"role": "user", "content": [{"text": user_text}]})
    resp = runtime.converse(
        modelId="anthropic.claude-sonnet-4-6-20260119-v1:0",
        messages=messages,
        inferenceConfig={"maxTokens": 1024},
    )
    assistant_message = resp["output"]["message"]
    messages.append(assistant_message)
    return assistant_message["content"][0]["text"]
 
print(send("List three pop songs from the UK."))
print(send("Now name the bands. Just bands, no songs."))

3. ConverseStream — streaming responses

Same request shape, returns an event-stream. Events emitted (in order):

messageStart — assistant role
contentBlockStart — only for tool-use blocks
contentBlockDelta — partial text, reasoningContent, or toolUse JSON
contentBlockStop — block boundary
messageStop — stopReason: end_turn, tool_use, max_tokens, stop_sequence, guardrail_intervened
metadata — final usage (inputTokens, outputTokens, totalTokens, cacheReadInputTokens, cacheWriteInputTokens) + metrics.latencyMs

response = runtime.converse_stream(
    modelId="anthropic.claude-sonnet-4-6-20260119-v1:0",
    messages=[{"role": "user", "content": [{"text": "Write a haiku."}]}],
)
 
for event in response["stream"]:
    if "contentBlockDelta" in event:
        delta = event["contentBlockDelta"]["delta"]
        if "text" in delta:
            print(delta["text"], end="", flush=True)
    elif "messageStop" in event:
        print(f"\n[stopReason={event['messageStop']['stopReason']}]")
    elif "metadata" in event:
        usage = event["metadata"]["usage"]
        print(f"[tokens: {usage['inputTokens']} in / {usage['outputTokens']} out]")

import { ConverseStreamCommand } from "@aws-sdk/client-bedrock-runtime";
 
const stream = await client.send(
  new ConverseStreamCommand({
    modelId: "anthropic.claude-sonnet-4-6-20260119-v1:0",
    messages: [{ role: "user", content: [{ text: "Write a haiku." }] }],
  }),
);
 
for await (const event of stream.stream ?? []) {
  if (event.contentBlockDelta?.delta?.text) {
    process.stdout.write(event.contentBlockDelta.delta.text);
  } else if (event.messageStop) {
    console.log(`\n[stop=${event.messageStop.stopReason}]`);
  }
}

4. Tool use — unified across providers

Per docs.aws.amazon.com/bedrock/latest/userguide/tool-use.html. The same toolConfig shape works for Claude, Llama 3.1+, Mistral, Cohere, and Nova. Bedrock translates to each provider’s native tool format internally.

tool_config = {
    "tools": [
        {
            "toolSpec": {
                "name": "get_weather",
                "description": "Return current weather for a city.",
                "inputSchema": {
                    "json": {
                        "type": "object",
                        "properties": {
                            "city": {"type": "string"},
                            "unit": {"type": "string", "enum": ["C", "F"]},
                        },
                        "required": ["city"],
                    }
                },
            }
        }
    ],
    # Optional — force a particular tool, or `auto` (default), or `any` (require some tool)
    "toolChoice": {"auto": {}},
}
 
# Turn 1 — model decides to call the tool
resp = runtime.converse(
    modelId="anthropic.claude-sonnet-4-6-20260119-v1:0",
    messages=[{"role": "user", "content": [{"text": "What's the weather in Paris?"}]}],
    toolConfig=tool_config,
)
 
assistant_msg = resp["output"]["message"]
# `stopReason` will be "tool_use"; content has both text and toolUse blocks
tool_use_block = next(
    b for b in assistant_msg["content"] if "toolUse" in b
)["toolUse"]
print(tool_use_block)
# {"toolUseId": "abc123", "name": "get_weather", "input": {"city": "Paris", "unit": "C"}}
 
# Run the tool locally (your code)
def get_weather(city, unit="C"):
    return {"temperature": 22, "conditions": "sunny"}
 
tool_result = get_weather(**tool_use_block["input"])
 
# Turn 2 — send tool result back
resp2 = runtime.converse(
    modelId="anthropic.claude-sonnet-4-6-20260119-v1:0",
    messages=[
        {"role": "user", "content": [{"text": "What's the weather in Paris?"}]},
        assistant_msg,
        {
            "role": "user",
            "content": [
                {
                    "toolResult": {
                        "toolUseId": tool_use_block["toolUseId"],
                        "content": [{"json": tool_result}],
                        "status": "success",
                    }
                }
            ],
        },
    ],
    toolConfig=tool_config,
)
 
print(resp2["output"]["message"]["content"][0]["text"])

toolChoice options:

{"auto": {}} (default) — model chooses whether to call a tool.
{"any": {}} — model must call some tool.
{"tool": {"name": "get_weather"}} — model must call this specific tool.

Provider quirks:

Claude: {"any": {}} and {"tool": {...}} error if extended thinking is enabled — per the Claude docs, “forced tool use” is incompatible with extended thinking.
Llama: tool definitions are injected into the system prompt under the hood. Don’t add a system block that conflicts with tool-use instructions or behavior degrades.
Mistral: parallel tool calls work but Mistral sometimes emits malformed JSON for nested objects — keep your inputSchema flat where possible.
Nova: only emits one tool call per turn (no parallel tool use as of 2026-05) — for parallel-tool workloads use Claude.

5. Prompt caching via `cachePoint`

Per docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html. Supported on Claude 3.5+, Nova Pro/Lite/Premier, and Amazon Titan models. The breakpoint mechanism is the same as Anthropic’s native API but lives in the content array as a cachePoint block.

# Cache a long document prefix so subsequent queries against the same doc are cheap
LONG_DOC = open("contract.pdf", "rb").read()
 
resp = runtime.converse(
    modelId="anthropic.claude-sonnet-4-6-20260119-v1:0",
    messages=[
        {
            "role": "user",
            "content": [
                {"text": "Here is a contract:"},
                {
                    "document": {
                        "format": "pdf",
                        "name": "Contract",
                        "source": {"bytes": LONG_DOC},
                    }
                },
                {"cachePoint": {"type": "default"}},   # ← cache up to and including the doc
                {"text": "Summarize clause 4(b)."},
            ],
        }
    ],
)
 
# Subsequent requests with the same prefix hit the cache:
# response["usage"]["cacheReadInputTokens"] > 0 means you got a cache hit
print(resp["usage"])
# {"inputTokens": 42000, "outputTokens": 180,
#  "cacheReadInputTokens": 0, "cacheWriteInputTokens": 41850, ...}

Cache mechanics:

5-minute TTL by default. Hits refresh the TTL.
Cache writes cost 1.25× base input price; reads cost 0.1× (90% discount).
Up to 4 explicit cachePoints per request.
Cache key includes modelId, system, tools, and every content block before the breakpoint. Any byte change invalidates.
Cross-region inference profile? Each underlying region has its own cache, so cache-hit rate degrades if your traffic is sharded across regions.

6. Guardrails on Converse

resp = runtime.converse(
    modelId="anthropic.claude-sonnet-4-6-20260119-v1:0",
    messages=[{"role": "user", "content": [{"text": "Tell me what stocks to buy."}]}],
    guardrailConfig={
        "guardrailIdentifier": "abc-123-def",
        "guardrailVersion": "DRAFT",   # or a version number string like "1"
        "trace": "enabled",            # include guardrail trace in response
    },
)
 
# Check if guardrail intervened
if resp["stopReason"] == "guardrail_intervened":
    print("Guardrail blocked or modified the response")
    print(resp["trace"]["guardrail"])  # detailed trace of which filter fired

To evaluate only specific content blocks (not entire prompt), wrap them in guardContent:

"messages": [{
    "role": "user",
    "content": [
        {"text": "Background: <safe context>"},
        {"text": "Question: <evaluate this>", "qualifiers": ["guard_content"]},
    ],
}]

See bedrock-guardrails for the full filter taxonomy.

7. Legacy `InvokeModel` (when you need it)

For features Converse doesn’t expose — e.g. Claude’s tools array cache_control field for fine-grained breakpoints on tool definitions, or Claude’s metadata.user_id for abuse rate-limiting — you fall back to InvokeModel and pass the provider-native body.

import json
 
body = {
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 1024,
    "system": [
        {"type": "text", "text": "You are an assistant."},
        # Native Anthropic cache_control on system prompt
        {"type": "text", "text": "[100k of context]", "cache_control": {"type": "ephemeral"}},
    ],
    "messages": [{"role": "user", "content": "Question?"}],
    "metadata": {"user_id": "user-7281"},   # ← only available via InvokeModel
}
 
resp = runtime.invoke_model(
    modelId="anthropic.claude-sonnet-4-6-20260119-v1:0",
    body=json.dumps(body),
    contentType="application/json",
)
 
result = json.loads(resp["body"].read())
print(result["content"][0]["text"])

For streaming, invoke_model_with_response_stream returns event chunks identical to Anthropic’s native streaming format (message_start, content_block_delta, etc).

8. Errors and retries

Per docs.aws.amazon.com/bedrock/latest/APIReference/CommonErrors.html:

Exception	When	Handling
`ThrottlingException`	Over the per-region per-minute or per-day quota	Exponential backoff. Switch to a cross-region inference profile to multiply quota.
`ValidationException`	Bad request shape, model not enabled in region, model deprecated	Fix the request; not retryable.
`ServiceUnavailableException`	Capacity issue on Bedrock side	Retry with backoff. Or switch model / region.
`ModelTimeoutException`	Model didn’t finish within the request timeout (default 60s, max 300s)	Lower `maxTokens` or break work up. Or switch to streaming.
`ModelErrorException`	Underlying provider returned an error	Often transient. Retry once; if persistent the model is down.
`AccessDeniedException`	Model access not granted in the Bedrock console	Go to Bedrock console → Model access → enable.

boto3’s standard retry config retries throttling 3× by default with exponential backoff:

import botocore.config
 
cfg = botocore.config.Config(
    retries={"max_attempts": 10, "mode": "adaptive"},  # adaptive uses client-side token bucket
    read_timeout=300,
)
runtime = boto3.client("bedrock-runtime", config=cfg)

9. Request metadata for invocation logging

Per docs.aws.amazon.com/bedrock/latest/userguide/cost-mgmt-request-metadata.html. Tag every request with arbitrary key-value pairs that show up in CloudWatch invocation logs and S3 model-invocation-logs:

resp = runtime.converse(
    modelId="anthropic.claude-sonnet-4-6-20260119-v1:0",
    messages=[...],
    requestMetadata={
        "team": "search",
        "feature": "result-summarizer",
        "user_segment": "premium",
    },
)

CloudWatch metric filters can then break down spend / token usage / latency by any of those dimensions. Use this for per-team cost allocation when application inference profiles are too coarse.

Compendium

Explorer

Bedrock API and Converse

Bedrock API and Converse

See also

1. The two endpoints and clients

2. Converse — the unified API

Request fields

Content blocks (`ContentBlock`)

Minimal Python example

Minimal TypeScript example

Multi-turn conversation pattern

3. ConverseStream — streaming responses

4. Tool use — unified across providers

5. Prompt caching via `cachePoint`

6. Guardrails on Converse

7. Legacy `InvokeModel` (when you need it)

8. Errors and retries

9. Request metadata for invocation logging

Further reading

Graph View

Table of Contents

Compendium

Explorer

Bedrock API and Converse

Bedrock API and Converse

See also

1. The two endpoints and clients

2. Converse — the unified API

Request fields

Content blocks (ContentBlock)

Minimal Python example

Minimal TypeScript example

Multi-turn conversation pattern

3. ConverseStream — streaming responses

4. Tool use — unified across providers

5. Prompt caching via cachePoint

6. Guardrails on Converse

7. Legacy InvokeModel (when you need it)

8. Errors and retries

9. Request metadata for invocation logging

Further reading

Graph View

Table of Contents

Content blocks (`ContentBlock`)

5. Prompt caching via `cachePoint`

7. Legacy `InvokeModel` (when you need it)