OpenAI API and SDKs

OpenAI’s API surface in 2026 is centered on the Responses API (POST /v1/responses) — a unified, stateful, multimodal, tool-using endpoint designed for agentic workflows that replaces the older Assistants API and is now the primary recommendation in OpenAI’s docs. The legacy Chat Completions API (POST /v1/chat/completions) is still fully supported and remains in heavy production use; new builds should default to Responses unless they’re doing simple stateless single-turn completions. Both endpoints are spoken by the unified openai Python SDK (pip install openai, currently 2.x; v2.38+ as of May 2026) and openai Node SDK (npm install openai). Workload-identity auth is supported in both — Kubernetes service accounts, Azure managed identity, and GCP service accounts can mint short-lived tokens without the OPENAI_API_KEY env var.

See also

• openai-models-and-capabilities — model IDs the SDK calls into
• tool-use-and-function-calling-openai — tools parameter
• structured-outputs-openai — response_format parameter
• prompt-caching-openai — automatic; nothing in the SDK to opt into
• realtime-api-openai — Realtime is a separate WebSocket/WebRTC surface
• claude-api-and-sdks — sister doc for Claude
• gemini-api-and-sdks — sister doc for Gemini

1. Responses API vs Chat Completions — when to use which

Need	Use
Stateful, multimodal, tool-using, multi-turn agent	Responses
Hosted tools (web_search_preview, file_search, computer_use_preview, code_interpreter)	Responses (only)
Server-side conversation history (previous_response_id)	Responses
Reasoning models with full feature support (o3, o4-mini)	Responses
Single-turn classification / extraction / completion	Chat Completions (lighter weight)
Existing production code	Chat Completions (no need to migrate)
Highest portability across providers (most third-party libs target Chat Completions)	Chat Completions

The Responses API is a superset of Chat Completions in functionality — anything Chat Completions does, Responses can do, plus hosted tools, conversation state, parallel reasoning + tool execution, and streaming with richer event types.

2. Client initialization

Python

from openai import OpenAI, AsyncOpenAI
 
client = OpenAI()                              # picks up OPENAI_API_KEY env var
# Explicit
client = OpenAI(api_key="sk-...", organization="org-...", project="proj-...")
# Async
aclient = AsyncOpenAI()
# Workload identity (e.g. inside GKE / EKS / AKS)
client = OpenAI(workload_identity_provider="azure-managed-identity")

Node / TypeScript

import OpenAI from "openai";
const openai = new OpenAI();                   // picks up OPENAI_API_KEY
// Explicit
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

Azure OpenAI

from openai import AzureOpenAI
client = AzureOpenAI(
    azure_endpoint="https://<resource>.openai.azure.com",
    api_key="<azure-key>",                     # or use Azure AD
    api_version="2024-12-01-preview",
)
# Calls use deployment names instead of model IDs:
client.chat.completions.create(model="my-gpt-5-deployment", ...)

3. Responses API — basic call

resp = client.responses.create(
    model="gpt-5",
    input="Summarize the theory of relativity in one paragraph.",
)
print(resp.output_text)                        # the SDK convenience accessor
print(resp.usage.input_tokens, resp.usage.output_tokens)

input can be a string OR a structured list of Message objects (similar to messages in Chat Completions but a richer shape with multimodal parts).

With instructions (system prompt equivalent)

resp = client.responses.create(
    model="gpt-5",
    instructions="You are a concise physics tutor. Answer in two sentences.",
    input="What is general relativity?",
)

With conversation continuation

resp1 = client.responses.create(
    model="gpt-5",
    input="My name is Adam.",
    store=True,                                # store on the server
)
resp2 = client.responses.create(
    model="gpt-5",
    input="What's my name?",
    previous_response_id=resp1.id,             # server resumes context
)

previous_response_id makes the server stitch the conversation transparently — no need to send the full history client-side. Useful for long agentic sessions where the history is large.

Streaming

stream = client.responses.create(
    model="gpt-5",
    input="Tell me a story.",
    stream=True,
)
for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)
    elif event.type == "response.completed":
        print("\n[done]")

Streaming emits typed events — response.created, response.output_text.delta, response.output_text.done, response.function_call_arguments.delta, response.function_call.done, response.completed, response.failed, plus reasoning-specific events on o-series.

4. Responses API — full parameter surface

Parameter	Type	Purpose
model	string	Model ID, e.g. gpt-5
input	string | list	The user input — string or multimodal message list
instructions	string	System-level guidance
tools	list	Function tools + hosted tools
tool_choice	"auto" | "none" | "required" | {type, name}	Tool routing
parallel_tool_calls	bool	Default True; force serial with False
temperature	0-2	Sampling temperature
top_p	0-1	Nucleus sampling
max_output_tokens	int	Hard cap on output (incl. reasoning tokens on o-series)
reasoning	{effort: "low"|"medium"|"high"}	o-series reasoning depth
response_format	{type: "json_schema", ...}	Structured outputs (strict mode)
previous_response_id	string	Continue from a stored prior response
conversation	{id}	Use the new server-side conversation primitive
store	bool	Store the response on the server (for previous_response_id)
stream	bool	SSE streaming
metadata	dict	User-defined tags for observability
seed	int	(legacy) Deterministic sampling hint
prompt_cache_key	string	Route to cache shard (see prompt-caching-openai)
safety_identifier	string	Hashed user ID for safety analytics
truncation	"auto" | "disabled"	Auto-truncate input if it exceeds context
service_tier	"auto" | "default" | "flex"	Pricing/SLA tier

5. Responses API — response shape

{
  "id": "resp_abc123",
  "object": "response",
  "created_at": 1234567890,
  "status": "completed",                 // queued / in_progress / completed / incomplete / failed
  "model": "gpt-5",
  "output": [
    {
      "type": "message",
      "role": "assistant",
      "content": [
        {"type": "output_text", "text": "..."},
        {"type": "refusal", "refusal": "..."}
      ]
    },
    {
      "type": "function_call",
      "call_id": "fc_abc",
      "name": "get_weather",
      "arguments": "{\"city\":\"Paris\"}"
    },
    {
      "type": "reasoning",                // o-series only
      "summary": [{"type": "summary_text", "text": "..."}]
    }
  ],
  "output_text": "convenience accessor — flattens output_text parts",
  "previous_response_id": null,
  "usage": {
    "input_tokens": 50,
    "input_tokens_details": {"cached_tokens": 1024},
    "output_tokens": 200,
    "output_tokens_details": {"reasoning_tokens": 100},
    "total_tokens": 250
  }
}

The Python SDK exposes resp.output_text as a convenience accessor that joins all output_text parts. For function calls, iterate resp.output looking for type=="function_call" items.

6. Chat Completions API — basic call

The legacy / still-supported endpoint. Most third-party integrations target this shape.

resp = client.chat.completions.create(
    model="gpt-5",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hello"},
    ],
    temperature=0.7,
    max_tokens=500,
)
print(resp.choices[0].message.content)
print(resp.usage.prompt_tokens, resp.usage.completion_tokens)

Parameters specific to Chat Completions

• messages (instead of input / instructions)
• max_tokens (instead of max_output_tokens)
• prompt_tokens / completion_tokens in usage (instead of input_tokens / output_tokens)
• n for multiple choices (Responses doesn’t support n)
• logprobs / top_logprobs for token probabilities (Responses doesn’t expose these)
• frequency_penalty / presence_penalty (only on Chat Completions)
• logit_bias (only on Chat Completions)
• user (per-call user ID; Responses uses safety_identifier)
• service_tier ("auto" / "default" / "flex" / "scale")
• stream_options.include_usage for usage in the last streaming chunk

Streaming

stream = client.chat.completions.create(
    model="gpt-5",
    messages=[...],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print(f"\n[Tokens: in={chunk.usage.prompt_tokens} out={chunk.usage.completion_tokens}]")

7. Multimodal input

Vision (image input)

# Responses API
resp = client.responses.create(
    model="gpt-5",
    input=[
        {
            "role": "user",
            "content": [
                {"type": "input_text", "text": "What's in this image?"},
                {"type": "input_image", "image_url": "https://example.com/cat.jpg"},
            ],
        }
    ],
)

# Chat Completions equivalent (note different content-type names)
resp = client.chat.completions.create(
    model="gpt-5",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What's in this image?"},
                {"type": "image_url",
                 "image_url": {"url": "https://example.com/cat.jpg",
                               "detail": "high"}},
            ],
        }
    ],
)

See vision-and-multimodal-openai for full details (detail levels, sizing, base64 encoding).

Audio input/output (GPT-4o-audio family)

import base64
audio_b64 = base64.b64encode(open("input.wav", "rb").read()).decode()
 
resp = client.chat.completions.create(
    model="gpt-4o-audio-preview",
    modalities=["text", "audio"],
    audio={"voice": "alloy", "format": "wav"},
    messages=[
        {"role": "user", "content": [
            {"type": "input_audio", "input_audio": {"data": audio_b64, "format": "wav"}},
        ]},
    ],
)
audio_out = base64.b64decode(resp.choices[0].message.audio.data)

For low-latency bidirectional audio, use the Realtime API — see realtime-api-openai.

8. Async usage

import asyncio
from openai import AsyncOpenAI
 
async def main():
    aclient = AsyncOpenAI()
    resp = await aclient.responses.create(
        model="gpt-5", input="Hello",
    )
    print(resp.output_text)
 
asyncio.run(main())

Node SDK is async-native — every call returns a Promise.

9. Error handling

from openai import OpenAI
from openai import (
    APIError, APIConnectionError, AuthenticationError, BadRequestError,
    RateLimitError, InternalServerError, NotFoundError, PermissionDeniedError,
    UnprocessableEntityError, APITimeoutError,
)
 
try:
    resp = client.responses.create(...)
except RateLimitError as e:
    # 429 — back off + retry
    print(e.response.headers.get("retry-after"))
except BadRequestError as e:
    # 400 — schema problem
    print(e.body)
except APITimeoutError:
    # client-side timeout
    pass
except APIConnectionError:
    # network
    pass

The SDK retries 408 / 429 / 5xx automatically with exponential backoff. Configure:

client = OpenAI(max_retries=3, timeout=30.0)

10. File API + uploads

# Upload (for fine-tuning, batch, file_search vector stores, etc.)
f = client.files.create(
    file=open("training.jsonl", "rb"),
    purpose="fine-tune",                   # or "batch", "assistants", "vision"
)
print(f.id)                                # file-abc123
 
# List / retrieve / delete
for f in client.files.list():
    print(f.id, f.filename, f.purpose)
client.files.retrieve(f.id)
client.files.delete(f.id)
 
# Resumable upload (large files)
upload = client.uploads.create(
    purpose="batch",
    bytes=1_500_000_000,                   # 1.5 GB
    filename="huge-batch.jsonl",
    mime_type="application/jsonl",
)
# ... upload parts, then complete ...

Files persist until you delete them (different from Gemini’s 48h auto-expire).

11. Vector stores (for file_search hosted tool)

vs = client.vector_stores.create(name="My Knowledge Base")
client.vector_stores.files.create(vector_store_id=vs.id, file_id=f.id)
# Or batched:
client.vector_stores.file_batches.create(vector_store_id=vs.id, file_ids=[...])

Reference the vector store in a Responses tool call:

client.responses.create(
    model="gpt-5",
    input="Find info about X.",
    tools=[{"type": "file_search", "vector_store_ids": [vs.id]}],
)

See tool-use-and-function-calling-openai.

12. Embeddings

result = client.embeddings.create(
    model="text-embedding-3-large",
    input=["chunk 1", "chunk 2"],
    dimensions=1024,                       # Matryoshka truncation
    encoding_format="float",               # or "base64" for compact wire format
)
for emb in result.data:
    print(len(emb.embedding))
print(result.usage.total_tokens)

13. Audio (transcription + TTS)

# Speech-to-text (Whisper)
transcript = client.audio.transcriptions.create(
    file=open("recording.mp3", "rb"),
    model="whisper-1",
    response_format="json",                # or "verbose_json" / "srt" / "vtt" / "text"
    language="en",
)
print(transcript.text)
 
# Translation (any-language audio -> English text)
translation = client.audio.translations.create(
    file=open("japanese.mp3", "rb"),
    model="whisper-1",
)
 
# Text-to-speech
speech = client.audio.speech.create(
    model="tts-1-hd",
    voice="nova",
    input="Hello world",
    response_format="mp3",
    speed=1.0,
)
with open("out.mp3", "wb") as f:
    f.write(speech.read())

14. Image generation / edit

img = client.images.generate(
    model="dall-e-3",
    prompt="A cat astronaut on the moon, vivid style.",
    size="1024x1024",
    quality="hd",
    n=1,
)
url = img.data[0].url
 
# Edit (gpt-image-1 supports multi-input)
edit = client.images.edit(
    model="gpt-image-1",
    image=open("original.png", "rb"),
    mask=open("mask.png", "rb"),
    prompt="Replace the sky with stars.",
)

See vision-and-multimodal-openai for the full image surface.

15. Moderation

result = client.moderations.create(
    model="omni-moderation-latest",
    input="text or list of strings",
)
print(result.results[0].flagged)
print(result.results[0].categories)        # dict of category -> bool
print(result.results[0].category_scores)   # dict of category -> float

Free — does not count against billing.

16. Batch API

50 % discount, 24h SLA, JSONL input. See batch-and-fine-tuning-openai for details. Surface:

# Upload input JSONL
batch_file = client.files.create(file=open("requests.jsonl", "rb"), purpose="batch")
# Submit batch
batch = client.batches.create(
    input_file_id=batch_file.id,
    endpoint="/v1/chat/completions",       # or "/v1/responses", "/v1/embeddings", etc.
    completion_window="24h",
)
# Poll
while client.batches.retrieve(batch.id).status not in {"completed", "failed", "cancelled", "expired"}:
    time.sleep(60)
# Download output
output_file_id = client.batches.retrieve(batch.id).output_file_id
content = client.files.content(output_file_id).read()

17. Fine-tuning

job = client.fine_tuning.jobs.create(
    training_file="file-train",
    validation_file="file-val",
    model="gpt-4o-mini-2024-07-18",
    hyperparameters={"n_epochs": 3, "learning_rate_multiplier": 1.0},
    suffix="my-domain",
)
# Poll job.status; checkpoints listed via client.fine_tuning.jobs.checkpoints.list(job.id)
# After completion, call the new model:
client.chat.completions.create(model=job.fine_tuned_model, messages=[...])

See batch-and-fine-tuning-openai.

18. Beta resources (Assistants legacy + Realtime)

# Assistants API (now de-emphasized in favor of Responses)
asst = client.beta.assistants.create(...)
thread = client.beta.threads.create()
client.beta.threads.messages.create(thread.id, role="user", content="...")
run = client.beta.threads.runs.create(thread.id, assistant_id=asst.id)
# Poll run.status
 
# Realtime session (returns ephemeral token + WebSocket URL for client)
session = client.beta.realtime.sessions.create(
    model="gpt-realtime-2",
    modalities=["text", "audio"],
    instructions="You are a friendly voice assistant.",
)

See realtime-api-openai for the full Realtime flow.

19. REST API directly

curl https://api.openai.com/v1/responses \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5",
    "input": "Hello",
    "instructions": "You are concise."
  }'

Headers commonly used:

• OpenAI-Organization: org-... — select organization
• OpenAI-Project: proj-... — select project
• OpenAI-Beta: ... — beta-feature flags (assistants, etc.)

20. Rate limits + tiers

Per platform.openai.com/docs/guides/rate-limits:

Limit	Description
RPM	Requests per minute (per model)
TPM	Tokens per minute (per model)
RPD	Requests per day (per model)
TPD	Tokens per day (per model)

Spend-based tiers (Tier 1 / 2 / 3 / 4 / 5) unlock higher limits as cumulative spend increases. Free tier and Tier 1 are very tight on GPT-5 / o-series — production workloads typically need Tier 3+.

service_tier parameter trade-offs:

• "default" — best quality, standard latency
• "flex" — 50 % off for some models, relaxed SLA (slower, may queue)
• "scale" — premium for guaranteed latency / capacity (Enterprise)
• "auto" — server picks

21. Migration from Chat Completions to Responses

Chat Completions	Responses
messages=[...]	input=[...] or simple string
messages[0] with role: "system"	instructions=... (top-level)
max_tokens	max_output_tokens
choices[0].message.content	output_text (convenience) or scan output[]
usage.prompt_tokens / completion_tokens	usage.input_tokens / output_tokens
Manage history client-side	previous_response_id (server stitches)
tools=[{type: "function", function: {...}}]	tools=[{type: "function", name, parameters}] (flatter shape)
tool_call_id	call_id
Stream chunks via choices[].delta	Typed events (response.output_text.delta, etc.)

The Python SDK provides .parse() helpers on both APIs for structured outputs:

resp = client.responses.parse(model="gpt-5", input="...", text_format=MyPydanticModel)
parsed: MyPydanticModel = resp.output_parsed

22. Common pitfalls

1. Forgetting store=True when planning to use previous_response_id — without store, the server doesn’t retain the conversation and continuation fails.
2. Mixing Responses + Chat Completions tool shapes — the tool schema is slightly different. Responses tools are flatter ({type, name, parameters} vs {type: "function", function: {name, parameters}}).
3. Token counts in different fields — Chat Completions uses prompt_tokens, Responses uses input_tokens. Migration code that aggregates usage across both must normalize.
4. max_tokens on o-series — limits visible output but not reasoning tokens; an o3 call with max_output_tokens=100 can still burn 20k+ reasoning tokens. Budget accordingly.
5. Streaming on o1 — o1 / o1-pro don’t stream tokens; the response arrives atomic. Plan UX accordingly (or use o3 / o4-mini for streaming).
6. tool_choice: "required" doesn’t guarantee specific tool — it just forces some tool call. To force a specific tool, use tool_choice: {type: "function", function: {name: "..."}}.
7. Mixing previous_response_id with manual message history — pick one. They conflict.
8. response_format: json_object (loose) vs json_schema (strict) — loose JSON mode doesn’t enforce a schema; strict mode does. See structured-outputs-openai.
9. Default parallel_tool_calls: True on Chat Completions — older code that assumed serial tool calls breaks. Set explicitly.
10. Azure deployments != model IDs — Azure OpenAI calls use the deployment name, not the OpenAI model ID. Easy to mix up between environments.

23. Comparison to other vendors

Aspect	OpenAI	Anthropic Claude	Google Gemini
Primary endpoint	/v1/responses (Responses)	/v1/messages (Messages)	:generateContent (Gemini API)
Stateful conversations	previous_response_id (server-side)	Client manages history	client.chats.* (client-side)
Hosted tools	file_search, web_search_preview, computer_use_preview, code_interpreter	web_search_*, web_fetch_*, code_execution_*, computer_*, memory_*	google_search, code_execution, url_context
Multimodal input	text + image + audio (Realtime)	text + image + PDF	text + image + audio + video + PDF
Workload identity	Yes (K8s / Azure / GCP)	No (API key)	Yes (Vertex IAM / ADC)
Cloud mirror	Azure OpenAI	AWS Bedrock + Claude Platform on AWS, Vertex, Foundry	Vertex AI

24. Further reading

• Responses API guide — canonical reference
• Chat Completions guide — legacy primary
• openai-python — SDK source
• openai-node — JS SDK source
• API reference — full schema
• Rate limits — tier matrix
• Errors & retries — error codes