Compendium

Structured outputs (OpenAI)

8 min read

Structured outputs (OpenAI)

OpenAI ships two modes for structured output and the difference matters a lot: the older response_format: {type: "json_object"} is “loose JSON mode” — the model is told to produce valid JSON but the shape is its choice, with no enforcement beyond syntactic validity; the newer response_format: {type: "json_schema", json_schema: {..., strict: true}} is “strict mode” — constrained decoding guarantees the output matches a provided JSON Schema exactly. Strict mode shipped in late 2024 with gpt-4o-2024-08-06 and is now the recommended path for any extraction / classification / API-shaped response. The Python SDK adds Pydantic integration via .parse() helpers; the JS SDK adds Zod integration; raw JSON Schema works in any language.

See also

1. Two modes

Moderesponse_formatBehavior
JSON mode (loose){"type": "json_object"}Model produces valid JSON; shape is its choice. Must include “JSON” in the prompt.
Strict mode{"type": "json_schema", "json_schema": {"name": "...", "schema": {...}, "strict": true}}Constrained decoding; output guaranteed to match schema.

For new code, use strict mode. Loose JSON mode is a stepping stone from pre-strict-mode days and offers no guarantees beyond “the response will be parseable as JSON.”

2. Strict mode — basic shape

Chat Completions

client.chat.completions.create(
    model="gpt-5",
    messages=[
        {"role": "system", "content": "Extract entities from the text."},
        {"role": "user", "content": "John Doe, born 1985, lives in Paris."},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "person",
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "birth_year": {"type": "integer"},
                    "city": {"type": "string"},
                },
                "required": ["name", "birth_year", "city"],
                "additionalProperties": False,
            },
            "strict": True,
        },
    },
)

Responses API

client.responses.create(
    model="gpt-5",
    input="Extract entities: John Doe, born 1985, lives in Paris.",
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "person",
            "schema": {...},
            "strict": True,
        },
    },
)

3. Strict mode requirements

Per platform.openai.com/docs/guides/structured-outputs, strict-mode schemas must:

RequirementDetail
additionalProperties: falseOn every object — no extras allowed
All fields in requiredNo truly-optional fields; use null types for “missing”
No oneOf / allOfSchema unions not allowed in strict mode
anyOf — limitedAllowed in restricted form
No recursive $refSelf-referential schemas rejected
Total schema size limitSeveral hundred properties max
Total enum count limitA few thousand enum values max

Making fields “optional” with nullable types

{
    "type": "object",
    "properties": {
        "middle_name": {"type": ["string", "null"]},     # nullable
        "phone": {"type": ["string", "null"]},
    },
    "required": ["middle_name", "phone"],                # all in required
    "additionalProperties": False,
}

The model emits null when the value isn’t present in the input.

4. Pydantic integration (Python SDK)

The SDK provides .parse() helpers that take a Pydantic model and return a validated instance:

from pydantic import BaseModel
from openai import OpenAI
 
class Person(BaseModel):
    name: str
    birth_year: int
    city: str
 
client = OpenAI()
 
# Chat Completions
resp = client.chat.completions.parse(
    model="gpt-5",
    messages=[
        {"role": "system", "content": "Extract entity."},
        {"role": "user", "content": "John Doe, 1985, Paris."},
    ],
    response_format=Person,
)
person: Person = resp.choices[0].message.parsed
print(person.name, person.birth_year)
 
# Responses API
resp = client.responses.parse(
    model="gpt-5",
    input="John Doe, 1985, Paris.",
    text_format=Person,
)
person: Person = resp.output_parsed

The SDK auto-generates the JSON Schema from the Pydantic model, enables strict: true, validates the response, and returns the instance. If validation fails, raises LengthFinishReasonError or ContentFilterFinishReasonError.

Refusals

When the model refuses (per safety policy), .parsed is None and .refusal is populated:

if resp.choices[0].message.refusal:
    print(f"Refused: {resp.choices[0].message.refusal}")
else:
    person = resp.choices[0].message.parsed

Check .refusal before .parsed.

5. Zod integration (TypeScript SDK)

import OpenAI from "openai";
import { zodResponseFormat } from "openai/helpers/zod";
import { z } from "zod";
 
const Person = z.object({
  name: z.string(),
  birth_year: z.number().int(),
  city: z.string(),
});
 
const openai = new OpenAI();
const resp = await openai.chat.completions.parse({
  model: "gpt-5",
  messages: [
    { role: "system", content: "Extract entity." },
    { role: "user", content: "John Doe, 1985, Paris." },
  ],
  response_format: zodResponseFormat(Person, "person"),
});
 
const person = resp.choices[0].message.parsed;

6. Lists of objects

from pydantic import BaseModel
from typing import List
 
class Person(BaseModel):
    name: str
    age: int
 
class Roster(BaseModel):
    people: List[Person]
 
resp = client.chat.completions.parse(
    model="gpt-5",
    messages=[...],
    response_format=Roster,
)
roster: Roster = resp.choices[0].message.parsed
for p in roster.people:
    print(p.name, p.age)

The top-level schema in strict mode must be an object — wrap lists in a containing object as {"people": [...]}.

7. Enums for classification

from enum import Enum
from pydantic import BaseModel
 
class Sentiment(str, Enum):
    POSITIVE = "positive"
    NEUTRAL = "neutral"
    NEGATIVE = "negative"
 
class Result(BaseModel):
    sentiment: Sentiment
    confidence: float
 
resp = client.chat.completions.parse(
    model="gpt-5",
    messages=[
        {"role": "user", "content": "Classify: 'I love this product.'"},
    ],
    response_format=Result,
)
print(resp.choices[0].message.parsed.sentiment)   # Sentiment.POSITIVE

Strict mode enforces enums via constrained decoding — model literally cannot generate an out-of-enum value.

8. Streaming structured outputs

from openai.lib._parsing import partial_parse
 
with client.chat.completions.stream(
    model="gpt-5",
    messages=[...],
    response_format=Person,
) as stream:
    for event in stream:
        if event.type == "content.delta":
            print(event.delta, end="", flush=True)
            # event.parsed_partial — incrementally populated dict
        elif event.type == "content.done":
            print("\n[done]")
 
    final = stream.get_final_completion()
    person: Person = final.choices[0].message.parsed

Partial parsing lets you UI-render fields as they arrive (e.g., showing name while age is still streaming).

9. Loose JSON mode (legacy / fallback)

When the model doesn’t support strict mode (older models) or you want unconstrained JSON:

client.chat.completions.create(
    model="gpt-4-turbo",     # pre-strict era
    messages=[
        {"role": "system", "content": "Respond with valid JSON. ..."},
        {"role": "user", "content": "..."},
    ],
    response_format={"type": "json_object"},
)

Requirements for loose mode:

  • The word “JSON” must appear in the messages (system or user)
  • No schema enforcement — shape is the model’s choice
  • Always validate the parsed output (it’s syntactically valid JSON, not semantically what you want)

10. Schema gotchas

Pydantic + Optional

Pydantic v2 Optional[X] translates to ["X", "null"]. With Optional[str] = None:

class Foo(BaseModel):
    bar: Optional[str] = None        # type: ["string", "null"], default null

The SDK serializer adds it to required. Good.

Pydantic + Union

Union[str, int] translates to anyOf: [{type: "string"}, {type: "integer"}]. Strict mode has limited anyOf support — works for primitive unions but may fail for object unions. Test before relying on it.

Pydantic + Literal

from typing import Literal
class Foo(BaseModel):
    mode: Literal["fast", "thorough"]

Serializes to enum. Works in strict mode.

Nested Pydantic models

class Address(BaseModel):
    street: str
    city: str
 
class Person(BaseModel):
    name: str
    address: Address

Both classes need additionalProperties: false semantics. The SDK serializer handles this when you use Pydantic.

Recursive types

class Tree(BaseModel):
    value: int
    children: List["Tree"] = []     # recursive

Not supported in strict mode. Flatten:

class Node(BaseModel):
    id: int
    value: int
    parent_id: Optional[int]
 
class Forest(BaseModel):
    nodes: List[Node]

11. Strict-mode model support

ModelStrict mode
GPT-5 family✅
GPT-4o (2024-08-06 and later)✅
GPT-4o-mini (2024-07-18 and later)✅
o3 / o4-mini✅
o1 (later snapshots)✅
GPT-4 Turbo❌ — use loose json_object
GPT-3.5 Turbo❌ — use loose json_object

12. Cost considerations

Schema lives in the request — counts as input tokens. A typical extraction schema (10-20 fields) is 200-500 tokens. Schemas are cacheable (system-prompt portion of the cache prefix).

Constrained decoding has slight latency overhead — usually < 50 ms additional TTFT on modest schemas.

13. Combining with tools

You can have both tools[] and response_format set. The model decides: call a tool, or return a structured response. Useful pattern: “search the DB for matching customers, then return a structured summary.”

client.chat.completions.create(
    model="gpt-5",
    messages=[...],
    tools=[search_db_tool],
    response_format=Summary,
)

Common case: the model calls search_db, gets results, and then in the next turn returns structured Summary matching the schema.

14. Refusals

When the model refuses on safety grounds, the response object has a refusal field rather than parsed content:

choice = resp.choices[0].message
if choice.refusal:
    log_refusal(choice.refusal)
    # Decide: retry with rephrased prompt, surface to user, etc.
else:
    use(choice.parsed)

Refusals can happen even with strict: true — the model is constrained on the content space, not the decision to refuse.

15. Comparison to other vendors

AspectOpenAIClaudeGemini
API surfaceresponse_format: {type: "json_schema", strict: true}Tool with input_schema (no top-level response constraint)response_mime_type + response_schema
GuaranteeHard (constrained decoding)Soft (instruction-driven) + strict input on tool argsHard (constrained decoding)
Pydantic auto-serialization✅ (.parse() helpers)Via instructor library✅ (passing Pydantic model directly)
RecursionNot in strict modeLimitedNot supported
Enum constraintYes (constrained-decoded)Yes in tool argsYes (constrained-decoded)

16. Common pitfalls

  1. Forgetting "JSON" in the prompt for loose mode — model produces text not JSON.
  2. Forgetting additionalProperties: false — strict-mode rejection.
  3. Forgetting fields in required — strict-mode rejection (all must be required).
  4. Recursive Pydantic — strict mode rejects. Flatten the schema.
  5. Top-level array — strict mode requires top-level object. Wrap arrays.
  6. Trying to combine oneOf / allOf — not supported. Refactor to flat union of nullable fields.
  7. Refusals not checked — .parsed is None and .refusal is populated; surface to user.
  8. Strict mode on legacy models — GPT-4-turbo / GPT-3.5 don’t support it; falls back to error or worse, ignored.
  9. Large enums (thousands of values) — exceed the limit. Use external lookup instead.
  10. Schema mismatch in tool function vs response_format — the model has two different “structured output” targets and may confuse them. Be explicit about which schema applies to which.

17. Further reading

Graph View