LLM Structured Outputs: OpenAI vs Anthropic vs Google

If you parse JSON from a language model in production, you have learned the hard lesson: the model returns almost valid output. Almost. One missing quote, one field with the wrong type, one stray comment, and your pipeline breaks at 2 a.m. The fix the three major vendors converged on is the same idea under different names: schema-enforced structured outputs. This guide compares how OpenAI, Anthropic, and Google implement them, where the gotchas hide, and what to check before you ship.

Verdict: Which Structured Output Setup to Pick

• Chat Completions or Responses API users on OpenAI: set response_format with strict: true. It is the most documented path, and OpenAI publishes the clearest comparison of what “JSON mode” covers versus what full schema enforcement covers.
• Claude-based agent stacks on Anthropic: use strict: true on tool definitions for guaranteed tool-input shapes, and output_config.format for structured final responses. Watch the token cost of the injected system prompt and the cache behavior.
• Google-stack teams on Gemini: use response_schema in the generation config. It produces schema-conformant JSON, but Google says explicitly to validate values yourself.

No vendor eliminates client-side validation. Every doc page says some version of “validate anyway,” and the reasons are concrete: schemas cover subsets of JSON Schema, model lists change fast, and semantically wrong values can still parse as valid JSON. I treat enforcement as one layer of a small reliability stack: schema on the API, validation in code, retry on failure.

Why Model Outputs Break Without Schema Enforcement

Anthropic’s own documentation lists what goes wrong “even with careful prompting”: malformed JSON syntax, missing required fields, and incompatible types. The classic example is a string "2" arriving where your code expects the number 2. Parsing breaks, downstream code throws, and retry logic you thought you would never need becomes the busiest part of your system.

Also Read:

Vector Databases: pgvector vs Pinecone vs Qdrant

OpenAI’s docs draw the line clearly with a comparison table between Structured Outputs and JSON mode. The key quote: “While both ensure valid JSON is produced, only Structured Outputs ensure schema adherence.” JSON mode guarantees the output parses. It does not guarantee the output matches any specific schema. If your code reads data.user.email and the model decided that field was optional, JSON mode will not save you.

This distinction matters more as prompts get complex. OpenAI’s launch write-up for the feature described evals of complex JSON schema following where gpt-4o-2024-08-06 with Structured Outputs scored a perfect 100%, while gpt-4-0613 scored under 40% on the same benchmark. Those numbers are OpenAI’s own internal evals, not an independent benchmark, so read them as direction rather than law. The pattern still matches what teams report in production: prompting harder helps, but it does not converge to reliable.

Anthropic adds one more angle I find useful when explaining this to product teams: the difference between format and content. Schema enforcement fixes the shape of the output. It cannot fix the content. A model can return perfectly valid JSON with confidently wrong values inside. That is why every section of this guide ends in the same place: enforce the schema, then validate the values yourself.

Also Read:

Foreman Review: Reliability Gates for Scheduled AI Agents

How Enforcement Actually Works: Constrained Decoding

Under the hood, the technique is constrained decoding, sometimes called grammar-constrained sampling. At each generation step, the API masks the model’s token vocabulary down to tokens that keep the output consistent with your schema. The model cannot emit a closing bracket where the schema demands a value, because that token never appears in the menu.

OpenAI’s launch blog described the approach as a “deterministic, engineering-based approach to constrain the model’s outputs,” added on top of training. The training-only path reached 93% on their benchmark, which they judged insufficient for developers building robust applications. That number tells the real story: even a model trained specifically for schema following still misses enough cases that engineering enforcement was worth building.

Anthropic describes the same trick for its strict tool use mode. From their docs: “Setting strict: true on a tool definition guarantees Claude’s tool inputs match your JSON Schema by constraining the model’s token sampling to schema-valid outputs (a technique called grammar-constrained sampling).” The guarantee wording is unusually strong for API documentation. Without strict mode, they warn, Claude might return the incompatible types mentioned above. With strict mode, the response always contains the schema-valid value, as in their example where the output holds passengers: 2 rather than passengers: "2".

Also Read:

RAG vs Fine-Tuning: When to Use Which

The practical implication is that enforcement happens at sampling time, not as a post-processing check. That is what makes the guarantees strong and also what explains the cost: the constraint computation is real work on top of generation, and some vendors surface that as token overhead or cache effects, which I cover below.

OpenAI vs Anthropic vs Google: Feature Comparison

Two cells in that table deserve emphasis. First, refusals: OpenAI documents that when the model declines, the refusal replaces the structured output rather than fitting inside your schema. Your parsing code needs a branch for that shape. Second, the “schema subset” row hiding behind the “unsupported schema handling” line. Every vendor supports a subset of JSON Schema, and the subsets differ. That is the next section.

The Schema Subset Problem

This is the gotcha that breaks teams moving a working schema from one vendor to another. You write a schema using full JSON Schema features, it works with vendor A, and vendor B rejects it with a 400. Nothing in your logic changed. The supported subset did.

Also Read:

Gemini Skills Explained: Build Your Own AI Assistant

OpenAI documents the constraints it adds and says plainly: “If you turn on Structured Outputs by supplying strict: true and call the API with an unsupported JSON Schema, you will receive an error.” The extra constraints it supports include minLength, maxLength, pattern, and format for strings; minimum, maximum, and multipleOf for numbers; patternProperties for objects; and minItems and maxItems for arrays.

Anthropic’s list is stricter in some places and looser in others, which is exactly the trap. Anthropic does not support numerical constraints like minimum, maximum, or multipleOf, nor string constraints like minLength and maxLength. Array constraints are limited to minItems of 0 or 1. Recursive schemas are out, and additionalProperties must be set to false for objects. Using any unsupported feature produces a 400 error with details. Notice the asymmetry: the numeric constraints OpenAI accepts are the ones Anthropic rejects. Porting the same schema verbatim across the two will fail.

Google’s wording is the least specific of the three. Its docs state “Schema subset: Not all JSON Schema features are supported,” and add that “very large or deeply nested schemas may be rejected.” That is honest but leaves the boundary fuzzy, so test your actual schema against the API rather than inferring from the docs.

The practical rule I use: write schemas against the intersection of the three subsets when you need portability. Set additionalProperties: false on every object (Anthropic requires it; the others tolerate it). Keep schemas shallow. Avoid numeric and string constraints if the schema might run on Anthropic. Enforce those value-level rules in your own validation code instead.

// Do: portable schema (intersection of vendor subsets)
{
  "type": "object",
  "properties": {
    "email": { "type": "string" },
    "age": { "type": "integer" }
  },
  "required": ["email"],
  "additionalProperties": false
}

// Don't: numeric constraints break on Anthropic's 400 rule
{
  "type": "object",
  "properties": {
    "age": { "type": "integer", "minimum": 0, "maximum": 150 }
  },
  "additionalProperties": false
}

The JSON Schema reference is worth keeping bookmarked for the full spec, precisely so you know which features to avoid in the portable subset.

Cost, Latency, and Cache Trade-offs

Schema enforcement is not free, and the costs differ by vendor.

Anthropic is the most transparent about the mechanism. When structured outputs are enabled, Claude receives an additional injected system prompt describing the expected output format. From their docs: “Your input token count is slightly higher. The injected prompt costs you tokens like any other system prompt.” For a single call that is negligible. Across millions of agent turns it shows up on the invoice. The same page also warns that changing the output_config.format parameter invalidates the prompt cache for that conversation thread. If your system iterates on schema shapes frequently, you lose caching benefits you might be counting on. The fix is operational: freeze the schema shape early, treat schema changes as deploys, and version them. This pairs well with versioning your LLM prompts like any other code artifact.

Anthropic documents batch processing for structured outputs at a 50% discount. Batch pricing changes often in this industry, so confirm the current pricing page before you budget around it. I avoid printing per-million-token tables in articles like this one because they expire within months; the vendor docs and pricing pages are the source of truth at implementation time. For context on whether your workload even needs a large model, why small models win in production covers the cost side of model choice.

OpenAI’s cost story is similar in shape. Their batch API carries documented discounts, and the constrained-decoding work is engineering overhead the vendor absorbs rather than bills per-token, but the practical guidance is the same: keep schemas stable, prefer batch for bulk extraction jobs, and measure with your own traffic. No vendor publishes a head-to-head latency delta for schema-constrained decoding versus plain generation, so if latency is your constraint, benchmark your own schemas on your own traffic rather than trusting vendor numbers. When the benchmark says the schema path is fine but the code around it is slow, testing AI-generated code before you ship has the broader discipline for validating generated outputs end to end.

One more trade-off belongs here because it surprises teams: streaming. You can stream structured outputs on all three vendors, but libraries that validate the final object cannot validate partial chunks. The Vercel AI SDK documents this explicitly: “Partial outputs streamed via streamText cannot be validated against your provided schema, as incomplete data may not yet conform to the expected structure.” Plan your UI accordingly. Show a loading state until the stream completes, or validate on completion only.

Final Checklist and Next Steps

Before any schema-enforced path goes to production, run through this list. Each item maps to a documented vendor behavior covered above.

1. Audit the schema against the subset. Confirm every keyword you use is in the supported subset for the vendor you call. If you target more than one vendor, write to the intersection.
2. Set additionalProperties: false on every object. Anthropic requires it; the rest tolerate it. It also blocks the model from inventing extra fields you never asked for.
3. Handle refusals. On OpenAI, a refusal replaces the structured payload. Your parser needs a branch for that shape, not just a JSON parse.
4. Validate values anyway. Schema enforcement guarantees shape, not content. Check types, ranges, and enums in your own code. Google’s docs say this explicitly: “always validate values in your application.”
5. Test with adversarial inputs. Feed the path inputs designed to make the model deviate: missing context, contradictory fields, long tails. Watch for schema-valid but semantically wrong output and add error handling for it. If you chain multiple model calls, prompt chaining for reliable output is a good companion pattern.
6. Keep the schema stable for caching. On Anthropic, changing the format invalidates the prompt cache. Version schemas like code and deploy changes deliberately.
7. Retry on 400s deliberately. A 400 on Anthropic or OpenAI usually means the schema used an unsupported feature, not that the model misbehaved. Log the schema, fix the keyword, and retry with the corrected subset rather than retrying the same request.

The broader lesson from comparing all three: enforcement converges on the same technique, and the real differences live in subset quirks, cache behavior, and tooling. Pick the vendor whose subset fits your schema, whose guarantees match your error budget, and whose cost structure fits your volume. Then validate everything downstream, because a schema guarantees the container, not the contents.

Structured outputs show up in two places where teams need extra care: data extraction feeding a database, where wrong values corrupt downstream systems, and agent tool calls, where wrong shapes trigger the wrong actions. The second one has real security implications, and AI agent security risks is a solid next read if your schemas drive actions. If you are still deciding whether structured generation or retrieval fits your data problem, deciding between RAG and fine-tuning covers that decision.

For deeper reading, start with the three vendor docs directly: OpenAI’s structured outputs guide, Anthropic’s structured outputs docs and its strict tool use reference, and Google’s structured output documentation. The archived OpenAI launch announcement preserves the original eval numbers for reference, and the Vercel AI SDK output reference documents the streaming validation caveat for library users.