Structured outputs are the integration contract
Somewhere in every LLM-powered system there is a seam where probabilistic text meets deterministic software — a parser, a database insert, a downstream API call. That seam is an integration boundary, and integration boundaries need contracts. The schema is the contract. Parsing freeform model text with regex and optimism is signing that contract in pencil: it works in the demo, and it's an outage on a timer in production.
The numbers tell the story of how far this seam has been engineered. In the pre-2023 prompt-engineering era — "please respond in JSON" plus regex extraction — failure rates ran 5 to 20 percent. JSON mode, which guarantees syntax but not shape, gets you to roughly 95–99 percent. Strict schema enforcement via constrained decoding sits at 99.9–100. The mechanism behind that last number is genuinely elegant: the schema compiles to an automaton, and at every decoding step the engine masks away every token that could violate the grammar — modern engines like XGrammar do this in under 40 microseconds per token. The model cannot emit malformed output because malformed tokens are never sampleable. That is a contract enforced at the source, not checked after the fact.
Three layers, in order
Production systems layer the seam as defense-in-depth, and the ordering matters. Constrained decoding first, because it prevents structural failure outright. Validation-and-retry second, because valid structure can still carry wrong content — and here the practitioner numbers are usefully specific: one retry with the validation error fed back resolves about 70 percent of semantic failures, two resolve about 90, and field-level error messages ("amount must be positive, got -3") are 30–50 percent more effective than dumping a document-level error. This is the loop the Instructor library industrialised, at 3M+ monthly downloads. Repair last: lenient parsers like json_repair that patch trailing commas, unquoted keys, and truncated braces — strictly a fallback after the first two layers are exhausted, never a first line. A team whose plan is "we'll just repair the JSON" has built their contract entirely out of the escape hatch.
Treating the schema as a contract also means treating it as code. It belongs in version control with a changelog; additive changes are safe while renames and removals need versioned migrations — exactly the discipline of any API boundary, which is why this note is a sibling of data contracts. And the contract has failure clauses you must handle explicitly: a refusal stop reason overrides the schema (safety wins, and your parser must expect that), and max_tokens truncation hands you syntactically doomed output no grammar could save.
The honest counterweight: constraints are not free. Park et al. (NeurIPS 2024) measured 10–30 percent semantic quality degradation under heavy constraints, and format-restricting instructions can cost 20–40 percent on complex reasoning tasks. The mitigation is well understood — let the model reason in natural language first, then convert to the schema in a second stage, which cuts degradation below 5 percent — but it means the contract shouldn't be shoved into the middle of the thinking. Constrain the boundary, not the reasoning. And a schema guarantees shape, never truth: a perfectly valid response can be confidently wrong in every field, which is why the validation layer exists at all.
The test for whether you're taking the seam seriously: if the model returned syntactically perfect nonsense right now, would your system reject it, retry it with a useful error, or insert it into the database? Freeform parsing gives you the third answer by default. Contracts exist so that the default is refusal.