Structured outputs are the integration contract

I treat every structured model response as an API boundary, not as decorated prose. Somewhere after generation, deterministic software must parse a value, choose a branch, write a record, or call another service. The schema is therefore the integration contract: it defines what the model may say at that seam, what the application will accept, and how failure must be handled. “Please return JSON” plus regex is not a contract. It is an optimistic parser wrapped around a probabilistic producer.

I enforce that contract in three layers, in a deliberate order. First comes constrained decoding. A JSON Schema is compiled into a grammar representation, and the decoder masks tokens that would violate the current grammar state. That moves structural correctness into generation rather than hoping to recover it afterward. Prompt-only approaches reach roughly 85–95 percent compliance, JSON mode about 95–99 percent, and strict schema enforcement around 99.9–100 percent. Modern grammar engines can add under 40 microseconds per token. The first request can still pay compilation latency, so schema reuse and cache behavior belong in the latency budget.

Second comes semantic validation with a bounded reask. Grammar enforcement can ensure that amount is a number; it cannot ensure that the number is positive, plausible, authorized, or consistent with another field. I validate the parsed object against domain rules, then return a field-level correction that names the field, expected constraint, and actual value. One retry resolves roughly 70 percent of semantic failures and two resolve about 90 percent, while field-level errors are 30–50 percent more effective than document-level complaints. The reask should preserve the original request and failed response, add the specific validation error, and request a corrected object. After two attempts, diminishing returns become a retry storm, not resilience.

Third comes repair, and only after constrained generation and reasking are unavailable or exhausted. A lenient parser can recover missing delimiters, unquoted keys, trailing commas, single quotes, language-specific literals, and some truncated values. Streaming-aware modes can handle partial output; strict mode can turn unrecoverable input into an explicit exception; returning objects directly avoids serializing a repaired value only to parse it again. These are useful recovery mechanics, but they must remain visible in telemetry. A repaired object is not equivalent to an originally valid object, especially when the repair had to infer a delimiter or type.

The contract also needs lifecycle and failure clauses. I version schemas in source control, review them like API changes, and treat additive fields differently from renames or removals that require migration. I detect stop reasons before parsing: refusal takes precedence over the requested schema, while token-limit truncation should be classified as incomplete generation rather than malformed business data. I record validation failures, retry counts, retry success, repair use, and terminal failures separately. A single “JSON error rate” hides which layer is weak and makes provider or schema regressions hard to locate.

One precise concession: heavy output constraints can degrade semantic quality even while improving formal compliance. Reported degradation reaches 10–30 percent under heavy constrained generation, and format-restricting instructions can cost 20–40 percent on complex reasoning tasks. For work that needs substantial reasoning, I let the model reason in natural language first and convert the result into the schema in a second stage; this can reduce the degradation below 5 percent. The boundary is precise: constrain the handoff, not every intermediate thought.

My production test is not whether the model can emit valid JSON in a demo. It is whether syntactically valid nonsense is rejected, corrected with useful feedback, and prevented from reaching a side effect; whether a refusal and a truncation take different paths; whether retry exhaustion ends cleanly; and whether repair is measurable rather than silent. Constrained decoding establishes shape, validation protects meaning, and repair salvages damaged syntax. Together they make the seam governable. Remove the ordering, and the escape hatch quietly becomes the architecture.