Structured output is a technique that constrains a language model’s response to a machine-readable format — typically JSON conforming to a declared schema — instead of free-form prose. Downstream code can parse fields directly, which makes model responses composable with ordinary software: validation, storage, routing, and further agent steps.
Why structured output matters
A language model’s native output is prose, but almost nothing downstream of an agent consumes prose. Tool arguments must be parsed, routing decisions must be compared against known branches, extracted records must land in database columns, and the next step of a workflow must read what the previous step produced. When a model answers in free text, the developer is left scraping it with string matching and hoping the phrasing never drifts — a failure mode that surfaces intermittently and resists reproduction.
The stakes compound in multi-step systems. A single malformed intermediate result — a missing field, a polite sentence prepended to the JSON — can derail an entire run, and the failure often appears several steps later, where it is expensive to trace back. Structured output turns format errors from runtime surprises into build-time contracts.
How it works
- The developer declares a schema — commonly JSON Schema, or a typed class in the host language that the framework converts to one.
- The schema travels with the generation request as part of the API call.
- The provider enforces it in one of two ways. With constrained decoding, the inference engine compiles the schema into a grammar and masks any token that would violate it during sampling, so the output is syntactically valid by construction. With post-hoc validation, the model generates freely and the client parses and checks the result, re-prompting when validation fails.
- The calling code receives a parsed, typed object rather than a string to interpret.
One caveat: a schema guarantees shape, not truth. A response can be perfectly valid JSON and still be wrong, so schema enforcement is a floor for reliability rather than a complete quality check.
Structured output vs. function calling
Both are schema-constrained generation, which is why providers implement them with the same underlying machinery, and the difference is what the schema describes. In function calling, the schema describes a tool’s parameters, and the model’s structured emission is a request for the application to act — pick this tool, with these arguments. In structured output, the schema describes the final deliverable itself: the model’s answer is the payload. Function calling selects and parameterizes an action; structured output shapes a result.
In practice
An execution layer typically lets a developer attach an output type to an agent definition, so each result is validated against the declared schema before the step completes and the next turn of the agent loop receives typed data instead of raw text. Structured results also make quality control composable: guardrails can assert on specific fields, and an LLM judge can return its verdict as a structured score that evaluation pipelines aggregate. See how typed outputs are declared on an agent in agents and output types.