Structured output

Structured output constrains a model to answer in an exact machine-readable format — typically JSON that matches a schema you provide.

Prose is for people; software needs forms. A courier firm cannot dispatch parcels from "somewhere near the old post office in Baner, ask for Sharma-ji" — it needs pincode, phone, address lines in labelled boxes. Any model output that another program consumes has the same requirement, and "please reply in JSON" alone gets you compliance most of the time — with the failures arriving as midnight parsing crashes: a chatty preamble, a trailing comma, an invented field.

The reliable version is constrained decoding. The API takes your schema and enforces it during generation: at each step, tokens that would violate the schema are masked out of the logits before sampling, so an invalid character cannot be produced at all. The guarantee is grammatical, made at the level where text is chosen.

schema  : {"name": string, "pincode": string, "cod": boolean}
input   : "send it to Ramesh, Baner, 411045, cash on delivery"
output  : {"name": "Ramesh", "pincode": "411045", "cod": true}

Providers expose this as "JSON mode" or "structured outputs" (OpenAI, Anthropic, Gemini all do); local stacks get it from grammar-constrained decoding in llama.cpp, Outlines, or vLLM. One clear-eyed caveat: the schema guarantees shape, not truth — the pincode will always be a string, not always the right one. Validate meaning separately. Function-calling is this same machinery pointed at tool arguments.

Where to go next#

• Full lesson: Structured output
• Related terms: function-calling, logits, prompt, guardrails