Structured Output Patterns
Get machine-parseable results every time: explicit contracts, enum discipline, the NULL affordance, and validation-repair loops.
The problem with “return JSON”
Asking for JSON is not the same as getting JSON. You get prose wrapped in backticks, trailing commas, commentary before the object, or keys that almost match your schema.
Three moves fix nearly all of it.
1. State the contract, do not describe it
Show the exact shape, including types:
Return ONLY a JSON object matching this shape, no prose before or after:
{
"verdict": "pass" | "fail",
"severity": 1 | 2 | 3,
"issues": [{ "file": string, "line": number, "note": string }],
"confidence": number // 0.0 - 1.0
}
Enumerating "pass" | "fail" beats writing “one of pass or fail” — the model sees literal tokens to emit.
2. Give it permission to abstain
Without an escape hatch, models invent rather than admit uncertainty. Add an explicit option:
If the input does not contain enough information, return:
{ "verdict": "unknown", "missing": ["<what you would need>"] }
This single addition removes a large share of confident fabrication, because “I don’t know” becomes a legal answer.
3. Validate and repair, with a cap
attempt 1 → parse fail → retry with the error message appended
attempt 2 → parse fail → escalate or return null
Feed the parse error back verbatim (Unexpected token } at position 214) — models correct concrete errors well. Never loop indefinitely; two attempts then bail.
Choosing your enforcement level
| Approach | Reliability | Cost | Use when |
|---|---|---|---|
| Contract in prompt | Good | Free | Default start |
| Validate + retry | Very good | 1 extra call on failure | Majority of cases |
| Native JSON/schema mode | Near-perfect | Slight overhead | Strict pipelines |
Start with the prompt contract. Add validation immediately — it is cheap and catches the rest. Move to native schema enforcement only where a parse failure is unacceptable.
Layout tips that measurably help
- Put the format spec last, immediately before the input. Instructions at the end get followed more reliably.
- Repeat the field names in your prose, not just in the example — redundant cues help.
- Avoid deeply nested structures. Three levels is where accuracy starts dropping; flatten where you can.
- Prefer arrays of flat objects over objects keyed by arbitrary strings.
When not to use structured output
Chat responses, explanations, and anything a human reads directly do not need it. Forcing JSON into a conversational turn makes the answer worse and the integration no easier. Extract structure after the conversation, if you need it.