Home → Help
JSON mode still returns text that will not parse
Plain JSON mode only promises syntactically valid JSON when the model runs to completion. It does not promise your schema, and it cannot promise anything at all if generation stops early on max_tokens.
What you are seeing
JSONDecodeError on output that looks almost rightThe object is cut off mid-stringMarkdown fences around the JSONValid JSON that is missing fields your code requires
Why it happens
Hitting the output limit truncates the object mid-token. The result is unparseable through no fault of the prompt — check finish_reason before you check the parser.
Basic JSON mode has no schema. The model may return valid JSON with different keys, or with a nested shape you did not expect, and it will parse cleanly while breaking your code.
Some models wrap output in markdown fences when the prompt contains examples formatted that way. The body is fine; the wrapper is what fails.
How to fix it
- Check finish_reason before parsingA finish_reason of length means truncation. Raise max_tokens or ask for a smaller object rather than debugging the parser.
- Use schema-enforced output where the model supports itPassing a JSON schema, rather than asking for JSON in prose, is what actually constrains the keys. Validate the result anyway.
- Strip fences defensivelyA three-line pre-parse that removes leading and trailing code fences costs nothing and removes a whole class of failure.
Truncated output is still generated output, so it is still billed on APICLAN. Reading finish_reason and sizing max_tokens properly saves the retry, which is the actual cost.
Related
Last checked 2026-10-01. Written from problems diagnosed on a live
OpenAI-compatible gateway, not collected from other sites.