Why Raw Text Outputs Break Pipelines
Asking a model to “return JSON” in a prompt is unreliable. It can wrap the answer in markdown, add commentary around it, or invent keys. When your code needs typed data, two mechanisms exist to get it reliably: function calling and structured-output modes like JSON schema.
Function Calling in One Paragraph
You declare the functions the model may use, each with a name and a JSON-schema description of its parameters. Instead of replying in free text, the model emits a structured request to call one of your functions with arguments that match the schema. Your code executes the call and sends the result back, which is how agents chain tools.
JSON Mode for Plain Answers
When you do not need an actual function, JSON mode constrains the model to produce valid JSON that matches a schema you supply. This is the reliable way to extract fields, classify text, or generate config files without prompt-engineered formatting.
Validation Belongs in Your Code
Even with these modes, treat the output as untrusted input. Validate against the schema, coerce types, and have a fallback path for malformed output. Models occasionally produce a valid-JSON-but-wrong answer, which no schema enforcement can catch.
The Practical Pattern
- Declare a strict schema for every structured output you need.
- Route the request through function calling or JSON mode rather than asking nicely in prose.
- Validate the result in code before using it.
- Log schema failures and retry with a clarified prompt.
Teams that adopt this pattern stop debugging regex over model text and start shipping features.
Comments (167)