A Guide To Structured Output With Examples By Michał Nasternak Medium

Emily Johnson

-Mar 12, 2026, 8:04 PM

a guide to structured output with examples by michał nasternak medium

About LLM structured output, it seems like everything has been said already. We’ve got JSON schemas, Pydantic models, and enough Medium articles to fill a small library. Yet here’s what rarely gets mentioned at AI conferences (where a MacBook plastered with brain illustration stickers is practically the entry fee): you can use examples to force LLMs to shape their responses to... Few-shot examples represent a powerful alternative to model fine-tuning that eliminates the need for parameter optimization, dataset curation, and computational overhead. They provide immediate control over model behavior without requiring infrastructure changes or additional training cycles. Examples serve as behavioral constraints that guide the model toward desired output patterns while maintaining inference efficiency.

This approach proves particularly valuable in production environments where consistency and predictability are essential — because nothing ruins your day quite like an AI that decides to get creative with your API responses. It’s not only about the right format, what if you can show the model how you expect the reasoning behind the output (like chain-of-thought)? Instead of just demanding the right JSON structure, you’re essentially saying, “Hey, I want to see your mental math too.” It’s like having a transparent AI that can’t help but explain how it arrived... Field-like examples provide granular control over individual output components. Complete object examples demonstrate full-context relationships between fields and establish structural patterns for complex outputs. Structured Outputs is a powerful feature in OpenAI’s API that ensures responses adhere to a predefined JSON schema.

This feature is particularly useful for developers who need reliable, type-safe, and structured data from the model. In this blog, we’ll explore the key aspects of Structured Outputs, its benefits, use cases, and how to implement it effectively. Structured Outputs ensure that the model generates responses conforming to a supplied JSON schema. This eliminates concerns about missing keys, invalid values, or inconsistent formatting. By enforcing schema adherence, Structured Outputs provide a robust way to handle data exchange between applications and the model. Structured Outputs are ideal for scenarios where you need the model to respond in a specific format.

For example: Define the structure of the response using JSON Schema or a schema definition library like Pydantic (Python) or Zod (JavaScript). Pass the schema to the API using the response_format parameter. Structured Outputs is a new capability in the Chat Completions API and Assistants API that guarantees the model will always generate responses that adhere to your supplied JSON Schema. In this cookbook, we will illustrate this capability with a few examples. Structured Outputs can be enabled by setting the parameter strict: true in an API call with either a defined response format or function definitions.

Previously, the response_format parameter was only available to specify that the model should return a valid JSON. In addition to this, we are introducing a new way of specifying which JSON schema to follow. Function calling remains similar, but with the new parameter strict: true, you can now ensure that the schema provided for the functions is strictly followed. How can you ensure your large language models (LLMs) consistently produce outputs in the correct format - especially when performing tasks or calling APIs? What can you do to prevent your LLM from generating unpredictable or incomplete responses that can break your application? LLMs often produce unstructured outputs that require extensive post-processing before they can be used effectively.

This unpredictability leads to errors, wasted time, and increased costs. OpenAI and Google introduced structured outputs to solve this problem. Structured outputs ensure model responses follow a strict format, reduce errors and make it easier to integrate LLMs into applications requiring consistent, machine-readable data. This guide will explain how structured outputs work, how they can be implemented in OpenAI and Gemini models, the benefits they offer, and what challenges you could face when using them. Structured outputs ensure model-generated responses follow pre-defined formats, such as JSON, XML or Markdown. Previously, LLMs would generate responses in free-form text that doesn’t have a specific structure.

Instead, structured outputs provide an alternative where outputs are machine readable, consistent, and can be easily integrated into other systems. LLMs excel at generating text, but robust applications need clean, predictable JSON. Simply prompting for JSON is risky. Even GPT-4o fails roughly 1 in 7 times with broken formatting or missing fields. There are several ways to obtain more robust and reliable structured outputs. But what are the trade-offs, and when should each be used?

These methods can often feel like black boxes. Stick with me. By the end of this post, you’ll be your team’s go-to expert for getting structured data from LLMs. We’re opening up that black box. The most dependable way to get an AI model to produce data in the specific structure you need is by using a built-in structured output feature. We’ll call this Native Structured Output (NSO).

Major models from providers like OpenAI achieve 100% accuracy with two main techniques: This section provides examples of common use cases for working with OpenAI Structured Outputs using the openai-structured library. Extract structured movie reviews using OpenAI Structured Outputs with streaming: Analyze code using OpenAI Structured Outputs with custom rules and streaming: Configure buffer settings for different OpenAI Structured Outputs use cases: Use different models with version validation:

In the realm of AI-driven applications, ensuring consistent and predictable outputs is paramount. OpenAI’s introduction of Structured Outputs addresses this need by allowing developers to define the exact format of the model’s responses, ensuring they adhere to specified schemas. Structured Outputs enable developers to constrain the model’s responses to a predefined structure, typically defined using JSON Schema. This ensures that the outputs are not only valid JSON but also match the expected format, reducing the need for post-processing and error handling. 🔧 Using pydanticmodel with text_format under the method called client.responses.parse Step 1: Define the pydantic model in models/document_extraction.py

Step 2: Create a main.py and run the following code. You will have the structured output in JSON format. Learn the basics of structured LLM outputs with Instructor. This guide demonstrates how to extract consistent, validated data from language models.Large language models are powerful, but extracting structured data can be challenging.Structured outputs solve this by having LLMs return data in consistent, machine-readable... OpenAI recently announced support for Structured Outputs in its latest gpt-4o-2024–08–06 models. Structured outputs in relation to large language models (LLMs) are nothing new — developers have either used various prompt engineering techniques, or 3rd party tools.

In this article we will explain what structured outputs are, how they work, and how you can apply them in your own LLM based applications. Although OpenAI’s announcement makes it quite easy to implement using their APIs (as we will demonstrate here), you may want to instead opt for the open source Outlines package (maintained by the lovely folks... Mistral and LLaMA), as well as the proprietary APIs (Disclaimer: due to this issue Outlines does not as of this writing support structured JSON generation via OpenAI APIs; but that will change soon!). If RedPajama dataset is any indication, the overwhelming majority of pre-training data is human text. Therefore “natural language” is the native domain of LLMs — both in the input, as well as the output. When we build applications however, we would like to use machine-readable formal structures or schemas to encapsulate our data input/output.

This way we build robustness and determinism into our applications. Structured Outputs is a mechanism by which we enforce a pre-defined schema on the LLM output. This typically means that we enforce a JSON schema, however it is not limited to JSON only — we could in principle enforce XML, Markdown, or a completely custom-made schema. The benefits of Structured Outputs are two-fold: For this example, we will use the first sentence from Sam Altman’s Wikipedia entry…

A Guide To Structured Output With Examples By Michał Nasternak Medium

People Also Search

About LLM Structured Output, It Seems Like Everything Has Been

This Approach Proves Particularly Valuable In Production Environments Where Consistency

This Feature Is Particularly Useful For Developers Who Need Reliable,

For Example: Define The Structure Of The Response Using JSON

Previously, The Response_format Parameter Was Only Available To Specify That