Structured Outputs

📖
This page describes features extending the AI Proxy, which provides a unified API for accessing multiple AI providers. To learn more, see AI Proxy.

Quick Start

Generate structured JSON responses with guaranteed schema compliance.

import { z } from "zod";
import { zodResponseFormat } from "openai/helpers/zod";

// Define your schema
const UserSchema = z.object({
  name: z.string(),
  age: z.number(),
  email: z.string(),
  skills: z.array(z.string()),
});

const response = await openai.beta.chat.completions.parse({
  model: "openai/gpt-4o",
  messages: [
    {
      role: "user",
      content:
        "Extract info: John Doe, 30, [email protected], knows Python and React",
    },
  ],
  response_format: zodResponseFormat(UserSchema, "user"),
});

// Type-safe access
const user = response.choices[0].message.parsed;
console.log(user.name); // Guaranteed string
console.log(user.skills); // Guaranteed string array

Configuration Options

JSON Schema Mode

Parameter	Type	Description
`type`	`"json_schema"`	Enable schema validation
`json_schema.name`	string	Schema name/identifier
`json_schema.schema`	object	JSON Schema definition

Simple JSON Mode

Parameter	Type	Description
`type`	`"json_object"`	Basic JSON output (no schema)

Supported Models

Provider	Model	Schema Support	JSON Mode
OpenAI	`gpt-4o`	✅ Full	✅ Yes
OpenAI	`gpt-4o-mini`	✅ Full	✅ Yes
OpenAI	`gpt-4-turbo`	✅ Full	✅ Yes
Anthropic	`claude-3-5-sonnet`	⚠️ Partial	✅ Yes
Google	`gemini-1.5-pro`	⚠️ Partial	✅ Yes

Schema Examples

Simple Data Extraction

{
  "type": "json_schema",
  "json_schema": {
    "name": "contact_info",
    "schema": {
      "type": "object",
      "properties": {
        "name": { "type": "string" },
        "email": { "type": "string", "format": "email" },
        "phone": { "type": "string" },
        "company": { "type": "string" }
      },
      "required": ["name", "email"]
    }
  }
}

Complex Nested Structure

{
  "type": "json_schema",
  "json_schema": {
    "name": "product_analysis",
    "schema": {
      "type": "object",
      "properties": {
        "product": {
          "type": "object",
          "properties": {
            "name": { "type": "string" },
            "price": { "type": "number" },
            "category": { "type": "string" }
          }
        },
        "features": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "name": { "type": "string" },
              "importance": { "type": "integer", "minimum": 1, "maximum": 5 }
            }
          }
        },
        "summary": { "type": "string" }
      },
      "required": ["product", "features", "summary"]
    }
  }
}

Code examples

curl -X POST https://api.orq.ai/v2/proxy/chat/completions \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": "Extract user information from: John Doe, 30 years old, [email protected], Software Engineer"
      }
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "user_info",
        "schema": {
          "type": "object",
          "properties": {
            "name": {"type": "string"},
            "age": {"type": "integer"},
            "email": {"type": "string"},
            "occupation": {"type": "string"}
          },
          "required": ["name", "age", "email", "occupation"]
        }
      }
    }
  }'

from openai import OpenAI
from pydantic import BaseModel
import os

openai = OpenAI(
  api_key=os.environ.get("ORQ_API_KEY"),
  base_url="https://api.orq.ai/v2/proxy"
)

# Define your schema using Pydantic
class UserInfo(BaseModel):
    name: str
    age: int
    email: str
    occupation: str

response = openai.beta.chat.completions.parse(
    model="openai/gpt-4o",
    messages=[
        {
            "role": "user",
            "content": "Extract user information from: John Doe, 30 years old, [email protected], Software Engineer"
        }
    ],
    response_format=UserInfo,
)

# Access structured data directly
user = response.choices[0].message.parsed
print(f"Name: {user.name}")
print(f"Age: {user.age}")
print(f"Email: {user.email}")
print(f"Occupation: {user.occupation}")

import OpenAI from "openai";
import { z } from "zod";
import { zodResponseFormat } from "openai/helpers/zod";

const openai = new OpenAI({
  apiKey: process.env.ORQ_API_KEY,
  baseURL: "https://api.orq.ai/v2/proxy",
});

// Define your schema using Zod
const UserInfoSchema = z.object({
  name: z.string(),
  age: z.number(),
  email: z.string(),
  occupation: z.string(),
});

const response = await openai.beta.chat.completions.parse({
  model: "openai/gpt-4o",
  messages: [
    {
      role: "user",
      content:
        "Extract user information from: John Doe, 30 years old, [email protected], Software Engineer",
    },
  ],
  response_format: zodResponseFormat(UserInfoSchema, "user_info"),
});

// Access structured data with type safety
const user = response.choices[0].message.parsed;
console.log(`Name: ${user.name}`);
console.log(`Age: ${user.age}`);
console.log(`Email: ${user.email}`);
console.log(`Occupation: ${user.occupation}`);

Common Use Cases

Data Extraction

class InvoiceData(BaseModel):
    invoice_number: str
    date: str
    vendor: str
    total_amount: float
    line_items: List[dict]

response = openai.beta.chat.completions.parse(
    model="openai/gpt-4o",
    messages=[{
        "role": "user",
        "content": "Extract invoice data from this text: [invoice text]"
    }],
    response_format=InvoiceData
)

Content Analysis

const SentimentSchema = z.object({
  sentiment: z.enum(["positive", "negative", "neutral"]),
  confidence: z.number().min(0).max(1),
  key_phrases: z.array(z.string()),
  summary: z.string(),
});

const response = await openai.beta.chat.completions.parse({
  model: "openai/gpt-4o",
  messages: [
    {
      role: "user",
      content: "Analyze sentiment of this review: [review text]",
    },
  ],
  response_format: zodResponseFormat(SentimentSchema, "sentiment_analysis"),
});

Form Generation

class SurveyForm(BaseModel):
    title: str
    description: str
    questions: List[dict] = Field(
        ...,
        description="List of survey questions with type and options"
    )
    estimated_duration: int = Field(description="Minutes to complete")

response = openai.beta.chat.completions.parse(
    model="openai/gpt-4o",
    messages=[{
        "role": "user",
        "content": "Create a customer satisfaction survey for a restaurant"
    }],
    response_format=SurveyForm
)

Advanced Patterns

Conditional Schemas

const ResponseSchema = z.discriminatedUnion("type", [
  z.object({
    type: z.literal("success"),
    data: z.object({
      result: z.string(),
      metadata: z.record(z.any()),
    }),
  }),
  z.object({
    type: z.literal("error"),
    error: z.object({
      code: z.string(),
      message: z.string(),
    }),
  }),
]);

Dynamic Schema Generation

def create_schema_for_fields(fields: List[str]):
    properties = {}
    for field in fields:
        properties[field] = {"type": "string"}

    return {
        "type": "object",
        "properties": properties,
        "required": fields
    }

# Generate schema based on user input
fields = ["name", "email", "department"]
schema = create_schema_for_fields(fields)

Validation and Error Handling

try:
    response = openai.beta.chat.completions.parse(
        model="openai/gpt-4o",
        messages=[{"role": "user", "content": prompt}],
        response_format=MySchema
    )

    if response.choices[0].message.parsed:
        data = response.choices[0].message.parsed
        # Process validated data
    else:
        # Handle parsing failure
        print("Failed to parse response")

except Exception as e:
    print(f"Error: {e}")
    # Fallback to regular completion

Best Practices

Schema design:

Use descriptive field names
Add field descriptions for better results
Mark essential fields as required
Use appropriate data types and constraints

Error handling:

const parseWithFallback = async (prompt, schema) => {
  try {
    const response = await openai.beta.chat.completions.parse({
      model: "openai/gpt-4o",
      messages: [{ role: "user", content: prompt }],
      response_format: zodResponseFormat(schema, "data"),
    });

    return response.choices[0].message.parsed;
  } catch (error) {
    console.warn("Structured parsing failed, trying JSON mode");

    // Fallback to basic JSON mode
    const fallback = await openai.chat.completions.create({
      model: "openai/gpt-4o",
      messages: [
        {
          role: "user",
          content: `${prompt}\n\nRespond with valid JSON only.`,
        },
      ],
      response_format: { type: "json_object" },
    });

    return JSON.parse(fallback.choices[0].message.content);
  }
};

Performance optimization:

# Cache schemas for reuse
schema_cache = {}

def get_cached_schema(schema_name):
    if schema_name not in schema_cache:
        schema_cache[schema_name] = create_schema(schema_name)
    return schema_cache[schema_name]

Troubleshooting

Schema validation fails

Simplify complex nested structures
Ensure required fields are clearly specified
Check field types match expected data
Add field descriptions for clarity

Inconsistent outputs

Use more specific prompts
Add examples in the prompt
Increase model temperature for creativity
Switch to a more capable model

Performance issues

Reduce schema complexity
Cache schema definitions
Use appropriate models for task complexity
Consider breaking large schemas into smaller ones

Limitations

Limitation	Impact	Workaround
Schema complexity	Large schemas may fail	Break into smaller schemas
Model support	Not all models support schemas	Use JSON mode as fallback
Nested depth	Deep nesting may cause issues	Flatten structures when possible
Array validation	Complex array items challenging	Simplify item schemas
Performance cost	Schema validation adds latency	Cache and optimize schemas

Integration Examples

Database Integration

from sqlalchemy import create_engine
from pydantic import BaseModel

class DatabaseRecord(BaseModel):
    id: Optional[int] = None
    name: str
    email: str
    created_at: Optional[datetime] = None

# Generate structured data
response = openai.beta.chat.completions.parse(
    model="openai/gpt-4o",
    messages=[{"role": "user", content: "Create user record"}],
    response_format=DatabaseRecord
)

# Direct database insertion
record = response.choices[0].message.parsed
session.add(record)
session.commit()

API Integration

const ApiResponseSchema = z.object({
  status: z.enum(["success", "error"]),
  data: z.any(),
  message: z.string().optional(),
});

const generateApiResponse = async (query) => {
  const response = await openai.beta.chat.completions.parse({
    model: "openai/gpt-4o",
    messages: [{ role: "user", content: query }],
    response_format: zodResponseFormat(ApiResponseSchema, "api_response"),
  });

  // Return structured API response
  return response.choices[0].message.parsed;
};