Skip to main content

​
AI Gateway

​
Overview

Instructor is a library for extracting structured outputs from LLMs using Pydantic models. Connecting Instructor to Orq.ai’s AI Gateway provides type-safe structured extraction with access to 300+ models through a single configuration change.

​
Key Benefits

Orq.ai’s AI Gateway enhances Instructor applications with:

Complete Observability

Track every extraction, validation, and retry with detailed traces

Built-in Reliability

Automatic fallbacks, retries, and load balancing for production resilience

Cost Optimization

Real-time cost tracking and spend management across all AI operations

Multi-Provider Access

Access 300+ LLMs and 20+ providers through a single, unified integration

​
Prerequisites

Before integrating Instructor with Orq.ai, ensure the following are in place:
  • An Orq.ai account and API Key
  • Python 3.8 or higher
To set up an API key, see API keys & Endpoints.

​
Installation

pip install instructor openai

​
Configuration

Configure Instructor to use Orq.ai’s AI Gateway by patching an OpenAI client with a custom base_url:
Python
import instructor
from openai import OpenAI
import os

client = instructor.from_openai(OpenAI(
    base_url="https://api.orq.ai/v3/router",
    api_key=os.getenv("ORQ_API_KEY"),
))
base_url: https://api.orq.ai/v3/router

​
Basic Extraction

Python
import instructor
from pydantic import BaseModel
from openai import OpenAI
import os

client = instructor.from_openai(OpenAI(
    base_url="https://api.orq.ai/v3/router",
    api_key=os.getenv("ORQ_API_KEY"),
))

class UserInfo(BaseModel):
    name: str
    age: int

user_info = client.chat.completions.create(
    model="openai/gpt-4o",
    response_model=UserInfo,
    messages=[{"role": "user", "content": "John Doe is 30 years old."}],
)

print(user_info.name)   # John Doe
print(user_info.age)    # 30

​
Classification

Use Instructor with Literal types for classification:
Python
import instructor
from pydantic import BaseModel, Field
from openai import OpenAI
from typing import Literal
import os

client = instructor.from_openai(OpenAI(
    base_url="https://api.orq.ai/v3/router",
    api_key=os.getenv("ORQ_API_KEY"),
))

class Sentiment(BaseModel):
    label: Literal["positive", "negative", "neutral"]
    confidence: float = Field(ge=0.0, le=1.0)

result = client.chat.completions.create(
    model="openai/gpt-4o",
    response_model=Sentiment,
    messages=[{"role": "user", "content": "Classify: 'This product is absolutely amazing!'"}],
)
print(f"Sentiment: {result.label}, Confidence: {result.confidence:.2f}")

​
Model Selection

With Orq.ai, any supported model from 20+ providers can be used:
Python
import instructor
from pydantic import BaseModel
from openai import OpenAI
import os

class UserInfo(BaseModel):
    name: str
    age: int

# Use Claude
claude_client = instructor.from_openai(OpenAI(
    base_url="https://api.orq.ai/v3/router",
    api_key=os.getenv("ORQ_API_KEY"),
))

result = claude_client.chat.completions.create(
    model="anthropic/claude-sonnet-4-6",
    response_model=UserInfo,
    messages=[{"role": "user", "content": "Jane Smith is 25 years old."}],
)
print(f"{result.name}, {result.age}")

# Use Gemini
gemini_client = instructor.from_openai(OpenAI(
    base_url="https://api.orq.ai/v3/router",
    api_key=os.getenv("ORQ_API_KEY"),
))

result = gemini_client.chat.completions.create(
    model="google-ai/gemini-2.5-flash",
    response_model=UserInfo,
    messages=[{"role": "user", "content": "Bob Johnson is 40 years old."}],
)
print(f"{result.name}, {result.age}")