Skip to main content

​
AI Gateway

​
Overview

Pydantic AI is a Python agent framework designed to make it easier to build production-grade applications with Generative AI. Connecting Pydantic AI to Orq.ai’s AI Gateway transforms experimental agents into production-ready systems with enterprise-grade capabilities.

​
Key Benefits

Orq.ai’s AI Gateway enhances Pydantic AI agents with:

Complete Observability

Track every agent step, tool use, and interaction with detailed traces and analytics

Built-in Reliability

Automatic fallbacks, retries, and load balancing for production resilience

Cost Optimization

Real-time cost tracking and spend management across all AI operations

Multi-Provider Access

Access 300+ LLMs and 20+ providers through a single, unified integration

​
Prerequisites

Before integrating Pydantic AI with Orq.ai, ensure the following are in place:
  • An Orq.ai account and API Key
  • Python 3.9 or higher
  • Pydantic AI SDK installed
To set up an API key, see API keys & Endpoints.

​
Installation

Install Pydantic AI and the OpenAI SDK: 
pip install pydantic-ai openai

​
Configuration

Configure Pydantic AI to use Orq.ai’s AI Gateway by passing a custom OpenAI client:
Python
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider
from openai import AsyncOpenAI
import os

# Configure OpenAI client with Orq.ai AI Gateway
client = AsyncOpenAI(
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router"
)

# Create provider with custom client
provider = OpenAIProvider(openai_client=client)

# Create model and agent
model = OpenAIChatModel("openai/gpt-4o", provider=provider)
agent = Agent(model=model)
base_url: https://api.orq.ai/v3/router

​
Basic Agent Example

Here’s a complete example of creating and running a Pydantic AI agent through Orq.ai:
Python
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider
from openai import AsyncOpenAI
import os

# Configure client with Orq.ai AI Gateway
client = AsyncOpenAI(
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router"
)

# Create provider and model
provider = OpenAIProvider(openai_client=client)
model = OpenAIChatModel("openai/gpt-4o", provider=provider)

# Create agent
agent = Agent(
    model=model,
    system_prompt="You are a helpful assistant."
)

# Run the agent
result = agent.run_sync("Explain quantum computing in simple terms")
print(result.output)

​
Agent with Tools

Pydantic AI agents can use tools while routing through Orq.ai:
Python
from pydantic_ai import Agent, RunContext
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider
from openai import AsyncOpenAI
import os

# Configure client
client = AsyncOpenAI(
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router"
)

# Create provider and model
provider = OpenAIProvider(openai_client=client)
model = OpenAIChatModel("openai/gpt-4o", provider=provider)

# Create agent
agent = Agent(model=model)

@agent.tool
async def get_weather(ctx: RunContext[None], location: str) -> str:
    """Get current weather for a location."""
    return f"The weather in {location} is sunny and 75°F"

# Run agent with tool access
result = agent.run_sync("What's the weather in San Francisco?")
print(result.output)

​
Structured Outputs

Pydantic AI excels at structured outputs with type-safe validation:
Python
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider
from openai import AsyncOpenAI
from pydantic import BaseModel
import os

class CityInfo(BaseModel):
    name: str
    country: str
    population: int
    famous_for: str

# Configure client
client = AsyncOpenAI(
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router"
)

# Create provider and model
provider = OpenAIProvider(openai_client=client)
model = OpenAIChatModel("openai/gpt-4o", provider=provider)

# Create agent with structured output
agent = Agent(model=model, output_type=CityInfo)

result = agent.run_sync("Tell me about Paris")
print(f"{result.output.name}, {result.output.country}")
print(f"Population: {result.output.population:,}")
print(f"Famous for: {result.output.famous_for}")

​
Model Selection

With Orq.ai, any supported model from 20+ providers can be used:
Python
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider
from openai import AsyncOpenAI
import os

# Configure client
client = AsyncOpenAI(
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router"
)

# Create provider
provider = OpenAIProvider(openai_client=client)

# Use Claude
claude_model = OpenAIChatModel("anthropic/claude-sonnet-4-6", provider=provider)
claude_agent = Agent(model=claude_model)

# Use Gemini
gemini_model = OpenAIChatModel("google-ai/gemini-2.5-flash", provider=provider)
gemini_agent = Agent(model=gemini_model)

# Use any other model
groq_model = OpenAIChatModel("groq/llama-3.3-70b-versatile", provider=provider)
groq_agent = Agent(model=groq_model)