Skip to main content

​
AI Gateway

​
Overview

DSPy is a framework for programmatically optimizing LLM prompts and weights through composable modules and signatures. Connecting DSPy to Orq.ai’s AI Gateway provides access to 300+ models for prompt optimization pipelines with a single configuration change.

​
Key Benefits

Orq.ai’s AI Gateway enhances DSPy applications with:

Complete Observability

Track every signature execution, module call, and optimization step

Built-in Reliability

Automatic fallbacks, retries, and load balancing for production resilience

Cost Optimization

Real-time cost tracking and spend management across all AI operations

Multi-Provider Access

Access 300+ LLMs and 20+ providers through a single, unified integration

​
Prerequisites

Before integrating DSPy with Orq.ai, ensure the following are in place:
  • An Orq.ai account and API Key
  • Python 3.8 or higher
To set up an API key, see API keys & Endpoints.

​
Installation

pip install dspy

​
Configuration

Configure DSPy to use Orq.ai’s AI Gateway with dspy.LM:
Python
import dspy
import os

lm = dspy.LM(
    "openai/openai/gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v3/router",
)
dspy.configure(lm=lm)
api_base: https://api.orq.ai/v3/router
DSPy uses LiteLLM internally, which automatically prepends openai/ to the model name when routing to a custom api_base. Since the AI Gateway also requires the openai/ provider prefix, the model name must be double-prefixed so the final identifier reaching the gateway is correct: gpt-4o → openai/openai/gpt-4o.

​
Basic Example

Python
import dspy
import os

lm = dspy.LM(
    "openai/openai/gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v3/router",
)
dspy.configure(lm=lm)

class BasicQA(dspy.Signature):
    """Answer questions with short, accurate responses."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

qa = dspy.Predict(BasicQA)
result = qa(question="What is the capital of France?")
print(result.answer)

​
Chain of Thought

Use ChainOfThought for step-by-step reasoning:
Python
import dspy
import os

lm = dspy.LM(
    "openai/openai/gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v3/router",
)
dspy.configure(lm=lm)

class MathProblem(dspy.Signature):
    """Solve math problems step by step."""
    problem: str = dspy.InputField()
    answer: str = dspy.OutputField()

cot = dspy.ChainOfThought(MathProblem)
result = cot(problem="If a train travels 120 miles in 2 hours, what is its speed?")
print(result.answer)

​
Model Selection

With Orq.ai, any supported model from 20+ providers can be used:
Python
import dspy
import os

# Use Claude
claude_lm = dspy.LM(
    "openai/anthropic/claude-sonnet-4-6",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v3/router",
)

# Use Gemini
gemini_lm = dspy.LM(
    "openai/google-ai/gemini-2.5-flash",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v3/router",
)

class BasicQA(dspy.Signature):
    """Answer questions with short, accurate responses."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

qa = dspy.Predict(BasicQA)

with dspy.context(lm=claude_lm):
    result = qa(question="What is the largest planet?")
    print(result.answer)