DSPy framework integration - Orq.ai Documentation

AI Gateway

Overview

DSPy is a framework for programmatically optimizing LLM prompts and weights through composable modules and signatures. Connecting DSPy to Orq.ai’s AI Gateway provides access to 300+ models for prompt optimization pipelines with a single configuration change.

Key Benefits

Orq.ai’s AI Gateway enhances DSPy applications with:

Complete Observability

Track every signature execution, module call, and optimization step

Built-in Reliability

Automatic fallbacks, retries, and load balancing for production resilience

Cost Optimization

Real-time cost tracking and spend management across all AI operations

Multi-Provider Access

Access 300+ LLMs and 20+ providers through a single, unified integration

Prerequisites

Before integrating DSPy with Orq.ai, ensure the following are in place:

An Orq.ai account and API Key
Python 3.8 or higher

To set up an API key, see API keys & Endpoints.

Installation

pip install dspy

Configuration

Configure DSPy to use Orq.ai’s AI Gateway with dspy.LM:

Python

import dspy
import os

lm = dspy.LM(
    "openai/openai/gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v3/router",
)
dspy.configure(lm=lm)

api_base: https://api.orq.ai/v3/router

DSPy uses LiteLLM internally, which automatically prepends openai/ to the model name when routing to a custom api_base. Since the AI Gateway also requires the openai/ provider prefix, the model name must be double-prefixed so the final identifier reaching the gateway is correct: gpt-4o → openai/openai/gpt-4o.

Basic Example

Python

import dspy
import os

lm = dspy.LM(
    "openai/openai/gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v3/router",
)
dspy.configure(lm=lm)

class BasicQA(dspy.Signature):
    """Answer questions with short, accurate responses."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

qa = dspy.Predict(BasicQA)
result = qa(question="What is the capital of France?")
print(result.answer)

Chain of Thought

Use ChainOfThought for step-by-step reasoning:

Python

import dspy
import os

lm = dspy.LM(
    "openai/openai/gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v3/router",
)
dspy.configure(lm=lm)

class MathProblem(dspy.Signature):
    """Solve math problems step by step."""
    problem: str = dspy.InputField()
    answer: str = dspy.OutputField()

cot = dspy.ChainOfThought(MathProblem)
result = cot(problem="If a train travels 120 miles in 2 hours, what is its speed?")
print(result.answer)

Model Selection

With Orq.ai, any supported model from 20+ providers can be used:

Python

import dspy
import os

# Use Claude
claude_lm = dspy.LM(
    "openai/anthropic/claude-sonnet-4-6",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v3/router",
)

# Use Gemini
gemini_lm = dspy.LM(
    "openai/google-ai/gemini-2.5-flash",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v3/router",
)

class BasicQA(dspy.Signature):
    """Answer questions with short, accurate responses."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

qa = dspy.Predict(BasicQA)

with dspy.context(lm=claude_lm):
    result = qa(question="What is the largest planet?")
    print(result.answer)

​AI Gateway

​Overview

​Key Benefits

Complete Observability

Built-in Reliability

Cost Optimization

Multi-Provider Access

​Prerequisites

​Installation

​Configuration

​Basic Example

​Chain of Thought

​Model Selection

AI Gateway

Overview

Key Benefits

Prerequisites

Installation

Configuration

Basic Example

Chain of Thought

Model Selection