> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orq.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# DSPy framework integration

> Integrate DSPy with the AI Gateway for optimized LLM programs. Use Stanford's framework for automatic prompt optimization and reasoning-based AI systems.

## AI Gateway

### Overview

DSPy is a framework for programmatically optimizing LLM prompts and weights through composable modules and signatures. Connecting DSPy to Orq.ai's AI Gateway provides access to 300+ models for prompt optimization pipelines with a single configuration change.

### Key Benefits

Orq.ai's AI Gateway enhances DSPy applications with:

<CardGroup cols={2}>
  <Card title="Complete Observability" icon="chart-line">
    Track every signature execution, module call, and optimization step
  </Card>

  <Card title="Built-in Reliability" icon="shield-check">
    Automatic fallbacks, retries, and load balancing for production resilience
  </Card>

  <Card title="Cost Optimization" icon="chart-pie">
    Real-time cost tracking and spend management across all AI operations
  </Card>

  <Card title="Multi-Provider Access" icon="cubes">
    Access 300+ LLMs and 20+ providers through a single, unified integration
  </Card>
</CardGroup>

### Prerequisites

Before integrating DSPy with Orq.ai, ensure the following are in place:

* An Orq.ai account and [API Key](/docs/ai-gateway/configuration/api-keys)
* Python 3.8 or higher

<Info>
  To set up an API key, see [API keys & Endpoints](/docs/ai-gateway/configuration/api-keys).
</Info>

### Installation

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
pip install dspy
```

### Configuration

Configure DSPy to use Orq.ai's AI Gateway with `dspy.LM`:

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import dspy
import os

lm = dspy.LM(
    "openai/openai/gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v3/router",
)
dspy.configure(lm=lm)
```

> **api\_base**: `https://api.orq.ai/v3/router`

<Warning>
  DSPy uses LiteLLM internally, which automatically prepends `openai/` to the model name when routing to a custom `api_base`. Since the **AI Gateway** also requires the `openai/` provider prefix, the model name must be double-prefixed so the final identifier reaching the gateway is correct: `gpt-4o` → `openai/openai/gpt-4o`.
</Warning>

### Basic Example

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import dspy
import os

lm = dspy.LM(
    "openai/openai/gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v3/router",
)
dspy.configure(lm=lm)

class BasicQA(dspy.Signature):
    """Answer questions with short, accurate responses."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

qa = dspy.Predict(BasicQA)
result = qa(question="What is the capital of France?")
print(result.answer)
```

### Chain of Thought

Use `ChainOfThought` for step-by-step reasoning:

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import dspy
import os

lm = dspy.LM(
    "openai/openai/gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v3/router",
)
dspy.configure(lm=lm)

class MathProblem(dspy.Signature):
    """Solve math problems step by step."""
    problem: str = dspy.InputField()
    answer: str = dspy.OutputField()

cot = dspy.ChainOfThought(MathProblem)
result = cot(problem="If a train travels 120 miles in 2 hours, what is its speed?")
print(result.answer)
```

### Model Selection

With Orq.ai, any supported model from 20+ providers can be used:

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import dspy
import os

# Use Claude
claude_lm = dspy.LM(
    "openai/anthropic/claude-sonnet-4-6",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v3/router",
)

# Use Gemini
gemini_lm = dspy.LM(
    "openai/google-ai/gemini-2.5-flash",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v3/router",
)

class BasicQA(dspy.Signature):
    """Answer questions with short, accurate responses."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

qa = dspy.Predict(BasicQA)

with dspy.context(lm=claude_lm):
    result = qa(question="What is the largest planet?")
    print(result.answer)
```