> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orq.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenAI SDK drop-in integration

> Use the AI Router with OpenAI SDK. Replace OpenAI base URL with Orq.ai for routing, caching, monitoring, and multi-provider fallback capabilities.

<CardGroup cols={2}>
  <Card title="AI Router" icon="arrow-right-arrow-left" href="#ai-router">
    Route your LLM calls through the AI Router with a single base URL change. Zero vendor lock-in: always run on the best model at the lowest cost for your use case.
  </Card>

  <Card title="Observability" icon="chart-line" href="#observability">
    Instrument your code with OpenTelemetry to capture traces, logs, and metrics for every LLM call, agent step, and tool use.
  </Card>
</CardGroup>

## AI Router

### Overview

The OpenAI SDK provides powerful tools for building AI applications with GPT models. By connecting the SDK to Orq.ai's AI Router, you transform your OpenAI integration into a production-ready system with enterprise-grade capabilities, complete observability, and access to 300+ models across 20+ providers.

### Key Benefits

Orq.ai's AI Router enhances your OpenAI SDK with:

<CardGroup cols={2}>
  <Card title="Complete Observability" icon="chart-line">
    Track every API call, token usage, and model interaction with detailed traces and analytics
  </Card>

  <Card title="Built-in Reliability" icon="shield-check">
    Automatic fallbacks, retries, and load balancing for production resilience
  </Card>

  <Card title="Cost Optimization" icon="chart-pie">
    Real-time cost tracking and spend management across all your AI operations
  </Card>

  <Card title="Multi-Provider Access" icon="cubes">
    Access 300+ LLMs and 20+ providers through a single, unified integration
  </Card>
</CardGroup>

### Prerequisites

Before integrating OpenAI SDK with Orq.ai, ensure you have:

* An Orq.ai account and [API Key](/docs/administer/api-keys)
* Python 3.8+ or Node.js 18+ with TypeScript support
* OpenAI SDK installed

<Info>
  To setup your API key, see [API keys & Endpoints](/docs/administer/api-keys).
</Info>

<Info>
  To use libraries with private models, see [Onboarding Private Models](/docs/model-garden/using-ui#onboarding-private-models).
</Info>

### Installation

<CodeGroup>
  ```bash Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  pip install openai
  ```

  ```bash TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
  npm install openai
  ```
</CodeGroup>

### Configuration

While using the OpenAI SDK, set the **Base URL** to the [AI Router](/docs/router/using-the-router) to feed calls through our API without changing any other part of your code.

Using the **Orq.ai AI Router**, you benefit from Platform [Traces](/docs/observability/traces) and [Cost and Usage Monitoring](/docs/analytics/dashboards), keeping full compatibility and a unified API with all models while using the OpenAI SDK.

> **base\_url**: `https://api.orq.ai/v3/router`

### Text Generation

Basic text generation with the OpenAI SDK through Orq.ai:

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  from openai import OpenAI
  import os

  client = OpenAI(
    base_url="https://api.orq.ai/v3/router",
    api_key=os.getenv("ORQ_API_KEY"),
  )

  completion = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Hello!"}
    ]
  )

  print(completion.choices[0].message)
  ```

  ```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
  import OpenAI from "openai";

  const openai = new OpenAI({
    baseURL: 'https://api.orq.ai/v3/router',
    apiKey: process.env.ORQ_API_KEY,
  });

  async function main() {
    const completion = await openai.chat.completions.create({
      messages: [{ role: "system", content: "You are a helpful assistant." }],
      model: "openai/gpt-4o"
    });

    console.log(completion.choices[0]);
  }

  main();
  ```
</CodeGroup>

### Streaming Responses

Stream responses for real-time output:

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  from openai import OpenAI
  import os

  client = OpenAI(
    base_url="https://api.orq.ai/v3/router",
    api_key=os.getenv("ORQ_API_KEY"),
  )

  stream = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[
      {"role": "user", "content": "Write a short story about robots."}
    ],
    stream=True
  )

  for chunk in stream:
    if chunk.choices[0].delta.content:
      print(chunk.choices[0].delta.content, end="", flush=True)
  ```

  ```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
  import OpenAI from "openai";

  const openai = new OpenAI({
    baseURL: 'https://api.orq.ai/v3/router',
    apiKey: process.env.ORQ_API_KEY,
  });

  async function main() {
    const stream = await openai.chat.completions.create({
      model: "openai/gpt-4o",
      messages: [{ role: "user", content: "Write a short story about robots." }],
      stream: true,
    });

    for await (const chunk of stream) {
      process.stdout.write(chunk.choices[0]?.delta?.content || "");
    }
  }

  main();
  ```
</CodeGroup>

### Model Selection

With Orq.ai, you can use any supported model from 20+ providers:

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  from openai import OpenAI
  import os

  client = OpenAI(
    base_url="https://api.orq.ai/v3/router",
    api_key=os.getenv("ORQ_API_KEY"),
  )

  # Use Claude
  claude_response = client.chat.completions.create(
    model="anthropic/claude-sonnet-4-5-20250929",
    messages=[{"role": "user", "content": "Explain machine learning"}]
  )

  # Use Gemini
  gemini_response = client.chat.completions.create(
    model="google/gemini-2.5-flash",
    messages=[{"role": "user", "content": "Explain machine learning"}]
  )

  # Use Groq
  groq_response = client.chat.completions.create(
    model="groq/llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "Explain machine learning"}]
  )
  ```

  ```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
  import OpenAI from "openai";

  const openai = new OpenAI({
    baseURL: 'https://api.orq.ai/v3/router',
    apiKey: process.env.ORQ_API_KEY,
  });

  // Use Claude
  const claudeResponse = await openai.chat.completions.create({
    model: "anthropic/claude-sonnet-4-5-20250929",
    messages: [{ role: "user", content: "Explain machine learning" }],
  });

  // Use Gemini
  const geminiResponse = await openai.chat.completions.create({
    model: "google/gemini-2.5-flash",
    messages: [{ role: "user", content: "Explain machine learning" }],
  });

  // Use Groq
  const groqResponse = await openai.chat.completions.create({
    model: "groq/llama-3.3-70b-versatile",
    messages: [{ role: "user", content: "Explain machine learning" }],
  });
  ```
</CodeGroup>

## Observability

### Getting Started

Integrate OpenAI SDK with Orq.ai's observability to gain complete insights into model performance, token usage, API latency, and conversation flows using OpenTelemetry.

### Prerequisites

Before you begin, ensure you have:

* An Orq.ai account and [API Key](/docs/administer/api-keys)
* Python 3.8+ or Node.js 18+
* OpenAI SDK installed

### Install Dependencies

<CodeGroup>
  ```bash Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  # Core OpenTelemetry packages
  pip install opentelemetry-sdk opentelemetry-instrumentation opentelemetry-exporter-otlp

  # OpenAI SDK
  pip install openai

  # OpenAI Auto-Instrumentation
  pip install opentelemetry-instrumentation-openai
  ```

  ```bash TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
  # OpenTelemetry SDK (includes all core packages)
  npm install @opentelemetry/sdk-node @opentelemetry/api @opentelemetry/exporter-trace-otlp-http

  # OpenAI SDK
  npm install openai

  # OpenAI Auto-Instrumentation
  npm install @opentelemetry/instrumentation-openai
  ```
</CodeGroup>

### Configure Orq.ai

Set up your environment variables to connect to Orq.ai's OpenTelemetry collector:

**Unix/Linux/macOS:**

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.orq.ai/v2/otel"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer $ORQ_API_KEY"
export OTEL_RESOURCE_ATTRIBUTES="service.name=openai-app,service.version=1.0.0"
export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL="http/json"
```

**Windows (PowerShell):**

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
$env:OTEL_EXPORTER_OTLP_ENDPOINT = "https://api.orq.ai/v2/otel"
$env:OTEL_EXPORTER_OTLP_HEADERS = "Authorization=Bearer <ORQ_API_KEY>"
$env:OTEL_RESOURCE_ATTRIBUTES = "service.name=openai-app,service.version=1.0.0"
$env:OTEL_EXPORTER_OTLP_TRACES_PROTOCOL = "http/json"
```

**Using .env file:**

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
OTEL_EXPORTER_OTLP_ENDPOINT=https://api.orq.ai/v2/otel
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer <ORQ_API_KEY>
OTEL_RESOURCE_ATTRIBUTES=service.name=openai-app,service.version=1.0.0
OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/json
```

### Setup

Configure OpenTelemetry once at application startup:

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  from openai import OpenAI
  from opentelemetry import trace
  from opentelemetry.sdk.trace import TracerProvider
  from opentelemetry.sdk.trace.export import BatchSpanProcessor
  from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
  from opentelemetry.instrumentation.openai import OpenAIInstrumentor
  import os

  # Configure OpenTelemetry
  tracer_provider = TracerProvider()
  tracer_provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
  trace.set_tracer_provider(tracer_provider)

  # Instrument OpenAI
  OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)

  # Create OpenAI client
  client = OpenAI(
      base_url="https://api.orq.ai/v3/router",
      api_key=os.getenv("ORQ_API_KEY")
  )
  ```

  ```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
  import { NodeSDK } from "@opentelemetry/sdk-node";
  import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
  import { OpenAIInstrumentation } from "@opentelemetry/instrumentation-openai";
  import OpenAI from "openai";

  // Initialize SDK
  const sdk = new NodeSDK({
    traceExporter: new OTLPTraceExporter(),
    instrumentations: [new OpenAIInstrumentation()],
  });

  sdk.start();

  // Graceful shutdown
  process.on("SIGTERM", () => {
    sdk.shutdown()
      .then(() => console.log("Tracing terminated"))
      .catch((error) => console.log("Error terminating tracing", error))
      .finally(() => process.exit(0));
  });

  // Create OpenAI client
  const client = new OpenAI({
    baseURL: "https://api.orq.ai/v3/router",
    apiKey: process.env.ORQ_API_KEY,
  });
  ```
</CodeGroup>

### Examples

#### Basic Example

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  response = client.chat.completions.create(
      model="openai/gpt-4o",
      messages=[
          {"role": "system", "content": "You are a helpful assistant."},
          {"role": "user", "content": "What is quantum computing in one sentence?"}
      ]
  )

  print(response.choices[0].message.content)
  ```

  ```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
  const response = await client.chat.completions.create({
    model: "openai/gpt-4o",
    messages: [
      { role: "system", content: "You are a helpful assistant." },
      { role: "user", content: "What is quantum computing in one sentence?" },
    ],
  });

  console.log(response.choices[0].message.content);
  ```
</CodeGroup>

#### Streaming Example

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  from opentelemetry import trace

  tracer = trace.get_tracer(__name__)
  with tracer.start_as_current_span("streaming-completion") as span:
      stream = client.chat.completions.create(
          model="openai/gpt-4o",
          messages=[{"role": "user", "content": "Write a haiku about code."}],
          stream=True
      )

      full_response = ""
      for chunk in stream:
          if chunk.choices and chunk.choices[0].delta.content:
              content = chunk.choices[0].delta.content
              full_response += content
              print(content, end="", flush=True)

      span.set_attribute("response.length", len(full_response))
  ```

  ```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
  import { trace } from "@opentelemetry/api";

  const tracer = trace.getTracer("openai-app");

  await tracer.startActiveSpan("streaming-completion", async (span) => {
    try {
      const stream = await client.chat.completions.create({
        model: "openai/gpt-4o",
        messages: [{ role: "user", content: "Write a haiku about code." }],
        stream: true,
      });

      let fullResponse = "";
      for await (const chunk of stream) {
        const content = chunk.choices[0]?.delta?.content;
        if (content) {
          fullResponse += content;
          process.stdout.write(content);
        }
      }

      span.setAttribute("response.length", fullResponse.length);
    } finally {
      span.end();
    }
  });
  ```
</CodeGroup>

#### Custom Spans Example

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  from opentelemetry import trace

  tracer = trace.get_tracer(__name__)

  def analyze_document(document: str):
      with tracer.start_as_current_span("document-analysis") as span:
          span.set_attribute("document.length", len(document))

          # Prepare prompt
          prompt = f"Analyze this text: {document}"
          with tracer.start_as_current_span("prepare-prompt") as prep_span:
              prep_span.set_attribute("prompt.length", len(prompt))

          # Model inference
          with tracer.start_as_current_span("model-inference") as inf_span:
              inf_span.set_attribute("model", "gpt-4o")

              response = client.chat.completions.create(
                  model="openai/gpt-4o",
                  messages=[{"role": "user", "content": prompt}]
              )

              inf_span.set_attribute("tokens.total", response.usage.total_tokens)

          # Process result
          with tracer.start_as_current_span("process-result") as proc_span:
              result = response.choices[0].message.content
              proc_span.set_attribute("result.length", len(result))

          return result

  analysis = analyze_document("Machine learning is a subset of AI.")
  print(analysis)
  ```

  ```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
  import { trace } from "@opentelemetry/api";

  const tracer = trace.getTracer("openai-app");

  async function analyzeDocument(document: string) {
    return await tracer.startActiveSpan("document-analysis", async (span) => {
      try {
        span.setAttribute("document.length", document.length);

        // Prepare prompt
        const prompt = `Analyze this text: ${document}`;
        await tracer.startActiveSpan("prepare-prompt", async (prepSpan) => {
          try {
            prepSpan.setAttribute("prompt.length", prompt.length);
          } finally {
            prepSpan.end();
          }
        });

        // Model inference
        let response: Awaited<ReturnType<typeof client.chat.completions.create>>;
        await tracer.startActiveSpan("model-inference", async (infSpan) => {
          try {
            infSpan.setAttribute("model", "gpt-4o");

            response = await client.chat.completions.create({
              model: "openai/gpt-4o",
              messages: [{ role: "user", content: prompt }],
            });

            infSpan.setAttribute("tokens.total", response.usage?.total_tokens || 0);
          } finally {
            infSpan.end();
          }
        });

        // Process result
        await tracer.startActiveSpan("process-result", async (procSpan) => {
          try {
            const result = response!.choices[0].message.content || "";
            procSpan.setAttribute("result.length", result.length);
          } finally {
            procSpan.end();
          }
        });

        return response!.choices[0].message.content;
      } finally {
        span.end();
      }
    });
  }

  analyzeDocument("Machine learning is a subset of AI.")
    .then((analysis) => console.log(analysis));
  ```
</CodeGroup>

#### Advanced Workflows Example

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  from opentelemetry import trace

  tracer = trace.get_tracer(__name__)

  def content_generation_pipeline(topic: str):
      with tracer.start_as_current_span("content-pipeline") as pipeline_span:
          pipeline_span.set_attribute("pipeline.topic", topic)
          pipeline_span.set_attribute("pipeline.stages", 3)

          # Stage 1: Research
          with tracer.start_as_current_span("stage-research") as research_span:
              research_span.set_attribute("stage.name", "research")

              research = client.chat.completions.create(
                  model="openai/gpt-4o",
                  messages=[{"role": "user", "content": f"List 3 key facts about {topic}."}]
              )

              facts = research.choices[0].message.content
              research_span.set_attribute("facts.count", 3)
              research_span.set_attribute("tokens.used", research.usage.total_tokens)

          # Stage 2: Writing
          with tracer.start_as_current_span("stage-writing") as writing_span:
              writing_span.set_attribute("stage.name", "writing")

              writing = client.chat.completions.create(
                  model="openai/gpt-4o",
                  messages=[{"role": "user", "content": f"Write a brief introduction using these facts: {facts}"}]
              )

              content = writing.choices[0].message.content
              writing_span.set_attribute("content.length", len(content))

          # Stage 3: Review
          with tracer.start_as_current_span("stage-review") as review_span:
              review_span.set_attribute("stage.name", "review")

              review = client.chat.completions.create(
                  model="openai/gpt-4o",
                  messages=[{"role": "user", "content": f"Rate this content quality 1-10: {content}"}]
              )

              rating = review.choices[0].message.content
              review_span.set_attribute("quality.rating", rating)

          pipeline_span.set_attribute("pipeline.success", True)
          return {"content": content, "rating": rating}

  result = content_generation_pipeline("neural networks")
  print(f"Content: {result['content']}")
  print(f"Rating: {result['rating']}")
  ```

  ```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
  import { trace } from "@opentelemetry/api";

  const tracer = trace.getTracer("openai-app");

  async function contentGenerationPipeline(topic: string) {
    return await tracer.startActiveSpan("content-pipeline", async (pipelineSpan) => {
      try {
        pipelineSpan.setAttribute("pipeline.topic", topic);
        pipelineSpan.setAttribute("pipeline.stages", 3);

        let facts = "";
        let content = "";
        let rating = "";

        // Stage 1: Research
        await tracer.startActiveSpan("stage-research", async (researchSpan) => {
          try {
            researchSpan.setAttribute("stage.name", "research");

            const research = await client.chat.completions.create({
              model: "openai/gpt-4o",
              messages: [{ role: "user", content: `List 3 key facts about ${topic}.` }],
            });

            facts = research.choices[0].message.content || "";
            researchSpan.setAttribute("facts.count", 3);
            researchSpan.setAttribute("tokens.used", research.usage?.total_tokens || 0);
          } finally {
            researchSpan.end();
          }
        });

        // Stage 2: Writing
        await tracer.startActiveSpan("stage-writing", async (writingSpan) => {
          try {
            writingSpan.setAttribute("stage.name", "writing");

            const writing = await client.chat.completions.create({
              model: "openai/gpt-4o",
              messages: [{ role: "user", content: `Write a brief introduction using these facts: ${facts}` }],
            });

            content = writing.choices[0].message.content || "";
            writingSpan.setAttribute("content.length", content.length);
          } finally {
            writingSpan.end();
          }
        });

        // Stage 3: Review
        await tracer.startActiveSpan("stage-review", async (reviewSpan) => {
          try {
            reviewSpan.setAttribute("stage.name", "review");

            const review = await client.chat.completions.create({
              model: "openai/gpt-4o",
              messages: [{ role: "user", content: `Rate this content quality 1-10: ${content}` }],
            });

            rating = review.choices[0].message.content || "";
            reviewSpan.setAttribute("quality.rating", rating);
          } finally {
            reviewSpan.end();
          }
        });

        pipelineSpan.setAttribute("pipeline.success", true);
        return { content, rating };
      } finally {
        pipelineSpan.end();
      }
    });
  }

  contentGenerationPipeline("neural networks")
    .then((result) => {
      console.log(`Content: ${result.content}`);
      console.log(`Rating: ${result.rating}`);
    });
  ```
</CodeGroup>

### View Traces

View your traces in the [AI Studio](/docs/observability/overview) in the **Traces** tab.

<Frame caption="Traces will display detailed information about your OpenAI SDK calls">
  <img src="https://mintcdn.com/orqai/3I80b9x_G_KAvTRg/images/trace-openai.png?fit=max&auto=format&n=3I80b9x_G_KAvTRg&q=85&s=96e66cda79b01bbc08ea5d8b88bdd99b" alt="Traces will display detailed information about your OpenAI SDK calls" width="1672" height="945" data-path="images/trace-openai.png" />
</Frame>

<Info>
  Visit your [AI Studio](https://my.orq.ai) to view real-time analytics and traces.
</Info>

## Evaluations & Experiments

Once your agents are running, use **Evaluatorq** to score outputs across a dataset and **Experiments** to compare configurations side-by-side.

<CardGroup cols={2}>
  <Card title="Run Evaluations with Evaluatorq" icon="flask" href="/docs/evaluators/build#evaluatorq">
    Run parallel evaluations across your agents and compare results.
  </Card>

  <Card title="Run Experiments via the API" icon="flask-vial" href="/docs/experiments/api">
    Compare agent configurations and view results in the AI Studio.
  </Card>
</CardGroup>