Skip to main content

​
AI Gateway

​
Overview

The OpenAI SDK provides powerful tools for building AI applications with GPT models. Connecting the SDK to Orq.ai’s AI Gateway transforms any OpenAI integration into a production-ready system with enterprise-grade capabilities, complete observability, and access to 300+ models across 20+ providers.

​
Key Benefits

Orq.ai’s AI Gateway enhances the OpenAI SDK with:

Complete Observability

Track every API call, token usage, and model interaction with detailed traces and analytics

Built-in Reliability

Automatic fallbacks, retries, and load balancing for production resilience

Cost Optimization

Real-time cost tracking and spend management across all AI operations

Multi-Provider Access

Access 300+ LLMs and 20+ providers through a single, unified integration

​
Prerequisites

Before integrating the OpenAI SDK with Orq.ai, ensure the following are in place:
  • An Orq.ai account and API Key
  • Python 3.8+ or Node.js 18+ with TypeScript support
  • OpenAI SDK installed
To set up an API key, see API keys & Endpoints.
To use libraries with private models, see Onboarding Private Models.

​
Installation

npm install openai

​
Configuration

Set the Base URL to the AI Gateway to route calls through the API without changing any other part of the code. The Orq.ai AI Gateway provides Platform Traces and Cost and Usage Monitoring, keeping full compatibility and a unified API with all models while using the OpenAI SDK.
base_url: https://api.orq.ai/v3/router

​
Text Generation

Basic text generation with the OpenAI SDK through Orq.ai: 
import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "https://api.orq.ai/v3/router",
  apiKey: process.env.ORQ_API_KEY,
});

const response = await openai.responses.create({
  model: "openai/gpt-4o",
  instructions: "You are a helpful assistant.",
  input: "Hello!",
});

console.log(response.output_text);

​
Streaming Responses

Stream responses for real-time output: 
import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "https://api.orq.ai/v3/router",
  apiKey: process.env.ORQ_API_KEY,
});

const stream = await openai.responses.create({
  model: "openai/gpt-4o",
  input: "Write a short story about robots.",
  stream: true,
});

for await (const event of stream) {
  if (event.type === "response.output_text.delta") {
    process.stdout.write(event.delta);
  }
}

​
Model Selection

With Orq.ai, you can use any supported model from 20+ providers: 
import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "https://api.orq.ai/v3/router",
  apiKey: process.env.ORQ_API_KEY,
});

const claudeResponse = await openai.responses.create({
  model: "anthropic/claude-sonnet-4-6",
  input: "Explain machine learning",
});

const geminiResponse = await openai.responses.create({
  model: "google-ai/gemini-2.5-flash",
  input: "Explain machine learning",
});

const groqResponse = await openai.responses.create({
  model: "groq/llama-3.3-70b-versatile",
  input: "Explain machine learning",
});