Orq.ai Documentation - AI Gateway & LLM Collaboration Platform

AI Router

Route your LLM calls through the AI Router with a single base URL change. Zero vendor lock-in: always run on the best model at the lowest cost for your use case.

Observability

Instrument your code with OpenTelemetry to capture traces, logs, and metrics for every LLM call, agent step, and tool use.

AI Router

Overview

The Vercel AI SDK is a TypeScript toolkit for building AI-powered applications with streaming, structured outputs, and multi-model support. By connecting it to Orq.ai’s AI Router via the @orq-ai/vercel-provider, you get access to 300+ models with a single provider setup.

Key Benefits

Orq.ai’s AI Router enhances your Vercel AI applications with:

Complete Observability

Track every generation, stream, and structured output with detailed traces

Built-in Reliability

Automatic fallbacks, retries, and load balancing for production resilience

Cost Optimization

Real-time cost tracking and spend management across all your AI operations

Multi-Provider Access

Access 300+ LLMs and 20+ providers through a single, unified integration

Prerequisites

Before integrating Vercel AI with Orq.ai, ensure you have:

An Orq.ai account and API Key
Node.js 18 or higher

To setup your API key, see API keys & Endpoints.

Installation

npm install @orq-ai/vercel-provider ai

Configuration

Configure the Orq.ai provider with your API key:

TypeScript

import { createOrqAiProvider } from "@orq-ai/vercel-provider";

const orq = createOrqAiProvider({
  apiKey: process.env.ORQ_API_KEY,
});

base_url: https://api.orq.ai/v3/router

Text Generation

TypeScript

import { createOrqAiProvider } from "@orq-ai/vercel-provider";
import { generateText } from "ai";

const orq = createOrqAiProvider({
  apiKey: process.env.ORQ_API_KEY,
});

const { text } = await generateText({
  model: orq("openai/gpt-4o"),
  prompt: "Write a haiku about programming",
});

console.log(text);

Streaming Responses

TypeScript

import { createOrqAiProvider } from "@orq-ai/vercel-provider";
import { streamText } from "ai";

const orq = createOrqAiProvider({
  apiKey: process.env.ORQ_API_KEY,
});

const { textStream } = await streamText({
  model: orq("openai/gpt-4o"),
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "Explain quantum computing in two sentences." },
  ],
});

for await (const chunk of textStream) {
  process.stdout.write(chunk);
}

Structured Output

Use a JSON system prompt and parse the response:

TypeScript

import { createOrqAiProvider } from "@orq-ai/vercel-provider";
import { generateText } from "ai";

const orq = createOrqAiProvider({
  apiKey: process.env.ORQ_API_KEY,
});

const { text } = await generateText({
  model: orq("openai/gpt-4o"),
  messages: [
    {
      role: "system",
      content: "You are a data assistant. Always respond with valid JSON only, no markdown.",
    },
    {
      role: "user",
      content: "Generate information about France with fields: name, capital, population, languages.",
    },
  ],
});

const country = JSON.parse(text);
console.log(country);

Model Selection

With Orq.ai, you can use any supported model from 20+ providers:

TypeScript

import { createOrqAiProvider } from "@orq-ai/vercel-provider";
import { generateText } from "ai";

const orq = createOrqAiProvider({
  apiKey: process.env.ORQ_API_KEY,
});

// Use Claude
const claudeResult = await generateText({
  model: orq("anthropic/claude-sonnet-4-6"),
  prompt: "What is the largest planet?",
});

// Use Gemini
const geminiResult = await generateText({
  model: orq("google/gemini-2.5-flash"),
  prompt: "What is the largest planet?",
});

// Use Groq
const groqResult = await generateText({
  model: orq("groq/llama-3.3-70b-versatile"),
  prompt: "What is the largest planet?",
});

Observability

Getting Started

The Vercel AI SDK provides powerful React hooks and utilities for building AI-powered applications with built-in OpenTelemetry support. The SDK includes experimental telemetry features that automatically capture detailed traces of AI operations, making integration with Orq.ai straightforward for comprehensive observability.

Prerequisites

Before you begin, ensure you have:

An Orq.ai account and an API Key.
Vercel AI SDK v3.1+ (with telemetry support).
Node.js 18+ and TypeScript support.
API keys for your LLM providers (OpenAI, Anthropic, etc.).

Install Dependencies

# Core Vercel AI SDK with latest version
npm install ai@latest

# OpenTelemetry packages
npm install @opentelemetry/api @opentelemetry/sdk-node @opentelemetry/exporter-trace-otlp-http
npm install @opentelemetry/instrumentation @opentelemetry/resources
npm install @opentelemetry/semantic-conventions

# Provider SDKs (choose what you need)
npm install @ai-sdk/openai @ai-sdk/anthropic @ai-sdk/google

# Optional: For React applications
npm install @ai-sdk/react

Configure Orq.ai

Set up your environment variables to connect to Orq.ai’s OpenTelemetry collector: Unix/Linux/macOS:

export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.orq.ai/v2/otel"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer <ORQ_API_KEY>"
export OTEL_RESOURCE_ATTRIBUTES="service.name=vercel-ai-app,service.version=1.0.0"
export OPENAI_API_KEY="<YOUR_OPENAI_API_KEY>"

Windows (PowerShell):

$env:OTEL_EXPORTER_OTLP_ENDPOINT = "https://api.orq.ai/v2/otel"
$env:OTEL_EXPORTER_OTLP_HEADERS = "Authorization=Bearer <ORQ_API_KEY>"
$env:OTEL_RESOURCE_ATTRIBUTES = "service.name=vercel-ai-app,service.version=1.0.0"
$env:OPENAI_API_KEY = "<YOUR_OPENAI_API_KEY>"

Using .env file:

OTEL_EXPORTER_OTLP_ENDPOINT=https://api.orq.ai/v2/otel
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer <ORQ_API_KEY>
OTEL_RESOURCE_ATTRIBUTES=service.name=vercel-ai-app,service.version=1.0.0
OPENAI_API_KEY=<YOUR_OPENAI_API_KEY>

Integrations

The Vercel AI SDK has built-in OpenTelemetry support through the experimental_telemetry option. Here’s how to integrate it with Orq.ai:

Built-in Telemetry (Recommended)

The examples in this section call the openai provider from @ai-sdk/openai directly. This routes requests to OpenAI without going through the AI Router. To route through Orq.ai, replace openai with the provider from @orq-ai/vercel-provider as shown in the Text Generation section above.

The simplest way to enable telemetry is using the SDK’s native support:

// instrumentation.js
import { registerOTel, OTLPHttpJsonTraceExporter } from '@vercel/otel';

export function register() {
  registerOTel({
    serviceName: 'your-project-name',
    traceExporter: new OTLPHttpJsonTraceExporter({
      url: 'https://api.orq.ai/v2/otel/v1/traces',
      headers: {
        'Authorization': `Bearer ${process.env.ORQ_API_KEY}`,
      },
    }),
  });
}

// index.js
import './instrumentation.js'
import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";

// Simple usage with telemetry enabled
const result = await generateText({
  model: openai("gpt-4.1"),
  prompt: "Write a short story about a robot",
  experimental_telemetry: {
    isEnabled: true,
  },
});

// Advanced configuration with custom metadata
const resultWithMetadata = await generateText({
  model: openai("gpt-4.1"),
  prompt: "Explain quantum computing",
  experimental_telemetry: {
    isEnabled: true,
    functionId: "quantum-explanation",
    metadata: {
      userId: "user-123",
      requestId: "req-456",
      environment: "production",
    },
  },
});

// Control what data is recorded
const userPrompt = "What is the meaning of life, the universe, and everything?";
const resultWithPrivacy = await generateText({
  model: openai("gpt-4.1"),
  prompt: userPrompt,
  experimental_telemetry: {
    isEnabled: true,
    recordInputs: false, // Don't record prompts
    recordOutputs: false, // Don't record responses
  },
});

Here the main factor to enable telemetry is to include the following payload when generating text. This can be used across the board.

  experimental_telemetry: {
    isEnabled: true,
    recordInputs: false, // Don't record prompts
    recordOutputs: false, // Don't record responses
  },

Asset Capture in the Control Tower

When you instrument Vercel AI SDK with OpenTelemetry and send traces to Orq.ai, agents, tools, and models are automatically extracted from the spans and registered in Control Tower.

Installation

npm install ai @ai-sdk/openai \
               @opentelemetry/api \
               @opentelemetry/sdk-node \
               @opentelemetry/exporter-trace-otlp-http

Configuration

TypeScript

import { NodeSDK } from "@opentelemetry/sdk-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
import { BatchSpanProcessor } from "@opentelemetry/sdk-trace-node";

const exporter = new OTLPTraceExporter({
  url: "https://api.orq.ai/v2/otel/v1/traces",
  headers: {
    Authorization: `Bearer ${process.env.ORQ_API_KEY}`,
  },
});

const sdk = new NodeSDK({
  spanProcessor: new BatchSpanProcessor(exporter),
  serviceName: "my-ai-app",
});

sdk.start();

Agent Detection

The example below calls the openai provider from @ai-sdk/openai directly, bypassing the AI Router. Replace openai with the provider from @orq-ai/vercel-provider to route through Orq.ai.

The Vercel AI SDK does not have a built-in way to mark a span as an agent. To capture agents in Control Tower, use manual OpenTelemetry instrumentation with tracer.startActiveSpan(). The span name (e.g., "translator-agent") becomes the agent name in Control Tower. This approach captures tools and models alongside the agent span. Captures: agent/translator-agent, tool/translate, model/gpt-4o

TypeScript

import { NodeSDK } from "@opentelemetry/sdk-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
import { BatchSpanProcessor } from "@opentelemetry/sdk-trace-node";
import { openai } from "@ai-sdk/openai";
import { context, trace } from "@opentelemetry/api";
import { generateText, jsonSchema, stepCountIs, tool } from "ai";

const exporter = new OTLPTraceExporter({
  url: "https://api.orq.ai/v2/otel/v1/traces",
  headers: { Authorization: `Bearer ${process.env.ORQ_API_KEY}` },
});

const sdk = new NodeSDK({
  spanProcessor: new BatchSpanProcessor(exporter),
  serviceName: "my-ai-app",
});

sdk.start();

const translateTool = tool({
  description: "Translate text to a target language",
  inputSchema: jsonSchema<{ text: string; target_language: string }>({
    type: "object",
    properties: {
      text: { type: "string", description: "Text to translate" },
      target_language: { type: "string", description: "Target language" },
    },
    required: ["text", "target_language"],
  }),
  execute: async ({ text, target_language }) => {
    const translations: Record<string, Record<string, string>> = {
      spanish: { "Hello, how are you?": "¡Hola, ¿cómo estás?" },
      french: { "Hello, how are you?": "Bonjour, comment allez-vous?" },
      japanese: { "Hello, how are you?": "こんにちは、お元気ですか？" },
    };
    return (
      translations[target_language.toLowerCase()]?.[text] ||
      `[${target_language}] ${text}`
    );
  },
});

const tracer = trace.getTracer("ai-sdk-app");

await tracer.startActiveSpan("translator-agent", async (parentSpan) => {
  parentSpan.setAttribute("input", 'Translate "Hello, how are you?" to Spanish');

  await context.with(
    trace.setSpan(context.active(), parentSpan),
    async () => {
      try {
        const { text } = await generateText({
          model: openai("gpt-4o"),
          system: "You are a translator agent that helps users translate text to different languages.",
          prompt: 'Translate "Hello, how are you?" to Spanish',
          tools: { translate: translateTool },
          stopWhen: stepCountIs(3),
          experimental_telemetry: {
            isEnabled: true,
            functionId: "translator_agent",
          },
        });

        console.log(text);
        parentSpan.setAttribute("output", text);
      } finally {
        parentSpan.end();
      }
    },
  );
});

await sdk.shutdown();

Evaluations & Experiments

Once your agents are running, use Evaluatorq to score outputs across a dataset and Experiments to compare configurations side-by-side.

Run Evaluations with Evaluatorq

Run parallel evaluations across your agents and compare results.

Run Experiments via the API

Compare agent configurations and view results in the AI Studio.

Documentation Index

AI Router

Observability

​AI Router

​Overview

​Key Benefits

Complete Observability

Built-in Reliability

Cost Optimization

Multi-Provider Access

​Prerequisites

​Installation

​Configuration

​Text Generation

​Streaming Responses

​Structured Output

​Model Selection

​Observability

​Getting Started

​Prerequisites

​Install Dependencies

​Configure Orq.ai

​Integrations

​Built-in Telemetry (Recommended)

​Asset Capture in the Control Tower

​Installation

​Configuration

​Agent Detection

​Evaluations & Experiments

Run Evaluations with Evaluatorq

Run Experiments via the API

AI Router

Overview

Key Benefits

Prerequisites

Installation

Configuration

Text Generation

Streaming Responses

Structured Output

Model Selection

Observability

Getting Started

Prerequisites

Install Dependencies

Configure Orq.ai

Integrations

Built-in Telemetry (Recommended)

Asset Capture in the Control Tower

Installation

Configuration

Agent Detection

Evaluations & Experiments