Skip to main content

AI Gateway

Route your LLM calls through the AI Gateway with a single base URL change. Zero vendor lock-in: always run on the best model at the lowest cost for your use case.

Observability

Instrument your code with OpenTelemetry to capture traces, logs, and metrics for every LLM call, agent step, and tool use.

AI Gateway

Overview

The Vercel AI SDK provides TypeScript and Python toolkits for building AI-powered applications with streaming, tools, and multi-model support. The TypeScript SDK can also connect to Orq.ai’s AI Gateway through @orq-ai/vercel-provider for access to 500+ models with a single provider setup.

Key Benefits

Orq.ai’s AI Gateway enhances your Vercel AI applications with:

Complete Observability

Track every generation, stream, and structured output with detailed traces

Built-in Reliability

Automatic fallbacks, retries, and load balancing for production resilience

Cost Optimization

Real-time cost tracking and spend management across all your AI operations

Multi-Provider Access

Access 500+ LLMs and 30+ providers through a single, unified integration

Prerequisites

Before integrating Vercel AI with Orq.ai, ensure you have:
  • An Orq.ai account and API Key
  • Node.js 18 or higher
To setup your API key, see API keys & Endpoints.

Installation

Configuration

Configure the Orq.ai provider with your API key:
TypeScript
base_url: https://my.orq.ai/v3/router

Text Generation

TypeScript

Streaming Responses

TypeScript

Structured Output

Use a JSON system prompt and parse the response:
TypeScript

Model Selection

With Orq.ai, you can use any supported model from 30+ providers:
TypeScript

Observability

Getting Started

Both Vercel AI SDK implementations expose experimental OpenTelemetry support that captures agent runs, model calls, tool executions, token usage, and errors. Export those traces to Orq.ai by pointing a standard OpenTelemetry OTLP exporter at the Orq.ai collector: the Python SDK through its experimental_telemetry OpenTelemetry adapter, and the TypeScript SDK through its native experimental_telemetry option.
Vercel currently marks Python telemetry as experimental. Its API and emitted span attributes may change between releases.

Prerequisites

Before you begin, ensure you have:
  • An Orq.ai account and an API Key.
  • Vercel AI SDK for TypeScript v3.1+, or Vercel AI SDK for Python 0.4+.
  • Node.js 18+ (or Bun 1.3.5+) for TypeScript, or Python 3.12+ for Python.
  • API keys for your LLM providers (OpenAI, Anthropic, etc.).

Install Dependencies

The otel extra adds Vercel’s OpenTelemetry adapter. Replace anthropic with the provider extra the application uses, such as openai.

Configure the TypeScript telemetry environment

Set up your environment variables to connect to Orq.ai’s OpenTelemetry collector: Unix/Linux/macOS:
Windows (PowerShell):
Using .env file:

Integrations

Python AI SDK

The Python SDK maps its telemetry onto OpenTelemetry through an OtelAdapter. Configure a standard OpenTelemetry TracerProvider that exports to the Orq.ai collector, then register the adapter once at startup:
Python
register() installs the adapter globally, so every agent run, model call, and tool execution after it is traced. capture_content=True includes message content in spans. Call provider.force_flush() before a short-lived script exits so buffered spans are sent. Model ids in provider:model form call the provider directly using its API key, such as ANTHROPIC_API_KEY. Ids in provider/model form route through the Vercel AI Gateway and require AI_GATEWAY_API_KEY instead.

TypeScript AI SDK

The TypeScript SDK has built-in OpenTelemetry support through the experimental_telemetry option:
Built-in Telemetry (Recommended)
The examples in this section call the openai provider from @ai-sdk/openai directly. This routes requests to OpenAI without going through the AI Gateway. To route through Orq.ai, replace openai with the provider from @orq-ai/vercel-provider as shown in the Text Generation section above.
The simplest way to enable telemetry is using the SDK’s native support:
Here the main factor to enable telemetry is to include the following payload when generating text. This can be used across the board.

Asset Capture in the Control Tower

When you instrument Vercel AI SDK with OpenTelemetry and send traces to Orq.ai, agents, tools, and models are automatically extracted from the spans and registered in Control Tower.

Installation

Configuration

TypeScript

Agent Detection

The example below calls the openai provider from @ai-sdk/openai directly, bypassing the AI Gateway. Replace openai with the provider from @orq-ai/vercel-provider to route through Orq.ai.
The Vercel AI SDK does not have a built-in way to mark a span as an agent. To capture agents in Control Tower, use manual OpenTelemetry instrumentation with tracer.startActiveSpan(). The span name (e.g., "translator-agent") becomes the agent name in Control Tower. This approach captures tools and models alongside the agent span. Captures: agent/translator-agent, tool/translate, model/gpt-5.6-sol
TypeScript

Evaluations & Experiments

Once your agents are running, use Evaluatorq to score outputs across a dataset and Experiments to compare configurations side-by-side.

Run Evaluations with Evaluatorq

Run parallel evaluations across your agents and compare results.

Run Experiments via the API

Compare agent configurations and view results in the AI Studio.