Skip to main content

AI Gateway

Route your LLM calls through the AI Gateway with a single base URL change. Zero vendor lock-in: always run on the best model at the lowest cost for your use case.

Observability

Instrument your code with OpenTelemetry to capture traces, logs, and metrics for every LLM call, agent step, and tool use.

AI Gateway

Overview

The Vercel AI SDK provides TypeScript and Python toolkits for building AI-powered applications with streaming, tools, and multi-model support. The TypeScript SDK can also connect to Orq.ai’s AI Gateway through @orq-ai/vercel-provider for access to 500+ models with a single provider setup.

Key Benefits

Orq.ai’s AI Gateway enhances your Vercel AI applications with:

Complete Observability

Track every generation, stream, and structured output with detailed traces

Built-in Reliability

Automatic fallbacks, retries, and load balancing for production resilience

Cost Optimization

Real-time cost tracking and spend management across all your AI operations

Multi-Provider Access

Access 500+ LLMs and 30+ providers through a single, unified integration

Prerequisites

Before integrating Vercel AI with Orq.ai, ensure you have:
  • An Orq.ai account and API Key
  • Node.js 18 or higher
To setup your API key, see API keys & Endpoints.

Installation

Configuration

Configure the Orq.ai provider with your API key:
TypeScript
base_url: https://my.orq.ai/v3/router

Text Generation

TypeScript

Streaming Responses

TypeScript

Structured Output

Use a JSON system prompt and parse the response:
TypeScript

Model Selection

With Orq.ai, you can use any supported model from 30+ providers:
TypeScript

Observability

Getting Started

Both Vercel AI SDK implementations expose OpenTelemetry support that captures agent runs, model calls, tool executions, token usage, and errors. Export those traces to Orq.ai by pointing a standard OpenTelemetry OTLP exporter at the Orq.ai collector: the Python SDK through its experimental_telemetry OpenTelemetry adapter, and the TypeScript SDK through the @ai-sdk/otel package.
AI SDK for TypeScript v7 moved OpenTelemetry collection out of the ai package and into the separate @ai-sdk/otel package. Spans are only emitted once a telemetry integration is registered with registerTelemetry. Setting experimental_telemetry: { isEnabled: true } on its own emits nothing on v7, so traces never reach Orq.ai. See TypeScript AI SDK for the v7 setup.
Vercel currently marks Python telemetry as experimental. Its API and emitted span attributes may change between releases.

Prerequisites

Before you begin, ensure you have:
  • An Orq.ai account and an API Key.
  • Vercel AI SDK for TypeScript v7+, or v3.1 through v6 for the legacy setup. Vercel AI SDK for Python 0.4+.
  • Node.js 18+ (or Bun 1.3.5+) for TypeScript, or Python 3.12+ for Python.
  • API keys for your LLM providers (OpenAI, Anthropic, etc.).

Install Dependencies

The otel extra adds Vercel’s OpenTelemetry adapter. Replace anthropic with the provider extra the application uses, such as openai.

Configure the TypeScript telemetry environment

Set up your environment variables to connect to Orq.ai’s OpenTelemetry collector: Unix/Linux/macOS:
Windows (PowerShell):
Using .env file:

Integrations

Python AI SDK

The Python SDK maps its telemetry onto OpenTelemetry through an OtelAdapter. Configure a standard OpenTelemetry TracerProvider that exports to the Orq.ai collector, then register the adapter once at startup:
Python
register() installs the adapter globally, so every agent run, model call, and tool execution after it is traced. capture_content=True includes message content in spans. Call provider.force_flush() before a short-lived script exits so buffered spans are sent. Model ids in provider:model form call the provider directly using its API key, such as ANTHROPIC_API_KEY. Ids in provider/model form route through the Vercel AI Gateway and require AI_GATEWAY_API_KEY instead.

TypeScript AI SDK

Telemetry setup differs by major version. On v7 and later, OpenTelemetry collection lives in the @ai-sdk/otel package and is registered once at startup. On v6 and earlier, it is enabled per call through experimental_telemetry.
On v7, experimental_telemetry alone emits no spans. isEnabled gates a registered integration rather than installing one, so without registerTelemetry there is nothing to gate and the call produces no telemetry at all.
The examples in this section call the openai provider from @ai-sdk/openai directly. This routes requests to OpenAI without going through the AI Gateway. To route through Orq.ai, replace openai with the provider from @orq-ai/vercel-provider as shown in the Text Generation section above.
v7 and later
Register the OTLP exporter, then register the AI SDK telemetry integration once at startup. This is the Next.js setup, where the framework calls register() from its instrumentation hook. For standalone Node.js, wire the exporter with NodeSDK as shown in Asset Capture in the Control Tower.
Pass tracer: trace.getTracer('ai'). @ai-sdk/otel otherwise names its tracer gen_ai, which is the generic OpenTelemetry namespace rather than an AI SDK marker. Orq.ai identifies the framework from that instrumentation scope, so without it the traces are attributed to plain OpenTelemetry and the agents and tools they create are not linked to their runs, tokens, and cost in Control Tower.
Registration is global and opt-out: every AI SDK call emits spans afterwards, with no per-call flag.
Per-call options still apply through experimental_telemetry (or its telemetry alias). Set isEnabled: false to suppress a single call, recordInputs or recordOutputs to false to keep prompts or responses out of the spans, and functionId to name the operation. functionId becomes gen_ai.agent.name on the invoke_agent span, which is what Orq.ai registers as the agent.
OpenTelemetry emits the OpenTelemetry GenAI semantic conventions (gen_ai.* attributes). @ai-sdk/otel also exports LegacyOpenTelemetry, which emits the v6 ai.* attributes instead. Orq.ai ingests both. For the full set of integration options, see the AI SDK telemetry documentation.
v6 and earlier
Enable telemetry per call through the experimental_telemetry option:
Here the main factor to enable telemetry is to include the following payload when generating text. This can be used across the board.

Asset Capture in the Control Tower

When you instrument Vercel AI SDK with OpenTelemetry and send traces to Orq.ai, agents, tools, and models are automatically extracted from the spans and registered in Control Tower.

Installation

Configuration

TypeScript
registerTelemetry is required on AI SDK v7 and later. On v6 and earlier, omit it and pass experimental_telemetry: { isEnabled: true } on each call instead.

Agent Detection

The example below calls the openai provider from @ai-sdk/openai directly, bypassing the AI Gateway. Replace openai with the provider from @orq-ai/vercel-provider to route through Orq.ai.
The Vercel AI SDK does not have a built-in way to mark a span as an agent. To capture agents in Control Tower, use manual OpenTelemetry instrumentation with tracer.startActiveSpan(). The span name (e.g., "translator-agent") becomes the agent name in Control Tower. This approach captures tools and models alongside the agent span.
The example below targets AI SDK v7. On v6 and earlier, drop registerTelemetry and add isEnabled: true to the experimental_telemetry block instead.
Captures: agent/translator-agent, tool/translate, model/gpt-5.6-sol
TypeScript

Evaluations & Experiments

Once your agents are running, use Evaluatorq to score outputs across a dataset and Experiments to compare configurations side-by-side.

Run Evaluations with Evaluatorq

Run parallel evaluations across your agents and compare results.

Run Experiments via the API

Compare agent configurations and view results in the AI Studio.