AI Gateway
Route your LLM calls through the AI Gateway with a single base URL change. Zero vendor lock-in: always run on the best model at the lowest cost for your use case.
Observability
Instrument your code with OpenTelemetry to capture traces, logs, and metrics for every LLM call, agent step, and tool use.
AI Gateway
Overview
The Vercel AI SDK provides TypeScript and Python toolkits for building AI-powered applications with streaming, tools, and multi-model support. The TypeScript SDK can also connect to Orq.ai’s AI Gateway through@orq-ai/vercel-provider for access to 500+ models with a single provider setup.
Key Benefits
Orq.ai’s AI Gateway enhances your Vercel AI applications with:Complete Observability
Track every generation, stream, and structured output with detailed traces
Built-in Reliability
Automatic fallbacks, retries, and load balancing for production resilience
Cost Optimization
Real-time cost tracking and spend management across all your AI operations
Multi-Provider Access
Access 500+ LLMs and 30+ providers through a single, unified integration
Prerequisites
Before integrating Vercel AI with Orq.ai, ensure you have:- An Orq.ai account and API Key
- Node.js 18 or higher
To setup your API key, see API keys & Endpoints.
Installation
Configuration
Configure the Orq.ai provider with your API key:TypeScript
base_url: https://my.orq.ai/v3/router
Text Generation
TypeScript
Streaming Responses
TypeScript
Structured Output
Use a JSON system prompt and parse the response:TypeScript
Model Selection
With Orq.ai, you can use any supported model from 30+ providers:TypeScript
Observability
Getting Started
Both Vercel AI SDK implementations expose experimental OpenTelemetry support that captures agent runs, model calls, tool executions, token usage, and errors. Export those traces to Orq.ai by pointing a standard OpenTelemetry OTLP exporter at the Orq.ai collector: the Python SDK through itsexperimental_telemetry OpenTelemetry adapter, and the TypeScript SDK through its native experimental_telemetry option.
Prerequisites
Before you begin, ensure you have:- An Orq.ai account and an API Key.
- Vercel AI SDK for TypeScript v3.1+, or Vercel AI SDK for Python 0.4+.
- Node.js 18+ (or Bun 1.3.5+) for TypeScript, or Python 3.12+ for Python.
- API keys for your LLM providers (OpenAI, Anthropic, etc.).
Install Dependencies
- Python
- TypeScript
otel extra adds Vercel’s OpenTelemetry adapter. Replace anthropic with the provider extra the application uses, such as openai.Configure the TypeScript telemetry environment
Set up your environment variables to connect to Orq.ai’s OpenTelemetry collector: Unix/Linux/macOS:Integrations
Python AI SDK
The Python SDK maps its telemetry onto OpenTelemetry through anOtelAdapter. Configure a standard OpenTelemetry TracerProvider that exports to the Orq.ai collector, then register the adapter once at startup:
Python
register() installs the adapter globally, so every agent run, model call, and tool execution after it is traced. capture_content=True includes message content in spans. Call provider.force_flush() before a short-lived script exits so buffered spans are sent.
Model ids in provider:model form call the provider directly using its API key, such as ANTHROPIC_API_KEY. Ids in provider/model form route through the Vercel AI Gateway and require AI_GATEWAY_API_KEY instead.
TypeScript AI SDK
The TypeScript SDK has built-in OpenTelemetry support through theexperimental_telemetry option:
Built-in Telemetry (Recommended)
The examples in this section call the
openai provider from @ai-sdk/openai directly. This routes requests to OpenAI without going through the AI Gateway. To route through Orq.ai, replace openai with the provider from @orq-ai/vercel-provider as shown in the Text Generation section above.Asset Capture in the Control Tower
When you instrument Vercel AI SDK with OpenTelemetry and send traces to Orq.ai, agents, tools, and models are automatically extracted from the spans and registered in Control Tower.Installation
Configuration
TypeScript
Agent Detection
The example below calls the
openai provider from @ai-sdk/openai directly, bypassing the AI Gateway. Replace openai with the provider from @orq-ai/vercel-provider to route through Orq.ai.tracer.startActiveSpan(). The span name (e.g., "translator-agent") becomes the agent name in Control Tower. This approach captures tools and models alongside the agent span.
Captures: agent/translator-agent, tool/translate, model/gpt-5.6-sol
TypeScript
Evaluations & Experiments
Once your agents are running, use Evaluatorq to score outputs across a dataset and Experiments to compare configurations side-by-side.Run Evaluations with Evaluatorq
Run parallel evaluations across your agents and compare results.
Run Experiments via the API
Compare agent configurations and view results in the AI Studio.