AI Gateway
Route your LLM calls through the AI Gateway with a single base URL change. Zero vendor lock-in: always run on the best model at the lowest cost for your use case.
Observability
Instrument your code with OpenTelemetry to capture traces, logs, and metrics for every LLM call, agent step, and tool use.
AI Gateway
Overview
The Vercel AI SDK provides TypeScript and Python toolkits for building AI-powered applications with streaming, tools, and multi-model support. The TypeScript SDK can also connect to Orq.ai’s AI Gateway through@orq-ai/vercel-provider for access to 500+ models with a single provider setup.
Key Benefits
Orq.ai’s AI Gateway enhances your Vercel AI applications with:Complete Observability
Track every generation, stream, and structured output with detailed traces
Built-in Reliability
Automatic fallbacks, retries, and load balancing for production resilience
Cost Optimization
Real-time cost tracking and spend management across all your AI operations
Multi-Provider Access
Access 500+ LLMs and 30+ providers through a single, unified integration
Prerequisites
Before integrating Vercel AI with Orq.ai, ensure you have:- An Orq.ai account and API Key
- Node.js 18 or higher
To setup your API key, see API keys & Endpoints.
Installation
Configuration
Configure the Orq.ai provider with your API key:TypeScript
base_url: https://my.orq.ai/v3/router
Text Generation
TypeScript
Streaming Responses
TypeScript
Structured Output
Use a JSON system prompt and parse the response:TypeScript
Model Selection
With Orq.ai, you can use any supported model from 30+ providers:TypeScript
Observability
Getting Started
Both Vercel AI SDK implementations expose OpenTelemetry support that captures agent runs, model calls, tool executions, token usage, and errors. Export those traces to Orq.ai by pointing a standard OpenTelemetry OTLP exporter at the Orq.ai collector: the Python SDK through itsexperimental_telemetry OpenTelemetry adapter, and the TypeScript SDK through the @ai-sdk/otel package.
Prerequisites
Before you begin, ensure you have:- An Orq.ai account and an API Key.
- Vercel AI SDK for TypeScript v7+, or v3.1 through v6 for the legacy setup. Vercel AI SDK for Python 0.4+.
- Node.js 18+ (or Bun 1.3.5+) for TypeScript, or Python 3.12+ for Python.
- API keys for your LLM providers (OpenAI, Anthropic, etc.).
Install Dependencies
- Python
- TypeScript
otel extra adds Vercel’s OpenTelemetry adapter. Replace anthropic with the provider extra the application uses, such as openai.Configure the TypeScript telemetry environment
Set up your environment variables to connect to Orq.ai’s OpenTelemetry collector: Unix/Linux/macOS:Integrations
Python AI SDK
The Python SDK maps its telemetry onto OpenTelemetry through anOtelAdapter. Configure a standard OpenTelemetry TracerProvider that exports to the Orq.ai collector, then register the adapter once at startup:
Python
register() installs the adapter globally, so every agent run, model call, and tool execution after it is traced. capture_content=True includes message content in spans. Call provider.force_flush() before a short-lived script exits so buffered spans are sent.
Model ids in provider:model form call the provider directly using its API key, such as ANTHROPIC_API_KEY. Ids in provider/model form route through the Vercel AI Gateway and require AI_GATEWAY_API_KEY instead.
TypeScript AI SDK
Telemetry setup differs by major version. On v7 and later, OpenTelemetry collection lives in the@ai-sdk/otel package and is registered once at startup. On v6 and earlier, it is enabled per call through experimental_telemetry.
The examples in this section call the
openai provider from @ai-sdk/openai directly. This routes requests to OpenAI without going through the AI Gateway. To route through Orq.ai, replace openai with the provider from @orq-ai/vercel-provider as shown in the Text Generation section above.v7 and later
Register the OTLP exporter, then register the AI SDK telemetry integration once at startup. This is the Next.js setup, where the framework callsregister() from its instrumentation hook. For standalone Node.js, wire the exporter with NodeSDK as shown in Asset Capture in the Control Tower.
experimental_telemetry (or its telemetry alias). Set isEnabled: false to suppress a single call, recordInputs or recordOutputs to false to keep prompts or responses out of the spans, and functionId to name the operation. functionId becomes gen_ai.agent.name on the invoke_agent span, which is what Orq.ai registers as the agent.
OpenTelemetry emits the OpenTelemetry GenAI semantic conventions (gen_ai.* attributes). @ai-sdk/otel also exports LegacyOpenTelemetry, which emits the v6 ai.* attributes instead. Orq.ai ingests both. For the full set of integration options, see the AI SDK telemetry documentation.
v6 and earlier
Enable telemetry per call through theexperimental_telemetry option:
Asset Capture in the Control Tower
When you instrument Vercel AI SDK with OpenTelemetry and send traces to Orq.ai, agents, tools, and models are automatically extracted from the spans and registered in Control Tower.Installation
Configuration
TypeScript
registerTelemetry is required on AI SDK v7 and later. On v6 and earlier, omit it and pass experimental_telemetry: { isEnabled: true } on each call instead.Agent Detection
The example below calls the
openai provider from @ai-sdk/openai directly, bypassing the AI Gateway. Replace openai with the provider from @orq-ai/vercel-provider to route through Orq.ai.tracer.startActiveSpan(). The span name (e.g., "translator-agent") becomes the agent name in Control Tower. This approach captures tools and models alongside the agent span.
The example below targets AI SDK v7. On v6 and earlier, drop
registerTelemetry and add isEnabled: true to the experimental_telemetry block instead.agent/translator-agent, tool/translate, model/gpt-5.6-sol
TypeScript
Evaluations & Experiments
Once your agents are running, use Evaluatorq to score outputs across a dataset and Experiments to compare configurations side-by-side.Run Evaluations with Evaluatorq
Run parallel evaluations across your agents and compare results.
Run Experiments via the API
Compare agent configurations and view results in the AI Studio.