AI Gateway

What is the AI Gateway?

Orq.ai AI Gateway is a unified gateway that provides seamless access to 15+ AI providers through a single, standardized API. The AI Gateway eliminates vendor lock-in and simplifies AI integration across your applications.

It is built for companies delivering AI-powered applications who need reliable, scalable access to multiple AI providers.

👍
Key Benefits

Unified Interface: One API for OpenAI, Anthropic, Google AI, AWS Bedrock, and 10+ more providers

Intelligent Routing: Automatic provider selection based on model availability and performance

Built-in Reliability: Automatic retries, fallbacks, and error handling with provider-specific optimizations

Cost Optimization: Smart routing to minimize costs while maximizing performance

Real-time Monitoring: Complete observability with metrics, tracing, and analytics

Supported Model Types

Type	Description	Streaming	Providers
Chat Completions	Conversational AI with message history	✅	All text providers
Text Completions	Single-turn text generation	✅	OpenAI, compatible providers
Embeddings	Vector representations for RAG	❌	OpenAI, Jina, Cohere
Image Generation	AI-powered image creation	❌	Fal, LeonardoAI, OpenAI
Image Editing	Modify existing images	❌	OpenAI, LeonardoAI
Vision	Analyze images and answer questions	✅	OpenAI, Anthropic, Google
Speech-to-Text	Transcription and translation	❌	OpenAI, AWS
Text-to-Speech	High-quality audio generation	❌	ElevenLabs, OpenAI
Moderation	Content safety and compliance	❌	OpenAI
Reranking	Optimize search results	❌	Jina, Cohere

📘
To learn more about supported models, see Supported Models.

Quick Start

Basic Chat Completion

curl -X POST https://api.orq.ai/v2/proxy/chat/completions \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Hello, world!"}]
  }'

Streaming Response

curl -X POST https://api.orq.ai/v2/proxy/chat/completions \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Tell me a story"}],
    "stream": true
  }'

Multi-Provider Fallback

curl -X POST https://api.orq.ai/v2/proxy/chat/completions \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": ["gpt-4", "claude-3-5-sonnet-20241022", "gemini-1.5-pro"],
    "messages": [{"role": "user", "content": "Analyze this data"}]
  }'

Getting Started

Essential information for integrating with the AI Gateway:

Supported Models

Complete list of supported models across all 15+ AI providers and model types.

Framework Integrations

Direct integrations with popular AI libraries including LangChain, DSPy, Instructor, and more.

OpenAI-Compatible API

Unified OpenAI-compatible interface that standardizes input/output formats across providers.

Advanced Features

Explore the AI Gateway's comprehensive feature set designed for production AI applications:

Streaming

Real-time streaming responses for chat completions and text generation across all providers.

Tool Calling

Enable AI models to call external functions with structured parameters and automatic validation.

Fallbacks

Automatic provider fallback strategies for improved reliability and availability.

Retries & Error Handling

Advanced retry strategies with exponential backoff and provider-specific error handling.

Structured Outputs

Generate JSON and structured data with guaranteed schema compliance.

Vision & Multimodal

Analyze images alongside text for multimodal AI interactions with support for multiple formats and streaming.

Caching

Intelligent response caching system for reduced latency and cost optimization.

Load Balancing

Intelligent load distribution across multiple providers for optimal performance.

Timeouts

Intelligent timeout management with automatic handling and graceful degradation.

Using Prompts

Leverage pre-created prompts from Orq.ai for consistent, version-controlled prompt management.

Thread Management

Organize and track multi-turn conversations with comprehensive conversation analysis.

Reasoning Models

Advanced reasoning capabilities with chain-of-thought and transparent thinking processes.

App Tracking

Track and identify AI requests from specific applications for better monitoring.

Contact Tracking

Associate requests with contact identifiers for user-level observability and analytics.

Dynamic Inputs

Replace variables in prompts using dynamic inputs for personalized responses.

PDF Input

Process PDF documents for text extraction and analysis across multiple providers.