Skip to main content

AI Gateway

Route your LLM calls through the AI Gateway with a single base URL change. Zero vendor lock-in: always run on the best model at the lowest cost for your use case.

AI Gateway

Overview

LlamaIndex Agents is an agentic framework built on top of LlamaIndex that enables LLMs to reason, use tools, and execute multi-step workflows. By connecting LlamaIndex Agents to Orq.ai’s AI Gateway, you get production-ready agents with enterprise-grade capabilities without changing your existing code.

Key Benefits

Orq.ai’s AI Gateway enhances your LlamaIndex Agents with:

Complete Observability

Track every agent step, tool use, and reasoning trace with detailed analytics

Built-in Reliability

Automatic fallbacks, retries, and load balancing for production resilience

Cost Optimization

Real-time cost tracking and spend management across all your AI operations

Multi-Provider Access

Access 300+ LLMs and 20+ providers through a single, unified integration

Prerequisites

Before integrating LlamaIndex Agents with Orq.ai, ensure you have:
  • An Orq.ai account and API Key
  • Python 3.9 or higher
  • LlamaIndex installed in your project
To setup your API key, see API keys & Endpoints.

Installation

Install LlamaIndex and required dependencies:

Configuration

Configure LlamaIndex Agents to use Orq.ai’s AI Gateway with the OpenAILike class:
Python
api_base: https://api.orq.ai/v3/router

Basic Agent Example

Here’s a complete example using ReActAgent with a simple tool:
Python

Agent with Multiple Tools

Build agents with multiple tools for complex reasoning:
Python

Model Selection

With Orq.ai, you can use any supported model from 20+ providers:
Python