Skip to main content

AI Router

Route your LLM calls through the AI Router with a single base URL change. Zero vendor lock-in: always run on the best model at the lowest cost for your use case.

Observability

Instrument your code with OpenTelemetry to capture traces, logs, and metrics for every LLM call, agent step, and tool use.

​
AI Router

​
Overview

LlamaIndex is a powerful framework for building RAG (Retrieval-Augmented Generation) applications with LLMs. By connecting LlamaIndex to Orq.ai’s AI Router, you transform experimental RAG applications into production-ready systems with enterprise-grade capabilities.

​
Key Benefits

Orq.ai’s AI Router enhances your LlamaIndex applications with:

Complete Observability

Track document indexing, retrieval performance, and query processing with detailed traces

Built-in Reliability

Automatic fallbacks, retries, and load balancing for production resilience

Cost Optimization

Real-time cost tracking and spend management across all your AI operations

Multi-Provider Access

Access 300+ LLMs and 20+ providers through a single, unified integration

​
Prerequisites

Before integrating LlamaIndex with Orq.ai, ensure you have:
  • An Orq.ai account and API Key
  • Python 3.8 or higher
  • LlamaIndex installed in your project
To setup your API key, see API keys & Endpoints.

​
Installation

Install LlamaIndex and required dependencies: 
pip install llama-index llama-index-llms-openai-like

​
Configuration

Configure LlamaIndex to use Orq.ai’s AI Router with the OpenAILike class:
Python
from llama_index.llms.openai_like import OpenAILike
import os

# Configure OpenAI-compatible LLM with Orq.ai AI Router
llm = OpenAILike(
    model="gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v2/router",
    is_chat_model=True,
)
api_base: https://api.orq.ai/v2/router

​
Basic RAG Example

Here’s a complete example of building a RAG application with LlamaIndex through Orq.ai:
Python
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.llms.openai_like import OpenAILike
from llama_index.embeddings.openai import OpenAIEmbedding
import os

# Configure LLM with Orq.ai AI Router
llm = OpenAILike(
    model="gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v2/router",
    is_chat_model=True,
)

# Configure embeddings through Orq.ai (required - LlamaIndex defaults to OpenAI)
embed_model = OpenAIEmbedding(
    model="text-embedding-3-small",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v2/router",
)

# Set as global defaults
Settings.llm = llm
Settings.embed_model = embed_model

# Load documents and create index
documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents)

# Query the index
query_engine = index.as_query_engine()
response = query_engine.query("What is the main topic of these documents?")
print(response)

​
Model Selection

With Orq.ai, you can use any supported model from 20+ providers:
Python
from llama_index.llms.openai_like import OpenAILike
import os

# Use Claude
claude_llm = OpenAILike(
    model="claude-sonnet-4-5-20250929",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v2/router",
    is_chat_model=True,
)

# Use Gemini
gemini_llm = OpenAILike(
    model="gemini-2.5-flash",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v2/router",
    is_chat_model=True,
)

# Use any other model
groq_llm = OpenAILike(
    model="llama-3.3-70b-versatile",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v2/router",
    is_chat_model=True,
)

​
Streaming Responses

LlamaIndex supports streaming with Orq.ai:
Python
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.llms.openai_like import OpenAILike
from llama_index.embeddings.openai import OpenAIEmbedding
import os

# Configure LLM
llm = OpenAILike(
    model="gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v2/router",
    is_chat_model=True,
)

# Configure embeddings
embed_model = OpenAIEmbedding(
    model="text-embedding-3-small",
    api_key=os.getenv("ORQ_API_KEY"),
    api_base="https://api.orq.ai/v2/router",
)

Settings.llm = llm
Settings.embed_model = embed_model

# Create index and query engine
documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine(streaming=True)

# Stream response
streaming_response = query_engine.query("Explain the main concepts")
for text in streaming_response.response_gen:
    print(text, end="", flush=True)
print()

​
Observability

​
Getting Started

Integrate LlamaIndex with Orq.ai’s observability to gain comprehensive insights into document indexing, retrieval performance, query processing, and LLM interactions using OpenTelemetry.

​
Prerequisites

Before you begin, ensure you have:
  • An Orq.ai account and API Key
  • LlamaIndex installed in your project
  • Python 3.8+
  • OpenAI API key (or other LLM provider credentials)

​
Install Dependencies

# Core LlamaIndex and OpenInference packages
pip install llama-index openinference-instrumentation-llama-index

# OpenTelemetry packages
pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http

# LLM providers
pip install openai anthropic

# Optional: For advanced vector stores and embeddings
pip install llama-index-vector-stores-chroma llama-index-embeddings-openai

​
Configure Orq.ai

Set up your environment variables to connect to Orq.ai’s OpenTelemetry collector: Unix/Linux/macOS: 
export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.orq.ai/v2/otel"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer <ORQ_API_KEY>"
export OTEL_RESOURCE_ATTRIBUTES="service.name=llamaindex-app,service.version=1.0.0"
export OPENAI_API_KEY="<YOUR_OPENAI_API_KEY>"
Windows (PowerShell): 
$env:OTEL_EXPORTER_OTLP_ENDPOINT = "https://api.orq.ai/v2/otel"
$env:OTEL_EXPORTER_OTLP_HEADERS = "Authorization=Bearer <ORQ_API_KEY>"
$env:OTEL_RESOURCE_ATTRIBUTES = "service.name=llamaindex-app,service.version=1.0.0"
$env:OPENAI_API_KEY = "<YOUR_OPENAI_API_KEY>"
Using .env file: 
OTEL_EXPORTER_OTLP_ENDPOINT=https://api.orq.ai/v2/otel
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer <ORQ_API_KEY>
OTEL_RESOURCE_ATTRIBUTES=service.name=llamaindex-app,service.version=1.0.0
OPENAI_API_KEY=<YOUR_OPENAI_API_KEY>

​
Integration Example

We’ll be using OpenInference as TracerProvider with LlamaIndex
Python
from openinference.instrumentation.llama_index import LlamaIndexInstrumentor
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk import trace as trace_sdk
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

# Initialize OpenTelemetry
tracer_provider = trace_sdk.TracerProvider()
tracer_provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter(
    endpoint="https://api.orq.ai/v2/otel/v1/traces",
    headers={"Authorization": "Bearer <ORQ_API_KEY>"}
)))
trace.set_tracer_provider(tracer_provider)

# Instrument LlamaIndex
LlamaIndexInstrumentor().instrument(tracer_provider=tracer_provider)

# Your LlamaIndex code is automatically traced
documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents)

query_engine = index.as_query_engine()
response = query_engine.query("What is the main topic of these documents?")

​
View Traces

View your traces in the AI Studio in the Traces tab.
Visit your AI Studio to view real-time analytics and traces.