Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.orq.ai/llms.txt

Use this file to discover all available pages before exploring further.

AI Router

Route your LLM calls through the AI Router with a single base URL change. Zero vendor lock-in: always run on the best model at the lowest cost for your use case.

Observability

Instrument your code with OpenTelemetry to capture traces, logs, and metrics for every LLM call, agent step, and tool use.

​
AI Router

​
Overview

LangGraph is a framework for building stateful, multi-actor AI applications with LLMs. It extends LangChain with graph-based agent orchestration, cycles, and controllability. By connecting LangGraph to Orq.ai’s AI Router, you get production-ready agentic workflows with access to 300+ models.

​
Key Benefits

Orq.ai’s AI Router enhances your LangGraph applications with:

Complete Observability

Track every agent step, tool use, and graph transition with detailed traces

Built-in Reliability

Automatic fallbacks, retries, and load balancing for production resilience

Cost Optimization

Real-time cost tracking and spend management across all your AI operations

Multi-Provider Access

Access 300+ LLMs and 20+ providers through a single, unified integration

​
Prerequisites

Before integrating LangGraph with Orq.ai, ensure you have:
  • An Orq.ai account and API Key
  • Python 3.8 or higher
To setup your API key, see API keys & Endpoints.

​
Installation

pip install langgraph langchain-openai langchain-core

​
Configuration

Configure LangGraph to use Orq.ai’s AI Router by passing a ChatOpenAI instance with a custom base_url:
Python
from langchain_openai import ChatOpenAI
import os

llm = ChatOpenAI(
    model="gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router",
)
base_url: https://api.orq.ai/v3/router

​
Basic Agent Example

Here’s a complete example using create_agent with a tool:
Python
from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os

llm = ChatOpenAI(
    model="gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router",
)

@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is sunny and 72°F"

agent = create_agent(llm, tools=[get_weather])

result = agent.invoke({"messages": [("user", "What's the weather in San Francisco?")]})
print(result["messages"][-1].content)

​
Agent with Multiple Tools

Python
from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os

llm = ChatOpenAI(
    model="gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router",
)

@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is sunny and 72°F"

@tool
def add(a: int, b: int) -> int:
    """Add two integers."""
    return a + b

@tool
def multiply(a: int, b: int) -> int:
    """Multiply two integers."""
    return a * b

agent = create_agent(
    llm,
    tools=[get_weather, add, multiply],
    system_prompt="You are a helpful assistant with access to weather and math tools.",
)

result = agent.invoke({
    "messages": [("user", "What is 15 * 4? Also check the weather in Tokyo.")]
})
print(result["messages"][-1].content)

​
Streaming

Stream agent steps as they happen:
Python
from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os

llm = ChatOpenAI(
    model="gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router",
)

@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is sunny and 72°F"

agent = create_agent(llm, tools=[get_weather])

for chunk in agent.stream(
    {"messages": [("user", "What's the weather in Paris?")]},
    stream_mode="updates",
):
    print(chunk)

​
Model Selection

With Orq.ai, you can use any supported model from 20+ providers:
Python
from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os

@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is sunny and 72°F"

# Use Claude
claude_agent = create_agent(
    ChatOpenAI(
        model="claude-sonnet-4-5-20250929",
        api_key=os.getenv("ORQ_API_KEY"),
        base_url="https://api.orq.ai/v3/router",
    ),
    tools=[get_weather],
)

# Use Gemini
gemini_agent = create_agent(
    ChatOpenAI(
        model="gemini-2.5-flash",
        api_key=os.getenv("ORQ_API_KEY"),
        base_url="https://api.orq.ai/v3/router",
    ),
    tools=[get_weather],
)

result = claude_agent.invoke({"messages": [("user", "What's the weather in London?")]})
print(result["messages"][-1].content)

​
Observability

orq_ai_sdk.langchain provides a global setup() function that automatically instruments all LangGraph components. Call it once at the top of your application and every LLM call, graph node, tool execution, and retrieval is traced automatically, no callback wiring needed.

Zero configuration

One setup() call and tracing is live, no callbacks, no OpenTelemetry exporters, no extra wiring.

Full graph visibility

Traces preserve the parent-child structure of your graph so you see exactly which node triggered each LLM call or tool use.

Token usage and costs

Input and output token counts are captured on every LLM call and synced to Orq.ai for cost tracking.

Asset Capture

Agents, tools, and models are automatically registered in Control Tower from your traces.

​
Installation

pip install orq-ai-sdk langchain langchain-openai langgraph
orq-ai-sdk is the Orq.ai Python SDK. @orq-ai/node is the Orq.ai Node.js SDK.

​
Environment Variables

export ORQ_API_KEY="your-orq-api-key"
export OPENAI_API_KEY="your-openai-api-key"

​
Examples

Call setup() at the top of your entry point, before invoking any graphs or chains.
Agent with a single tool: captures agent/weather_agent, tool/get_weather, model/gpt-4o-mini 
from orq_ai_sdk.langchain import setup

setup()

from typing import Annotated, Literal, TypedDict
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage
from langchain.tools import tool
from langgraph.graph import StateGraph, START
from langgraph.graph.message import add_messages
from langgraph.prebuilt import ToolNode

class State(TypedDict):
    messages: Annotated[list, add_messages]

@tool
def get_weather(location: str) -> str:
    """Get weather for a location."""
    data = {"tokyo": "Sunny, 22°C", "paris": "Cloudy, 15°C", "new york": "Rainy, 18°C"}
    return data.get(location.lower(), f"No data for {location}")

tools = [get_weather]
llm   = ChatOpenAI(model="gpt-4o-mini", temperature=0).bind_tools(tools)

def weather_agent(state: State):
    return {"messages": [llm.invoke(state["messages"])]}

def route(state: State) -> Literal["tools", "__end__"]:
    last = state["messages"][-1]
    return "tools" if hasattr(last, "tool_calls") and last.tool_calls else "__end__"

graph = StateGraph(State)
graph.add_node("weather_agent", weather_agent)
graph.add_node("tools", ToolNode(tools))
graph.add_edge(START, "weather_agent")
graph.add_conditional_edges("weather_agent", route)
graph.add_edge("tools", "weather_agent")

app = graph.compile()
result = app.invoke({"messages": [HumanMessage(content="Weather in Tokyo?")]})
print(result["messages"][-1].content)
Agent with multiple tools: captures agent/assistant_agent, tool/calculator, tool/get_time, model/gpt-4o-mini 
from orq_ai_sdk.langchain import setup

setup()

from datetime import datetime
from typing import Annotated, Literal, TypedDict
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage
from langchain.tools import tool
from langgraph.graph import StateGraph, START
from langgraph.graph.message import add_messages
from langgraph.prebuilt import ToolNode

class State(TypedDict):
    messages: Annotated[list, add_messages]

@tool
def calculator(expression: str) -> str:
    """Evaluate a math expression."""
    try:
        return str(eval(expression))
    except Exception:
        return "Error"

@tool
def get_time() -> str:
    """Get the current time."""
    return datetime.now().strftime("%H:%M:%S")

tools = [calculator, get_time]
llm   = ChatOpenAI(model="gpt-4o-mini", temperature=0).bind_tools(tools)

def assistant_agent(state: State):
    return {"messages": [llm.invoke(state["messages"])]}

def route(state: State) -> Literal["tools", "__end__"]:
    last = state["messages"][-1]
    return "tools" if hasattr(last, "tool_calls") and last.tool_calls else "__end__"

graph = StateGraph(State)
graph.add_node("assistant_agent", assistant_agent)
graph.add_node("tools", ToolNode(tools))
graph.add_edge(START, "assistant_agent")
graph.add_conditional_edges("assistant_agent", route)
graph.add_edge("tools", "assistant_agent")

app = graph.compile()
result = app.invoke({"messages": [HumanMessage(content="What is 25 * 4? Also what time is it?")]})
print(result["messages"][-1].content)

​
Viewing Traces

Traces appear in the Orq.ai Studio under the Traces tab. Each run is captured as a tree reflecting your graph structure: top-level chain spans for each node, with LLM calls, tool executions, and retrievals nested underneath.
LangGraph trace in Orq.ai

​
Evaluations & Experiments

Once your agents are running, use Evaluatorq to score outputs across a dataset and Experiments to compare configurations side-by-side.

Run Evaluations with Evaluatorq

Run parallel evaluations across your agents and compare results.

Run Experiments via the API

Compare agent configurations and view results in the AI Studio.