> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orq.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# LangGraph framework integration
> Connect LangGraph to Orq.ai's AI Router for complete observability, built-in reliability, and access to 300+ LLMs across 20+ providers.
Route your LLM calls through the AI Router with a single base URL change. Zero vendor lock-in: always run on the best model at the lowest cost for your use case.
Instrument your code with OpenTelemetry to capture traces, logs, and metrics for every LLM call, agent step, and tool use.
## AI Router
### Overview
LangGraph is a framework for building stateful, multi-actor AI applications with LLMs. It extends LangChain with graph-based agent orchestration, cycles, and controllability. By connecting LangGraph to Orq.ai's AI Router, you get production-ready agentic workflows with access to 300+ models.
### Key Benefits
Orq.ai's AI Router enhances your LangGraph applications with:
Track every agent step, tool use, and graph transition with detailed traces
Automatic fallbacks, retries, and load balancing for production resilience
Real-time cost tracking and spend management across all your AI operations
Access 300+ LLMs and 20+ providers through a single, unified integration
### Prerequisites
Before integrating LangGraph with Orq.ai, ensure you have:
* An Orq.ai account and [API Key](/docs/administer/api-keys)
* Python 3.8 or higher
To setup your API key, see [API keys & Endpoints](/docs/administer/api-keys).
### Installation
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
pip install langgraph langchain-openai langchain-core
```
### Configuration
Configure LangGraph to use Orq.ai's AI Router by passing a `ChatOpenAI` instance with a custom `base_url`:
```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from langchain_openai import ChatOpenAI
import os
llm = ChatOpenAI(
model="gpt-4o",
api_key=os.getenv("ORQ_API_KEY"),
base_url="https://api.orq.ai/v3/router",
)
```
> **base\_url**: `https://api.orq.ai/v3/router`
### Basic Agent Example
Here's a complete example using `create_agent` with a tool:
```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os
llm = ChatOpenAI(
model="gpt-4o",
api_key=os.getenv("ORQ_API_KEY"),
base_url="https://api.orq.ai/v3/router",
)
@tool
def get_weather(location: str) -> str:
"""Get the current weather for a location."""
return f"The weather in {location} is sunny and 72°F"
agent = create_agent(llm, tools=[get_weather])
result = agent.invoke({"messages": [("user", "What's the weather in San Francisco?")]})
print(result["messages"][-1].content)
```
### Agent with Multiple Tools
```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os
llm = ChatOpenAI(
model="gpt-4o",
api_key=os.getenv("ORQ_API_KEY"),
base_url="https://api.orq.ai/v3/router",
)
@tool
def get_weather(location: str) -> str:
"""Get the current weather for a location."""
return f"The weather in {location} is sunny and 72°F"
@tool
def add(a: int, b: int) -> int:
"""Add two integers."""
return a + b
@tool
def multiply(a: int, b: int) -> int:
"""Multiply two integers."""
return a * b
agent = create_agent(
llm,
tools=[get_weather, add, multiply],
system_prompt="You are a helpful assistant with access to weather and math tools.",
)
result = agent.invoke({
"messages": [("user", "What is 15 * 4? Also check the weather in Tokyo.")]
})
print(result["messages"][-1].content)
```
### Streaming
Stream agent steps as they happen:
```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os
llm = ChatOpenAI(
model="gpt-4o",
api_key=os.getenv("ORQ_API_KEY"),
base_url="https://api.orq.ai/v3/router",
)
@tool
def get_weather(location: str) -> str:
"""Get the current weather for a location."""
return f"The weather in {location} is sunny and 72°F"
agent = create_agent(llm, tools=[get_weather])
for chunk in agent.stream(
{"messages": [("user", "What's the weather in Paris?")]},
stream_mode="updates",
):
print(chunk)
```
### Model Selection
With Orq.ai, you can use any supported model from 20+ providers:
```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os
@tool
def get_weather(location: str) -> str:
"""Get the current weather for a location."""
return f"The weather in {location} is sunny and 72°F"
# Use Claude
claude_agent = create_agent(
ChatOpenAI(
model="claude-sonnet-4-5-20250929",
api_key=os.getenv("ORQ_API_KEY"),
base_url="https://api.orq.ai/v3/router",
),
tools=[get_weather],
)
# Use Gemini
gemini_agent = create_agent(
ChatOpenAI(
model="gemini-2.5-flash",
api_key=os.getenv("ORQ_API_KEY"),
base_url="https://api.orq.ai/v3/router",
),
tools=[get_weather],
)
result = claude_agent.invoke({"messages": [("user", "What's the weather in London?")]})
print(result["messages"][-1].content)
```
## Observability
`orq_ai_sdk.langchain` provides a global `setup()` function that automatically instruments all LangGraph components. Call it once at the top of your application and every LLM call, graph node, tool execution, and retrieval is traced automatically, no callback wiring needed.
One `setup()` call and tracing is live, no callbacks, no OpenTelemetry exporters, no extra wiring.
Traces preserve the parent-child structure of your graph so you see exactly which node triggered each LLM call or tool use.
Input and output token counts are captured on every LLM call and synced to **Orq.ai** for cost tracking.
Agents, tools, and models are automatically registered in [Control Tower](/docs/control-tower/assets) from your traces.
### Installation
```bash Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
pip install orq-ai-sdk langchain langchain-openai langgraph
```
```bash Node.js theme={"theme":{"light":"github-light","dark":"github-dark"}}
npm install @orq-ai/node @langchain/core @langchain/langgraph @langchain/openai zod
```
`orq-ai-sdk` is the [Orq.ai Python SDK](https://github.com/orq-ai/orq-python/tree/main). `@orq-ai/node` is the [Orq.ai Node.js SDK](https://github.com/orq-ai/orq-node).
### Environment Variables
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
export ORQ_API_KEY="your-orq-api-key"
export OPENAI_API_KEY="your-openai-api-key"
```
### Examples
Call `setup()` at the top of your entry point, before invoking any graphs or chains.
**Agent with a single tool**: captures `agent/weather_agent`, `tool/get_weather`, `model/gpt-4o-mini`
```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from orq_ai_sdk.langchain import setup
setup()
from typing import Annotated, Literal, TypedDict
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage
from langchain.tools import tool
from langgraph.graph import StateGraph, START
from langgraph.graph.message import add_messages
from langgraph.prebuilt import ToolNode
class State(TypedDict):
messages: Annotated[list, add_messages]
@tool
def get_weather(location: str) -> str:
"""Get weather for a location."""
data = {"tokyo": "Sunny, 22°C", "paris": "Cloudy, 15°C", "new york": "Rainy, 18°C"}
return data.get(location.lower(), f"No data for {location}")
tools = [get_weather]
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0).bind_tools(tools)
def weather_agent(state: State):
return {"messages": [llm.invoke(state["messages"])]}
def route(state: State) -> Literal["tools", "__end__"]:
last = state["messages"][-1]
return "tools" if hasattr(last, "tool_calls") and last.tool_calls else "__end__"
graph = StateGraph(State)
graph.add_node("weather_agent", weather_agent)
graph.add_node("tools", ToolNode(tools))
graph.add_edge(START, "weather_agent")
graph.add_conditional_edges("weather_agent", route)
graph.add_edge("tools", "weather_agent")
app = graph.compile()
result = app.invoke({"messages": [HumanMessage(content="Weather in Tokyo?")]})
print(result["messages"][-1].content)
```
```ts Node.js theme={"theme":{"light":"github-light","dark":"github-dark"}}
import { HumanMessage } from '@langchain/core/messages';
import { tool } from '@langchain/core/tools';
import { StateGraph, START, MessagesAnnotation } from '@langchain/langgraph';
import { ToolNode } from '@langchain/langgraph/prebuilt';
import { ChatOpenAI } from '@langchain/openai';
import { setup } from '@orq-ai/node/langchain';
import { z } from 'zod';
setup();
const getWeather = tool(
async ({ location }: { location: string }) => {
const data: Record = {
tokyo: 'Sunny, 22°C',
paris: 'Cloudy, 15°C',
'new york': 'Rainy, 18°C',
};
return data[location.toLowerCase()] ?? `No data for ${location}`;
},
{
name: 'get_weather',
description: 'Get weather for a location.',
schema: z.object({ location: z.string() }),
},
);
const tools = [getWeather];
const llm = new ChatOpenAI({ model: 'gpt-4o-mini', temperature: 0 }).bindTools(tools);
function weatherAgent(state: typeof MessagesAnnotation.State) {
return llm.invoke(state.messages).then((response) => ({ messages: [response] }));
}
function route(state: typeof MessagesAnnotation.State) {
const last = state.messages[state.messages.length - 1];
const toolCalls = last.tool_calls;
return toolCalls?.length ? 'tools' : '__end__';
}
const graph = new StateGraph(MessagesAnnotation)
.addNode('weather_agent', weatherAgent)
.addNode('tools', new ToolNode(tools))
.addEdge(START, 'weather_agent')
.addConditionalEdges('weather_agent', route)
.addEdge('tools', 'weather_agent')
.compile();
const result = await graph.invoke({
messages: [new HumanMessage('Weather in Tokyo?')],
});
console.log(result.messages[result.messages.length - 1].content);
```
**Agent with multiple tools**: captures `agent/assistant_agent`, `tool/calculator`, `tool/get_time`, `model/gpt-4o-mini`
```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from orq_ai_sdk.langchain import setup
setup()
from datetime import datetime
from typing import Annotated, Literal, TypedDict
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage
from langchain.tools import tool
from langgraph.graph import StateGraph, START
from langgraph.graph.message import add_messages
from langgraph.prebuilt import ToolNode
class State(TypedDict):
messages: Annotated[list, add_messages]
@tool
def calculator(expression: str) -> str:
"""Evaluate a math expression."""
try:
return str(eval(expression))
except Exception:
return "Error"
@tool
def get_time() -> str:
"""Get the current time."""
return datetime.now().strftime("%H:%M:%S")
tools = [calculator, get_time]
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0).bind_tools(tools)
def assistant_agent(state: State):
return {"messages": [llm.invoke(state["messages"])]}
def route(state: State) -> Literal["tools", "__end__"]:
last = state["messages"][-1]
return "tools" if hasattr(last, "tool_calls") and last.tool_calls else "__end__"
graph = StateGraph(State)
graph.add_node("assistant_agent", assistant_agent)
graph.add_node("tools", ToolNode(tools))
graph.add_edge(START, "assistant_agent")
graph.add_conditional_edges("assistant_agent", route)
graph.add_edge("tools", "assistant_agent")
app = graph.compile()
result = app.invoke({"messages": [HumanMessage(content="What is 25 * 4? Also what time is it?")]})
print(result["messages"][-1].content)
```
```ts Node.js theme={"theme":{"light":"github-light","dark":"github-dark"}}
import { HumanMessage } from '@langchain/core/messages';
import { tool } from '@langchain/core/tools';
import { StateGraph, START, MessagesAnnotation } from '@langchain/langgraph';
import { ToolNode } from '@langchain/langgraph/prebuilt';
import { ChatOpenAI } from '@langchain/openai';
import { setup } from '@orq-ai/node/langchain';
import { z } from 'zod';
setup();
const calculator = tool(
async ({ expression }: { expression: string }) => {
try {
return String(Function(`"use strict"; return (${expression})`)()); // demo only — use a safe parser in production
} catch {
return 'Error';
}
},
{
name: 'calculator',
description: 'Evaluate a math expression.',
schema: z.object({ expression: z.string() }),
},
);
const getTime = tool(
async () => new Date().toLocaleTimeString('en-GB', { hour12: false }),
{
name: 'get_time',
description: 'Get the current time.',
schema: z.object({}),
},
);
const tools = [calculator, getTime];
const llm = new ChatOpenAI({ model: 'gpt-4o-mini', temperature: 0 }).bindTools(tools);
function assistantAgent(state: typeof MessagesAnnotation.State) {
return llm.invoke(state.messages).then((response) => ({ messages: [response] }));
}
function route(state: typeof MessagesAnnotation.State) {
const last = state.messages[state.messages.length - 1];
const toolCalls = last.tool_calls;
return toolCalls?.length ? 'tools' : '__end__';
}
const graph = new StateGraph(MessagesAnnotation)
.addNode('assistant_agent', assistantAgent)
.addNode('tools', new ToolNode(tools))
.addEdge(START, 'assistant_agent')
.addConditionalEdges('assistant_agent', route)
.addEdge('tools', 'assistant_agent')
.compile();
const result = await graph.invoke({
messages: [new HumanMessage('What is 25 * 4? Also what time is it?')],
});
console.log(result.messages[result.messages.length - 1].content);
```
### Viewing Traces
Traces appear in the **Orq.ai** Studio under the [Traces](/docs/observability/traces) tab. Each run is captured as a tree reflecting your graph structure: top-level chain spans for each node, with LLM calls, tool executions, and retrievals nested underneath. The graph panel shows your node topology; clicking a span reveals token counts, cost, input, and output.
A `classifier` node labels each request as `weather`, `math`, or `chat` and conditional edges dispatch it to a specialized agent. Uses `OrqLangchainCallback`: an alternative to `setup()` for explicit per-graph instrumentation.
`OrqLangchainCallback` attaches tracing to a specific compiled graph via `.with_config`. Use it when you want to instrument only certain graphs, or when you prefer not to use global auto-instrumentation.
```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import os
from typing import Annotated, Literal, TypedDict
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages
from orq_ai_sdk.langchain import OrqLangchainCallback
@tool
def get_weather(location: str) -> str:
"""Return current weather for a location."""
return f"Sunny in {location}, 22C"
classifier_llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
weather_llm = ChatOpenAI(model="gpt-4o-mini", temperature=0).bind_tools([get_weather])
math_llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
chat_llm = ChatOpenAI(model="gpt-4o-mini", temperature=0.7)
class State(TypedDict):
messages: Annotated[list, add_messages]
route: str
def classifier(state: State):
user_msg = state["messages"][-1].content if state["messages"] else ""
response = classifier_llm.invoke([{
"role": "user",
"content": (
"Classify the following user question into exactly ONE of: weather, math, chat. "
f"Respond with only the single word.\n\nUser: {user_msg}"
),
}])
label = (response.content or "").strip().lower().split()[0] if response.content else "chat"
return {"route": label if label in {"weather", "math", "chat"} else "chat"}
def weather_agent(state: State):
return {"messages": [weather_llm.invoke(state["messages"])]}
def weather_tools(state: State):
last = state["messages"][-1]
return {"messages": [
{"role": "tool", "content": get_weather.invoke(tc["args"]), "tool_call_id": tc["id"]}
for tc in (getattr(last, "tool_calls", []) or [])
]}
def weather_finalize(state: State):
return {"messages": [weather_llm.invoke(state["messages"])]}
def math_solver(state: State):
return {"messages": [math_llm.invoke([
{"role": "system", "content": "You are a careful math solver. Show one short step, then the final number."},
*state["messages"],
])]}
def chitchat(state: State):
return {"messages": [chat_llm.invoke([
{"role": "system", "content": "You are friendly and brief. Reply in one or two sentences."},
*state["messages"],
])]}
def route(state: State) -> Literal["weather_agent", "math_solver", "chitchat"]:
return {"weather": "weather_agent", "math": "math_solver", "chat": "chitchat"}[
state.get("route", "chat")
]
def weather_needs_tool(state: State) -> Literal["weather_tools", "end"]:
last = state["messages"][-1]
return "weather_tools" if (hasattr(last, "tool_calls") and last.tool_calls) else "end"
g = StateGraph(State)
g.add_node("classifier", classifier)
g.add_node("weather_agent", weather_agent)
g.add_node("weather_tools", weather_tools)
g.add_node("weather_finalize", weather_finalize)
g.add_node("math_solver", math_solver)
g.add_node("chitchat", chitchat)
g.add_edge(START, "classifier")
g.add_conditional_edges("classifier", route, {
"weather_agent": "weather_agent",
"math_solver": "math_solver",
"chitchat": "chitchat",
})
g.add_conditional_edges("weather_agent", weather_needs_tool, {
"weather_tools": "weather_tools",
"end": END,
})
g.add_edge("weather_tools", "weather_finalize")
g.add_edge("weather_finalize", END)
g.add_edge("math_solver", END)
g.add_edge("chitchat", END)
orq_handler = OrqLangchainCallback(api_key=os.environ["ORQ_API_KEY"])
graph = g.compile().with_config({"callbacks": [orq_handler]})
for question in [
"What's the weather in Tokyo?",
"What is 15 * 4 + 7?",
"Tell me a one-line joke.",
]:
result = graph.invoke({"messages": [{"role": "user", "content": question}], "route": ""})
print(f"route: {result.get('route')} | answer: {result['messages'][-1].content[:120]!r}")
```
## Evaluations & Experiments
Once your agents are running, use **Evaluatorq** to score outputs across a dataset and **Experiments** to compare configurations side-by-side.
Run parallel evaluations across your agents and compare results.
Compare agent configurations and view results in the AI Studio.