LangGraph framework integration

AI Gateway

Route your LLM calls through the AI Gateway with a single base URL change. Zero vendor lock-in: always run on the best model at the lowest cost for your use case.

Observability

Instrument your code with OpenTelemetry to capture traces, logs, and metrics for every LLM call, agent step, and tool use.

AI Gateway

Overview

LangGraph is a framework for building stateful, multi-actor AI applications with LLMs. It extends LangChain with graph-based agent orchestration, cycles, and controllability. By connecting LangGraph to Orq.ai’s AI Gateway, you get production-ready agentic workflows with access to 300+ models.

Key Benefits

Orq.ai’s AI Gateway enhances your LangGraph applications with:

Complete Observability

Track every agent step, tool use, and graph transition with detailed traces

Built-in Reliability

Automatic fallbacks, retries, and load balancing for production resilience

Cost Optimization

Real-time cost tracking and spend management across all your AI operations

Multi-Provider Access

Access 300+ LLMs and 20+ providers through a single, unified integration

Prerequisites

Before integrating LangGraph with Orq.ai, ensure you have:

An Orq.ai account and API Key
Python 3.8 or higher

To setup your API key, see API keys & Endpoints.

Installation

pip install langgraph langchain-openai langchain-core

Configuration

Configure LangGraph to use Orq.ai’s AI Gateway by passing a ChatOpenAI instance with a custom base_url:

Python

from langchain_openai import ChatOpenAI
import os

llm = ChatOpenAI(
    model="gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router",
)

base_url: https://api.orq.ai/v3/router

Basic Agent Example

Here’s a complete example using create_agent with a tool:

Python

from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os

llm = ChatOpenAI(
    model="gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router",
)

@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is sunny and 72°F"

agent = create_agent(llm, tools=[get_weather])

result = agent.invoke({"messages": [("user", "What's the weather in San Francisco?")]})
print(result["messages"][-1].content)

Agent with Multiple Tools

Python

from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os

llm = ChatOpenAI(
    model="gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router",
)

@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is sunny and 72°F"

@tool
def add(a: int, b: int) -> int:
    """Add two integers."""
    return a + b

@tool
def multiply(a: int, b: int) -> int:
    """Multiply two integers."""
    return a * b

agent = create_agent(
    llm,
    tools=[get_weather, add, multiply],
    system_prompt="You are a helpful assistant with access to weather and math tools.",
)

result = agent.invoke({
    "messages": [("user", "What is 15 * 4? Also check the weather in Tokyo.")]
})
print(result["messages"][-1].content)

Streaming

Stream agent steps as they happen:

Python

from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os

llm = ChatOpenAI(
    model="gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router",
)

@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is sunny and 72°F"

agent = create_agent(llm, tools=[get_weather])

for chunk in agent.stream(
    {"messages": [("user", "What's the weather in Paris?")]},
    stream_mode="updates",
):
    print(chunk)

Model Selection

With Orq.ai, you can use any supported model from 20+ providers:

Python

from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os

@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is sunny and 72°F"

# Use Claude
claude_agent = create_agent(
    ChatOpenAI(
        model="claude-sonnet-4-5-20250929",
        api_key=os.getenv("ORQ_API_KEY"),
        base_url="https://api.orq.ai/v3/router",
    ),
    tools=[get_weather],
)

# Use Gemini
gemini_agent = create_agent(
    ChatOpenAI(
        model="gemini-2.5-flash",
        api_key=os.getenv("ORQ_API_KEY"),
        base_url="https://api.orq.ai/v3/router",
    ),
    tools=[get_weather],
)

result = claude_agent.invoke({"messages": [("user", "What's the weather in London?")]})
print(result["messages"][-1].content)

Observability

orq_ai_sdk.langchain provides a global setup() function that automatically instruments all LangGraph components. Call it once at the top of your application and every LLM call, graph node, tool execution, and retrieval is traced automatically, no callback wiring needed.

Zero configuration

One setup() call and tracing is live, no callbacks, no OpenTelemetry exporters, no extra wiring.

Full graph visibility

Traces preserve the parent-child structure of your graph so you see exactly which node triggered each LLM call or tool use.

Token usage and costs

Input and output token counts are captured on every LLM call and synced to Orq.ai for cost tracking.

Asset Capture

Agents, tools, and models are automatically registered in Control Tower from your traces.

Installation

pip install orq-ai-sdk langchain langchain-openai langgraph

npm install @orq-ai/node @langchain/core @langchain/langgraph @langchain/openai zod

orq-ai-sdk is the Orq.ai Python SDK. @orq-ai/node is the Orq.ai Node.js SDK.

Environment Variables

export ORQ_API_KEY="your-orq-api-key"
export OPENAI_API_KEY="your-openai-api-key"

Examples

Call setup() at the top of your entry point, before invoking any graphs or chains.

Agent with a single tool: captures agent/weather_agent, tool/get_weather, model/gpt-4o-mini

from orq_ai_sdk.langchain import setup

setup()

from typing import Annotated, Literal, TypedDict
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage
from langchain.tools import tool
from langgraph.graph import StateGraph, START
from langgraph.graph.message import add_messages
from langgraph.prebuilt import ToolNode

class State(TypedDict):
    messages: Annotated[list, add_messages]

@tool
def get_weather(location: str) -> str:
    """Get weather for a location."""
    data = {"tokyo": "Sunny, 22°C", "paris": "Cloudy, 15°C", "new york": "Rainy, 18°C"}
    return data.get(location.lower(), f"No data for {location}")

tools = [get_weather]
llm   = ChatOpenAI(model="gpt-4o-mini", temperature=0).bind_tools(tools)

def weather_agent(state: State):
    return {"messages": [llm.invoke(state["messages"])]}

def route(state: State) -> Literal["tools", "__end__"]:
    last = state["messages"][-1]
    return "tools" if hasattr(last, "tool_calls") and last.tool_calls else "__end__"

graph = StateGraph(State)
graph.add_node("weather_agent", weather_agent)
graph.add_node("tools", ToolNode(tools))
graph.add_edge(START, "weather_agent")
graph.add_conditional_edges("weather_agent", route)
graph.add_edge("tools", "weather_agent")

app = graph.compile()
result = app.invoke({"messages": [HumanMessage(content="Weather in Tokyo?")]})
print(result["messages"][-1].content)

import { HumanMessage } from '@langchain/core/messages';
import { tool } from '@langchain/core/tools';
import { StateGraph, START, MessagesAnnotation } from '@langchain/langgraph';
import { ToolNode } from '@langchain/langgraph/prebuilt';
import { ChatOpenAI } from '@langchain/openai';
import { setup } from '@orq-ai/node/langchain';
import { z } from 'zod';

setup();

const getWeather = tool(
  async ({ location }: { location: string }) => {
    const data: Record<string, string> = {
      tokyo: 'Sunny, 22°C',
      paris: 'Cloudy, 15°C',
      'new york': 'Rainy, 18°C',
    };
    return data[location.toLowerCase()] ?? `No data for ${location}`;
  },
  {
    name: 'get_weather',
    description: 'Get weather for a location.',
    schema: z.object({ location: z.string() }),
  },
);

const tools = [getWeather];
const llm = new ChatOpenAI({ model: 'gpt-4o-mini', temperature: 0 }).bindTools(tools);

function weatherAgent(state: typeof MessagesAnnotation.State) {
  return llm.invoke(state.messages).then((response) => ({ messages: [response] }));
}

function route(state: typeof MessagesAnnotation.State) {
  const last = state.messages[state.messages.length - 1];
  const toolCalls = last.tool_calls;
  return toolCalls?.length ? 'tools' : '__end__';
}

const graph = new StateGraph(MessagesAnnotation)
  .addNode('weather_agent', weatherAgent)
  .addNode('tools', new ToolNode(tools))
  .addEdge(START, 'weather_agent')
  .addConditionalEdges('weather_agent', route)
  .addEdge('tools', 'weather_agent')
  .compile();

const result = await graph.invoke({
  messages: [new HumanMessage('Weather in Tokyo?')],
});
console.log(result.messages[result.messages.length - 1].content);

Agent with multiple tools: captures agent/assistant_agent, tool/calculator, tool/get_time, model/gpt-4o-mini

from orq_ai_sdk.langchain import setup

setup()

from datetime import datetime
from typing import Annotated, Literal, TypedDict
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage
from langchain.tools import tool
from langgraph.graph import StateGraph, START
from langgraph.graph.message import add_messages
from langgraph.prebuilt import ToolNode

class State(TypedDict):
    messages: Annotated[list, add_messages]

@tool
def calculator(expression: str) -> str:
    """Evaluate a math expression."""
    try:
        return str(eval(expression))
    except Exception:
        return "Error"

@tool
def get_time() -> str:
    """Get the current time."""
    return datetime.now().strftime("%H:%M:%S")

tools = [calculator, get_time]
llm   = ChatOpenAI(model="gpt-4o-mini", temperature=0).bind_tools(tools)

def assistant_agent(state: State):
    return {"messages": [llm.invoke(state["messages"])]}

def route(state: State) -> Literal["tools", "__end__"]:
    last = state["messages"][-1]
    return "tools" if hasattr(last, "tool_calls") and last.tool_calls else "__end__"

graph = StateGraph(State)
graph.add_node("assistant_agent", assistant_agent)
graph.add_node("tools", ToolNode(tools))
graph.add_edge(START, "assistant_agent")
graph.add_conditional_edges("assistant_agent", route)
graph.add_edge("tools", "assistant_agent")

app = graph.compile()
result = app.invoke({"messages": [HumanMessage(content="What is 25 * 4? Also what time is it?")]})
print(result["messages"][-1].content)

import { HumanMessage } from '@langchain/core/messages';
import { tool } from '@langchain/core/tools';
import { StateGraph, START, MessagesAnnotation } from '@langchain/langgraph';
import { ToolNode } from '@langchain/langgraph/prebuilt';
import { ChatOpenAI } from '@langchain/openai';
import { setup } from '@orq-ai/node/langchain';
import { z } from 'zod';

setup();

const calculator = tool(
  async ({ expression }: { expression: string }) => {
    try {
      return String(Function(`"use strict"; return (${expression})`)()); // demo only — use a safe parser in production
    } catch {
      return 'Error';
    }
  },
  {
    name: 'calculator',
    description: 'Evaluate a math expression.',
    schema: z.object({ expression: z.string() }),
  },
);

const getTime = tool(
  async () => new Date().toLocaleTimeString('en-GB', { hour12: false }),
  {
    name: 'get_time',
    description: 'Get the current time.',
    schema: z.object({}),
  },
);

const tools = [calculator, getTime];
const llm = new ChatOpenAI({ model: 'gpt-4o-mini', temperature: 0 }).bindTools(tools);

function assistantAgent(state: typeof MessagesAnnotation.State) {
  return llm.invoke(state.messages).then((response) => ({ messages: [response] }));
}

function route(state: typeof MessagesAnnotation.State) {
  const last = state.messages[state.messages.length - 1];
  const toolCalls = last.tool_calls;
  return toolCalls?.length ? 'tools' : '__end__';
}

const graph = new StateGraph(MessagesAnnotation)
  .addNode('assistant_agent', assistantAgent)
  .addNode('tools', new ToolNode(tools))
  .addEdge(START, 'assistant_agent')
  .addConditionalEdges('assistant_agent', route)
  .addEdge('tools', 'assistant_agent')
  .compile();

const result = await graph.invoke({
  messages: [new HumanMessage('What is 25 * 4? Also what time is it?')],
});
console.log(result.messages[result.messages.length - 1].content);

Viewing Traces

Traces appear in the Orq.ai Studio under the Traces tab. Each run is captured as a tree reflecting your graph structure: top-level chain spans for each node, with LLM calls, tool executions, and retrievals nested underneath. The graph panel shows your node topology; clicking a span reveals token counts, cost, input, and output.

Orq.ai Studio Traces view showing a LangGraph run: graph panel with classifier, route, and math_solver nodes; span hierarchy on the right with token count and estimated cost. — LangGraph branching classifier trace in Orq.ai Studio.

See the code that produced this trace

A classifier node labels each request as weather, math, or chat and conditional edges dispatch it to a specialized agent. Uses OrqLangchainCallback: an alternative to setup() for explicit per-graph instrumentation.

OrqLangchainCallback attaches tracing to a specific compiled graph via .with_config. Use it when you want to instrument only certain graphs, or when you prefer not to use global auto-instrumentation.

import os
from typing import Annotated, Literal, TypedDict
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages
from orq_ai_sdk.langchain import OrqLangchainCallback

@tool
def get_weather(location: str) -> str:
    """Return current weather for a location."""
    return f"Sunny in {location}, 22C"

classifier_llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
weather_llm    = ChatOpenAI(model="gpt-4o-mini", temperature=0).bind_tools([get_weather])
math_llm       = ChatOpenAI(model="gpt-4o-mini", temperature=0)
chat_llm       = ChatOpenAI(model="gpt-4o-mini", temperature=0.7)

class State(TypedDict):
    messages: Annotated[list, add_messages]
    route: str

def classifier(state: State):
    user_msg = state["messages"][-1].content if state["messages"] else ""
    response = classifier_llm.invoke([{
        "role": "user",
        "content": (
            "Classify the following user question into exactly ONE of: weather, math, chat. "
            f"Respond with only the single word.\n\nUser: {user_msg}"
        ),
    }])
    label = (response.content or "").strip().lower().split()[0] if response.content else "chat"
    return {"route": label if label in {"weather", "math", "chat"} else "chat"}

def weather_agent(state: State):
    return {"messages": [weather_llm.invoke(state["messages"])]}

def weather_tools(state: State):
    last = state["messages"][-1]
    return {"messages": [
        {"role": "tool", "content": get_weather.invoke(tc["args"]), "tool_call_id": tc["id"]}
        for tc in (getattr(last, "tool_calls", []) or [])
    ]}

def weather_finalize(state: State):
    return {"messages": [weather_llm.invoke(state["messages"])]}

def math_solver(state: State):
    return {"messages": [math_llm.invoke([
        {"role": "system", "content": "You are a careful math solver. Show one short step, then the final number."},
        *state["messages"],
    ])]}

def chitchat(state: State):
    return {"messages": [chat_llm.invoke([
        {"role": "system", "content": "You are friendly and brief. Reply in one or two sentences."},
        *state["messages"],
    ])]}

def route(state: State) -> Literal["weather_agent", "math_solver", "chitchat"]:
    return {"weather": "weather_agent", "math": "math_solver", "chat": "chitchat"}[
        state.get("route", "chat")
    ]

def weather_needs_tool(state: State) -> Literal["weather_tools", "end"]:
    last = state["messages"][-1]
    return "weather_tools" if (hasattr(last, "tool_calls") and last.tool_calls) else "end"

g = StateGraph(State)
g.add_node("classifier", classifier)
g.add_node("weather_agent", weather_agent)
g.add_node("weather_tools", weather_tools)
g.add_node("weather_finalize", weather_finalize)
g.add_node("math_solver", math_solver)
g.add_node("chitchat", chitchat)
g.add_edge(START, "classifier")
g.add_conditional_edges("classifier", route, {
    "weather_agent": "weather_agent",
    "math_solver":   "math_solver",
    "chitchat":      "chitchat",
})
g.add_conditional_edges("weather_agent", weather_needs_tool, {
    "weather_tools": "weather_tools",
    "end":           END,
})
g.add_edge("weather_tools", "weather_finalize")
g.add_edge("weather_finalize", END)
g.add_edge("math_solver", END)
g.add_edge("chitchat", END)

orq_handler = OrqLangchainCallback(api_key=os.environ["ORQ_API_KEY"])
graph = g.compile().with_config({"callbacks": [orq_handler]})

for question in [
    "What's the weather in Tokyo?",
    "What is 15 * 4 + 7?",
    "Tell me a one-line joke.",
]:
    result = graph.invoke({"messages": [{"role": "user", "content": question}], "route": ""})
    print(f"route: {result.get('route')} | answer: {result['messages'][-1].content[:120]!r}")

Evaluations & Experiments

Once your agents are running, use Evaluatorq to score outputs across a dataset and Experiments to compare configurations side-by-side.

Run Evaluations with Evaluatorq

Run parallel evaluations across your agents and compare results.

Run Experiments via the API

Compare agent configurations and view results in the AI Studio.

AI Providers

Coding Assistants

Frameworks

Automations

AI Gateway

Observability

AI Gateway

Overview

Key Benefits

Complete Observability

Built-in Reliability

Cost Optimization

Multi-Provider Access

Prerequisites

Installation

Configuration

Basic Agent Example

Agent with Multiple Tools

Streaming

Model Selection

Observability

Zero configuration

Full graph visibility

Token usage and costs

Asset Capture

Installation

Environment Variables

Examples

Viewing Traces

Evaluations & Experiments

Run Evaluations with Evaluatorq

Run Experiments via the API

AI Gateway

Observability

​AI Gateway

​Overview

​Key Benefits

Complete Observability

Built-in Reliability

Cost Optimization

Multi-Provider Access

​Prerequisites

​Installation

​Configuration

​Basic Agent Example

​Agent with Multiple Tools

​Streaming

​Model Selection

​Observability

Zero configuration

Full graph visibility

Token usage and costs

Asset Capture

​Installation

​Environment Variables

​Examples

​Viewing Traces

​Evaluations & Experiments

Run Evaluations with Evaluatorq

Run Experiments via the API

AI Gateway

Overview

Key Benefits

Prerequisites

Installation

Configuration

Basic Agent Example

Agent with Multiple Tools

Streaming

Model Selection

Observability

Installation

Environment Variables

Examples

Viewing Traces

Evaluations & Experiments