> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orq.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# LangChain framework integration

> Connect Langchain to the AI Router for enhanced LLM orchestration. Use Orq.ai as a drop-in provider for chains, agents, and RAG applications.

<CardGroup cols={2}>
  <Card title="AI Router" icon="arrow-right-arrow-left" href="#ai-router">
    Route your LLM calls through the AI Router with a single base URL change. Zero vendor lock-in: always run on the best model at the lowest cost for your use case.
  </Card>

  <Card title="Observability" icon="chart-line" href="#observability">
    Attach the native Orq callback handler to your LangGraph to capture traces for every LLM call, graph node, tool use, and retrieval.
  </Card>
</CardGroup>

## AI Router

### Overview

LangChain is a framework for building LLM-powered applications through composable chains, agents, and integrations with external data sources. By connecting LangChain to Orq.ai's AI Router, you access 300+ models through a single base URL change.

### Key Benefits

Orq.ai's AI Router enhances your LangChain applications with:

<CardGroup cols={2}>
  <Card title="Complete Observability" icon="chart-line">
    Track every chain step, tool use, and LLM call with detailed traces
  </Card>

  <Card title="Built-in Reliability" icon="shield-check">
    Automatic fallbacks, retries, and load balancing for production resilience
  </Card>

  <Card title="Cost Optimization" icon="chart-pie">
    Real-time cost tracking and spend management across all your AI operations
  </Card>

  <Card title="Multi-Provider Access" icon="cubes">
    Access 300+ LLMs and 20+ providers through a single, unified integration
  </Card>
</CardGroup>

### Prerequisites

Before integrating LangChain with Orq.ai, ensure you have:

* An Orq.ai account and [API Key](/docs/administer/api-keys)
* Python 3.8 or higher

<Info>
  To set up your API key, see [API keys & Endpoints](/docs/administer/api-keys).
</Info>

### Installation

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
pip install langchain langchain-openai
```

### Configuration

Configure LangChain to use Orq.ai's AI Router via `ChatOpenAI` with a custom `base_url`:

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from langchain_openai import ChatOpenAI
import os

llm = ChatOpenAI(
    model="gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router",
)
```

> **base\_url**: `https://api.orq.ai/v3/router`

### Basic Example

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from langchain_openai import ChatOpenAI
import os

llm = ChatOpenAI(
    model="gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router",
)

result = llm.invoke("Explain quantum computing in simple terms.")
print(result.content)
```

### Chains

Build composable chains using LangChain's pipe operator:

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
import os

llm = ChatOpenAI(
    model="gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router",
)

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant."),
    ("user", "{input}"),
])

chain = prompt | llm
result = chain.invoke({"input": "Tell me a joke about programming."})
print(result.content)
```

### Streaming

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from langchain_openai import ChatOpenAI
import os

llm = ChatOpenAI(
    model="gpt-4o",
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router",
)

for chunk in llm.stream("Write a short poem about the ocean."):
    print(chunk.content, end="", flush=True)
print()
```

### Model Selection

With Orq.ai, you can use any supported model from 20+ providers:

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from langchain_openai import ChatOpenAI
import os

# Use Claude
claude = ChatOpenAI(
    model="anthropic/claude-sonnet-4-5",
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router",
)

# Use Gemini
gemini = ChatOpenAI(
    model="gemini-2.5-flash",
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router",
)

# Use Groq
groq = ChatOpenAI(
    model="groq/llama-3.3-70b-versatile",
    api_key=os.getenv("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router",
)
```

## Observability

`orq_ai_sdk.langchain` provides a global `setup()` function that automatically instruments all LangChain and LangGraph components. Call it once at the top of your application and every LLM call, graph node, tool execution, and retrieval is traced automatically, no callback wiring needed.

<CardGroup cols={2}>
  <Card title="Zero configuration" icon="wand-magic-sparkles">
    One `setup()` call and tracing is live, no callbacks, no OpenTelemetry exporters, no extra wiring.
  </Card>

  <Card title="Full graph visibility" icon="diagram-project">
    Traces preserve the parent-child structure of your LangGraph so you see exactly which node triggered each LLM call or tool use.
  </Card>

  <Card title="Token usage and costs" icon="chart-pie">
    Input and output token counts are captured on every LLM call and synced to **Orq.ai** for cost tracking.
  </Card>

  <Card title="Retrieval tracking" icon="database">
    Retrieval events include the query and all returned documents, making RAG pipelines fully inspectable.
  </Card>
</CardGroup>

### Installation

<CodeGroup>
  ```bash Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  pip install orq-ai-sdk langchain-core langchain-openai langgraph
  ```

  ```bash Node.js theme={"theme":{"light":"github-light","dark":"github-dark"}}
  npm install @orq-ai/node @langchain/core @langchain/langgraph @langchain/openai
  ```
</CodeGroup>

<Info>
  `orq-ai-sdk` is the [Orq.ai Python SDK](https://github.com/orq-ai/orq-python/tree/main). `@orq-ai/node` is the [Orq.ai Node.js SDK](https://github.com/orq-ai/orq-node).
</Info>

### Environment Variables

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
export ORQ_API_KEY="your-orq-api-key"
export OPENAI_API_KEY="your-openai-api-key" # required because the examples call OpenAI models directly
```

Or set them in code:

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  import os
  os.environ["ORQ_API_KEY"] = "your-orq-api-key"
  os.environ["OPENAI_API_KEY"] = "your-openai-api-key"
  ```

  ```ts Node.js theme={"theme":{"light":"github-light","dark":"github-dark"}}
  process.env.ORQ_API_KEY = 'your-orq-api-key';
  process.env.OPENAI_API_KEY = 'your-openai-api-key';
  ```
</CodeGroup>

### Basic Example

<Note>
  Call `setup()` at the top of your entry point, before invoking any graphs or chains. It globally instruments LangChain so that all subsequent executions are traced automatically.
</Note>

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  from orq_ai_sdk.langchain import setup

  setup()

  from typing import Annotated
  from typing_extensions import TypedDict
  from langchain_openai import ChatOpenAI
  from langgraph.graph import StateGraph, START, END
  from langgraph.graph.message import add_messages

  class State(TypedDict):
      messages: Annotated[list, add_messages]

  graph_builder = StateGraph(State)
  llm = ChatOpenAI(model="gpt-4o", temperature=0.2)

  def chatbot(state: State):
      return {"messages": [llm.invoke(state["messages"])]}

  graph_builder.add_node("chatbot", chatbot)
  graph_builder.add_edge(START, "chatbot")
  graph_builder.add_edge("chatbot", END)

  graph = graph_builder.compile()

  result = graph.invoke({"messages": [{"role": "user", "content": "Hello!"}]})
  print(result["messages"][-1].content)
  ```

  ```ts Node.js theme={"theme":{"light":"github-light","dark":"github-dark"}}
  import { StateGraph, START, END, MessagesAnnotation } from '@langchain/langgraph';
  import { ChatOpenAI } from '@langchain/openai';
  import { setup } from '@orq-ai/node/langchain';

  setup();

  const llm = new ChatOpenAI({ model: 'gpt-4o', temperature: 0.2 });

  function chatbot(state: typeof MessagesAnnotation.State) {
    return llm.invoke(state.messages).then((response) => ({ messages: [response] }));
  }

  const graph = new StateGraph(MessagesAnnotation)
    .addNode('chatbot', chatbot)
    .addEdge(START, 'chatbot')
    .addEdge('chatbot', END)
    .compile();

  const result = await graph.invoke({
    messages: [{ role: 'user', content: 'Hello!' }],
  });
  console.log(result.messages[result.messages.length - 1].content);
  ```
</CodeGroup>

### Async Example

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  from orq_ai_sdk.langchain import setup

  setup()

  import asyncio
  from typing import Annotated
  from typing_extensions import TypedDict
  from langchain_openai import ChatOpenAI
  from langgraph.graph import StateGraph, START, END
  from langgraph.graph.message import add_messages

  class State(TypedDict):
      messages: Annotated[list, add_messages]

  graph_builder = StateGraph(State)
  llm = ChatOpenAI(model="gpt-4o", temperature=0.2)

  def chatbot(state: State):
      return {"messages": [llm.invoke(state["messages"])]}

  graph_builder.add_node("chatbot", chatbot)
  graph_builder.add_edge(START, "chatbot")
  graph_builder.add_edge("chatbot", END)

  graph = graph_builder.compile()

  async def main():
      result = await graph.ainvoke({"messages": [{"role": "user", "content": "Hello!"}]})
      print(result["messages"][-1].content)

  asyncio.run(main())
  ```

  ```ts Node.js theme={"theme":{"light":"github-light","dark":"github-dark"}}
  import { StateGraph, START, END, MessagesAnnotation } from '@langchain/langgraph';
  import { ChatOpenAI } from '@langchain/openai';
  import { setup } from '@orq-ai/node/langchain';

  setup();

  const llm = new ChatOpenAI({ model: 'gpt-4o', temperature: 0.2 });

  function chatbot(state: typeof MessagesAnnotation.State) {
    return llm.invoke(state.messages).then((response) => ({ messages: [response] }));
  }

  const graph = new StateGraph(MessagesAnnotation)
    .addNode('chatbot', chatbot)
    .addEdge(START, 'chatbot')
    .addEdge('chatbot', END)
    .compile();

  const result = await graph.invoke({
    messages: [{ role: 'user', content: 'Hello!' }],
  });
  console.log(result.messages[result.messages.length - 1].content);
  ```
</CodeGroup>

### Viewing Traces

Traces appear in the **Orq.ai** Studio under the [Traces](/docs/observability/traces) tab. Each run is captured as a tree reflecting your graph structure: top-level chain spans for each node, with LLM calls, tool executions, and retrievals nested underneath.

<Frame caption="LangChain trace in the AI Studio showing the full graph execution hierarchy">
  <img src="https://mintcdn.com/orqai/iyV9S3k3CoZi07Cd/images/trace-langchain.png?fit=max&auto=format&n=iyV9S3k3CoZi07Cd&q=85&s=c411644a4b7f69acc30374a709ed60f7" alt="LangChain trace in the AI Studio" width="1658" height="787" data-path="images/trace-langchain.png" />
</Frame>

#### What Gets Traced

| Event                | Details captured                            |
| -------------------- | ------------------------------------------- |
| Graph nodes (chains) | Node name, inputs, outputs, duration        |
| LLM calls            | Messages, model, token usage, finish reason |
| Tool executions      | Tool name, input, output, duration          |
| Retrievals           | Query, returned documents                   |
| Agent actions        | Action taken, finish output                 |

## Evaluations & Experiments

Once your agents are running, use **Evaluatorq** to score outputs across a dataset and **Experiments** to compare configurations side-by-side.

<CardGroup cols={2}>
  <Card title="Run Evaluations with Evaluatorq" icon="flask" href="/docs/evaluators/build#evaluatorq">
    Run parallel evaluations across your agents and compare results.
  </Card>

  <Card title="Run Experiments via the API" icon="flask-vial" href="/docs/experiments/api">
    Compare agent configurations and view results in the AI Studio.
  </Card>
</CardGroup>