> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orq.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Claude Code
> Connect Anthropic Claude Code CLI to Orq.ai via MCP for workspace access, experiment management, and trace analysis from the terminal.
Access the **Orq.ai** workspace directly from Claude Code. Manage experiments, query traces, and configure agents using natural language.
AI Gateway Beta} icon="arrow-right-arrow-left" href="#ai-gateway">
Route Claude Code's model calls through the **AI Gateway**.
## MCP
Claude Code is Anthropic's official CLI that brings Claude's capabilities to the terminal and development workflow. With the **Orq MCP** integration, access all **Orq.ai** features directly through Claude Code's conversational interface.
### Prerequisites
* [Claude Code CLI](https://github.com/anthropics/claude-code) installed
* Active Orq.ai account
* [Orq.ai API key](/ai-studio/organization/api-keys)
### Installation
Add the Orq MCP server to Claude Code with a single command:
```bash wrap theme={"theme":{"light":"github-light","dark":"github-dark"}}
claude mcp add --transport http orq https://my.orq.ai/v2/mcp --header "Authorization: Bearer ${ORQ_API_KEY}"
```
Set the `ORQ_API_KEY` environment variable before running the command:
```bash wrap theme={"theme":{"light":"github-light","dark":"github-dark"}}
export ORQ_API_KEY="your-api-key-here"
```
### Verify Installation
Check that the Orq MCP is installed:
```bash wrap theme={"theme":{"light":"github-light","dark":"github-dark"}}
claude mcp list
```
You should see `orq` in the list of available MCP servers.
### Available Commands
Once integrated, you can ask Claude Code to perform these operations:
* `Create an agent with custom instructions and tools`
* `Get agent configuration for [agent-key]`
* `Update agent [agent-key] with new instructions or model`
* `Configure agent with evaluators and guardrails`
* `Invoke agent [agent-key] with input [message]`
* `Retrieve agent response [response-id]`
* `Create a deployment called [deployment-key]`
* `Get deployment configuration for [deployment-key]`
* `Create a skill called [skill-key]`
* `List all skills in my workspace`
* `Get skill [skill-key]`
* `Update skill [skill-key]`
* `Delete skill [skill-key]`
* `Get analytics overview for my workspace`
* `Show me workspace metrics for the last 7 days`
* `Query analytics filtered by deployment ID`
* `Create a dataset called "customer-queries"`
* `List all datapoints in dataset [dataset-key]`
* `Add datapoints to dataset [dataset-key]`
* `Update datapoint [datapoint-id]`
* `Delete specific datapoints in dataset [dataset-key]`
* `Delete dataset [dataset-key]`
* `Create an experiment from dataset [dataset-key]`
* `List all experiment runs`
* `Export experiment run [run-id] as CSV`
* `Run experiment and auto-evaluate results`
* `Get evaluator configuration for [evaluator-key]`
* `Create an LLM-as-a-Judge evaluator for tone`
* `Create a Python evaluator to check response length`
* `Add evaluator to experiment [experiment-key]`
* `Update evaluator [evaluator-key] with a new prompt`
* `Update Python evaluator [evaluator-key] with revised code`
* `List traces from the last 24 hours`
* `Show me traces with errors`
* `Get span details for trace [trace-id]`
* `Find the slowest traces from today`
* `Show all traces for thread [thread-id]`
* `List all available chat models`
* `List all available embedding models`
* `Invoke model [model-id] with prompt [message]`
* `Search for datasets named "customer"`
* `Find experiments in project [project-id]`
* `List directories in project [project-id]`
* `Search the Orq.ai docs for [topic]`
* `Delete agent [agent-key]`
* `Delete experiment [experiment-key]`
* `Delete evaluator [evaluator-key]`
* `Delete prompt [prompt-key]`
* `Delete knowledge base [knowledge-base-key]`
Use `delete_dataset` to delete a dataset along with all its datapoints.
### Usage Examples
#### Create an Experiment
```prompt wrap theme={"theme":{"light":"github-light","dark":"github-dark"}}
Create an experiment called "GPT-5.6 Sol vs Claude Sonnet 5 Comparison" using the "customer-queries" dataset
```
Claude Code will:
1. Use `search_entities` to find the "customer-queries" dataset
2. Use `create_experiment` with the specified name and dataset ID
3. Configure task columns with GPT-5.6 Sol and Claude Sonnet 5 models
4. Return the experiment ID and configuration details
#### Query Trace Analytics
```prompt wrap theme={"theme":{"light":"github-light","dark":"github-dark"}}
Has my system thrown any errors in the last 24 hours?
```
Claude Code will:
1. Calculate the time range for the last 24 hours
2. Use `list_traces` with error status filter
3. Analyze the error data
4. Provide a summary of total error count, error types and frequencies, affected traces, and time distribution
#### Create a Synthetic Dataset
```prompt wrap theme={"theme":{"light":"github-light","dark":"github-dark"}}
Create a dataset called "Product Questions" with 50 synthetic customer questions about e-commerce products
```
Claude Code will:
1. Generate 50 synthetic customer questions about e-commerce products
2. Use `create_dataset` to create a new dataset named "Product Questions"
3. Use `create_datapoints` to add all 50 questions to the dataset
4. Confirm creation with the dataset ID and summary
#### Performance Analysis
```prompt wrap theme={"theme":{"light":"github-light","dark":"github-dark"}}
Has my system's performance improved or decreased over the past week?
```
Claude Code will:
1. Use `query_analytics` with a 7-day time range
2. Analyze average latency trends over time
3. Review token usage patterns and cost variations
4. Compare error rate changes across the week
5. Provide insights on model performance comparisons and trends
#### Complete Experiment Creation
```prompt wrap theme={"theme":{"light":"github-light","dark":"github-dark"}}
I have a CSV file with 100 customer queries. Create a dataset, add an LLM evaluator for tone and accuracy, then run an experiment comparing GPT-5.6 Sol and Claude Sonnet 5
```
Claude Code will:
1. Read and parse the CSV file
2. Use `create_dataset` to create a new dataset with an auto-generated name
3. Use `create_datapoints` to add all 100 customer queries from the CSV
4. Use `create_llm_eval` to create an LLM-as-a-Judge evaluator for tone
5. Use `create_llm_eval` again to create an LLM-as-a-Judge evaluator for accuracy
6. Use `create_experiment` with the dataset ID and auto-run enabled
7. Configure two task columns (one for GPT-5.6 Sol, one for Claude Sonnet 5)
8. Execute the experiment automatically via the auto-run option
9. Summarize the results with evaluation scores for both models
#### Trace Investigation
```prompt wrap theme={"theme":{"light":"github-light","dark":"github-dark"}}
Show me the 10 slowest traces from yesterday and explain what might be causing the latency
```
Claude Code will:
1. Calculate yesterday's date range
2. Use `list_traces` with latency sorting (descending) and limit of 10
3. Use `list_spans` to retrieve span information for each trace
4. Analyze the execution patterns and span durations
5. Provide performance insights identifying bottlenecks
6. Suggest optimization opportunities based on the data
### Troubleshooting
1. Verify the API key is valid: `echo $ORQ_API_KEY`
2. Check the API key has the necessary permissions
3. Re-add the MCP with the correct API key
1. Verify the endpoint URL is correct
2. Check internet connectivity
3. Try removing and re-adding the integration
1. Get MCP server details: `claude mcp get orq`
2. Verify the MCP is properly installed: `claude mcp list`
## Plugins
The [`orq-ai/assistant-plugins`](https://github.com/orq-ai/assistant-plugins) marketplace exposes three plugins for Claude Code. Add the marketplace once, then install whichever plugins you need:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
claude plugin marketplace add orq-ai/assistant-plugins
```
| Plugin | Purpose |
| ------------ | ---------------------------------------------------------------------------------------------------------------------------------------- |
| `orq-skills` | Pre-built agentic workflows and slash commands for the Build, Evaluate, Optimize lifecycle. Includes the Orq MCP server. |
| `orq-mcp` | Standalone Orq MCP server. Use this if you only want platform tool access without the skills bundle. |
| `orq-trace` | Automatically traces Claude Code sessions to Orq.ai: captures sessions, turns, tool calls, and LLM responses as hierarchical OTLP spans. |
`orq-skills` already bundles the MCP server. Don't install `orq-mcp` alongside it or the MCP will be registered twice.
### Skills
**Skills** extend Claude Code with pre-built agentic workflows for the full Build, Evaluate, Optimize lifecycle. See the [Skills](/ai-studio/integrations/code-assistants/orq-skills) page for the full reference.
#### Installation
```bash Plugin (recommended) theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Requires marketplace to be added first (see Plugins section above)
# Installs skills, commands, agents, and the MCP server
claude plugin install orq-skills@orq-claude-plugin
```
```bash Local clone theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Clone and load locally as a fallback
git clone https://github.com/orq-ai/assistant-plugins.git ~/.orq-skills
claude --plugin-dir ~/.orq-skills
```
### MCP server only
If you only want the Orq MCP server without the skills bundle, install `orq-mcp` instead:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
claude plugin install orq-mcp@orq-claude-plugin
```
This gives Claude access to Orq.ai platform tools (agents, analytics, traces, experiments) without registering the agentic workflows.
### Session tracing
The `orq-trace` plugin captures every Claude Code session as a hierarchical trace in Orq.ai: useful for reviewing past coding sessions, sharing context with teammates, or analyzing tool-call patterns across runs.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
claude plugin install orq-trace@orq-claude-plugin
```
Set `ORQ_API_KEY` in the environment so the plugin can ship spans to the workspace. Sessions appear under [Traces](/ai-studio/observability/traces) once the next session starts.
### Commands
Quick slash-command actions available in Claude Code:
| Command | Description | Usage |
| --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- |
| **quickstart** | Interactive onboarding: credentials, MCP setup, skills tour | `/orq:quickstart` |
| **workspace** | Workspace overview: [Agents](/ai-studio/ai-engineering/build-agents), [Deployments](/ai-studio/ai-engineering/deployments), [Prompts](/ai-studio/prompts/prompts), [Datasets](/ai-studio/optimize/datasets), [Experiments](/ai-studio/optimize/experiments) | `/orq:workspace [section]` |
| **traces** | Query and summarize [Traces](/ai-studio/observability/traces) with filters | `/orq:traces [--deployment name] [--status error] [--last 24h]` |
| **models** | List available AI models by provider | `/orq:models [search-term]` |
| **analytics** | Usage [Analytics](/ai-studio/observability/quickstart): requests, cost, tokens, errors | `/orq:analytics [--last 24h] [--group-by model]` |
| **orq-manage-skills** | Manage **Orq.ai** [Skills](/ai-studio/ai-engineering/skills) (platform entities): list, get, create, update, retire, delete | `/orq:orq-manage-skills [list\|get\|create\|update\|retire\|delete] [name-or-id]` |
### Available Skills
Triggered by describing what you need. Claude Code picks the right skill automatically.
| Skill | Description | Source |
| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------- |
| **build-agent** | Design, create, and configure an **Orq.ai** [Agent](/ai-studio/ai-engineering/build-agents) with tools, instructions, [Knowledge Bases](/ai-gateway/features/knowledge-bases), and [Memory](/ai-studio/ai-engineering/memory-stores#manage-memories-and-documents) | [SKILL.md](https://github.com/orq-ai/assistant-plugins/blob/main/skills/orq-build-agent/SKILL.md) |
| **build-evaluator** | Create validated LLM-as-a-Judge [Evaluators](/ai-studio/marketplace#evaluators) following evaluation best practices | [SKILL.md](https://github.com/orq-ai/assistant-plugins/blob/main/skills/orq-build-evaluator/SKILL.md) |
| **evaluator-alignment** | Align an existing LLM judge (boolean, categorical, or numeric) to human judgment: measure how often it changes its mind, group the least reliable cases, rewrite the judge prompt, and recreate the [Evaluator](/ai-studio/marketplace#evaluators) after approval | [SKILL.md](https://github.com/orq-ai/assistant-plugins/blob/main/skills/orq-evaluator-alignment/SKILL.md) |
| **analyze-traces** | Read production [Traces](/ai-studio/observability/traces), identify what is failing, build failure taxonomies, and categorize issues | [SKILL.md](https://github.com/orq-ai/assistant-plugins/blob/main/skills/orq-analyze-traces/SKILL.md) |
| **run-experiment** | Create and run **Orq.ai** [Experiments](/ai-studio/optimize/experiments): compare configurations with specialized agent, conversation, and RAG evaluation | [SKILL.md](https://github.com/orq-ai/assistant-plugins/blob/main/skills/orq-run-experiment/SKILL.md) |
| **generate-synthetic-dataset** | Generate and curate evaluation [Datasets](/ai-studio/optimize/datasets): structured generation, quick from description, expansion, and dataset maintenance | [SKILL.md](https://github.com/orq-ai/assistant-plugins/blob/main/skills/orq-generate-synthetic-dataset/SKILL.md) |
| **invoke-deployment** | Invoke **Orq.ai** [Deployments](/ai-studio/ai-engineering/deployments), [Agents](/ai-studio/ai-engineering/build-agents), and models via the Python SDK or HTTP API, with correct variable substitution, streaming, and identity tracking | [SKILL.md](https://github.com/orq-ai/assistant-plugins/blob/main/skills/orq-invoke-deployment/SKILL.md) |
| **setup-observability** | Instrument LLM applications with Orq.ai tracing. Covers AI Gateway (zero-code traces) and OpenTelemetry/OpenInference. Guides from framework detection through baseline verification to trace enrichment | [SKILL.md](https://github.com/orq-ai/assistant-plugins/blob/main/skills/orq-setup-observability/SKILL.md) |
| **compare-agents** | Run cross-framework agent comparisons: compare any combination of Orq.ai, LangGraph, CrewAI, OpenAI Agents SDK, or Vercel AI SDK agents head-to-head on the same dataset using `evaluatorq` | [SKILL.md](https://github.com/orq-ai/assistant-plugins/blob/main/skills/orq-compare-agents/SKILL.md) |
| **red-team** | Run adversarial attacks against deployed agents or static datasets with the `evaluatorq` red team CLI. Covers OWASP-ASI (agentic: goal hijacking, tool misuse) and OWASP-LLM (model-level: prompt injection, system prompt leakage) | [SKILL.md](https://github.com/orq-ai/assistant-plugins/blob/main/skills/orq-red-team/SKILL.md) |
| **evaluatorq** | Write and run `evaluatorq` evaluation scripts (Python or TypeScript) for a single agent or deployment. Supports custom scorers, dataset-driven runs, and LLM-as-a-Judge [Evaluators](/ai-studio/marketplace#evaluators) | [SKILL.md](https://github.com/orq-ai/assistant-plugins/blob/main/skills/evaluatorq/SKILL.md) |
| **simulate-agent** | Run multi-turn simulations with `evaluatorq` primitives (`simulate()`, `generate_and_simulate()`, `wrap_simulation_agent()`): drive an agent under test with a simulated user and score each turn with a built-in judge | [SKILL.md](https://github.com/orq-ai/assistant-plugins/blob/main/skills/orq-simulate-agent/SKILL.md) |
| **manage-skills** | List, inspect, create, update, retire, and delete **Orq.ai** [Skills](/ai-studio/ai-engineering/skills) (platform entities). Handles naming rules, template integration (`{{skill.key}}`), reference scanning, and safe deletion | [SKILL.md](https://github.com/orq-ai/assistant-plugins/blob/main/skills/orq-manage-skills/SKILL.md) |
| **orq-cli** | Drive the `orq` command-line interface: install check, authentication, workspace selection, `orq doctor` troubleshooting, and read/write commands with JSON output | [SKILL.md](https://github.com/orq-ai/assistant-plugins/blob/main/skills/orq-cli/SKILL.md) |
## AI Gateway
Beta
Set the following environment variables before launching Claude Code. Once set, every model call Claude Code makes is automatically routed through the [**Orq.ai AI Gateway**](/ai-gateway/get-started/introduction) for the duration of that session.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
export ANTHROPIC_BASE_URL="https://my.orq.ai/v3/anthropic"
export ANTHROPIC_AUTH_TOKEN="$ORQ_API_KEY"
export ANTHROPIC_API_KEY="" # must be set to empty to prevent Claude Code from using the Anthropic API directly
export ANTHROPIC_MODEL="anthropic/claude-sonnet-5" # the anthropic/ prefix is required
```
Claude Code cannot modify the request body, so tag requests with `ANTHROPIC_CUSTOM_HEADERS` instead:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
export ANTHROPIC_CUSTOM_HEADERS=$'X-ORQ-METADATA-REPO: acme-api\nX-ORQ-METADATA-TICKET: PROJ-123'
```
See [Request Metadata](/ai-gateway/request-metadata) for the full header reference and precedence rules.
Claude Code requests routed through the **AI Gateway** appear in [Traces](/ai-studio/observability/traces).
See the [Anthropic Messages API](/ai-gateway/features/anthropic-messages-api) for endpoint details, authentication, and cache\_control breakpoints.
### Troubleshooting
The `ANTHROPIC_MODEL` value is missing the provider prefix. Use the `provider/model_id` format (e.g. `anthropic/claude-sonnet-5`, not `claude-sonnet-5`).
`ANTHROPIC_API_KEY` must be set to an empty string. When it holds a value, **Claude Code** uses the Anthropic API directly and requests bypass the **AI Gateway**.
Verify `ANTHROPIC_BASE_URL` is set to `https://my.orq.ai/v3/anthropic`. Requests sent to Anthropic's own endpoint bypass the **AI Gateway** and produce no **Traces**.
### Verification
Start a **Claude Code** session and send a prompt. The response appears in the terminal and the trace appears in [Traces](/ai-studio/observability/traces) with the model identifier `anthropic/claude-sonnet-5` (or whichever model `ANTHROPIC_MODEL` is set to).