Skip to main content

MCP Integration

Access the Orq.ai workspace directly from Claude Code. Manage experiments, query traces, and configure agents using natural language.

AI Gateway  Beta

Route Claude Code’s model calls through the AI Gateway.
Claude Code is a terminal CLI. The AI Gateway settings on this page are environment variables.

MCP

Claude Code is Anthropic’s official CLI that brings Claude’s capabilities to the terminal and development workflow. With the Orq MCP integration, access all Orq.ai features directly through Claude Code’s conversational interface.

Prerequisites

Installation

Add the Orq MCP server to Claude Code with a single command:
Set the ORQ_API_KEY environment variable before running the command:
Windows (PowerShell):

Verify Installation

Check that the Orq MCP is installed:
You should see orq in the list of available MCP servers.

Available Commands

Once integrated, you can ask Claude Code to perform these operations:
  • Create an agent with custom instructions and tools
  • Get agent configuration for [agent-key]
  • Update agent [agent-key] with new instructions or model
  • Configure agent with evaluators and guardrails
  • Invoke agent [agent-key] with input [message]
  • Retrieve agent response [response-id]
  • Create a deployment called [deployment-key]
  • Get deployment configuration for [deployment-key]
  • Create a skill called [skill-key]
  • List all skills in my workspace
  • Get skill [skill-key]
  • Update skill [skill-key]
  • Delete skill [skill-key]
  • Get analytics overview for my workspace
  • Show me workspace metrics for the last 7 days
  • Query analytics filtered by deployment ID
  • Create a dataset called "customer-queries"
  • List all datapoints in dataset [dataset-key]
  • Add datapoints to dataset [dataset-key]
  • Update datapoint [datapoint-id]
  • Delete specific datapoints in dataset [dataset-key]
  • Delete dataset [dataset-key]
  • Create an experiment from dataset [dataset-key]
  • List all experiment runs
  • Export experiment run [run-id] as CSV
  • Run experiment and auto-evaluate results
  • Get evaluator configuration for [evaluator-key]
  • Create an LLM-as-a-Judge evaluator for tone
  • Create a Python evaluator to check response length
  • Add evaluator to experiment [experiment-key]
  • Update evaluator [evaluator-key] with a new prompt
  • Update Python evaluator [evaluator-key] with revised code
  • List traces from the last 24 hours
  • Show me traces with errors
  • Get span details for trace [trace-id]
  • Find the slowest traces from today
  • Show all traces for thread [thread-id]
  • List all available chat models
  • List all available embedding models
  • Invoke model [model-id] with prompt [message]
  • Search for datasets named "customer"
  • Find experiments in project [project-id]
  • List directories in project [project-id]
  • Search the Orq.ai docs for [topic]
  • Delete agent [agent-key]
  • Delete experiment [experiment-key]
  • Delete evaluator [evaluator-key]
  • Delete prompt [prompt-key]
  • Delete knowledge base [knowledge-base-key]
Use delete_dataset to delete a dataset along with all its datapoints.

Usage Examples

Create an Experiment

Claude Code will:
  1. Use search_entities to find the “customer-queries” dataset
  2. Use create_experiment with the specified name and dataset ID
  3. Configure task columns with GPT-5.6 Sol and Claude Sonnet 5 models
  4. Return the experiment ID and configuration details

Query Trace Analytics

Claude Code will:
  1. Calculate the time range for the last 24 hours
  2. Use list_traces with error status filter
  3. Analyze the error data
  4. Provide a summary of total error count, error types and frequencies, affected traces, and time distribution

Create a Synthetic Dataset

Claude Code will:
  1. Generate 50 synthetic customer questions about e-commerce products
  2. Use create_dataset to create a new dataset named “Product Questions”
  3. Use create_datapoints to add all 50 questions to the dataset
  4. Confirm creation with the dataset ID and summary

Performance Analysis

Claude Code will:
  1. Use query_analytics with a 7-day time range
  2. Analyze average latency trends over time
  3. Review token usage patterns and cost variations
  4. Compare error rate changes across the week
  5. Provide insights on model performance comparisons and trends

Complete Experiment Creation

Claude Code will:
  1. Read and parse the CSV file
  2. Use create_dataset to create a new dataset with an auto-generated name
  3. Use create_datapoints to add all 100 customer queries from the CSV
  4. Use create_llm_eval to create an LLM-as-a-Judge evaluator for tone
  5. Use create_llm_eval again to create an LLM-as-a-Judge evaluator for accuracy
  6. Use create_experiment with the dataset ID and auto-run enabled
  7. Configure two task columns (one for GPT-5.6 Sol, one for Claude Sonnet 5)
  8. Execute the experiment automatically via the auto-run option
  9. Summarize the results with evaluation scores for both models

Trace Investigation

Claude Code will:
  1. Calculate yesterday’s date range
  2. Use list_traces with latency sorting (descending) and limit of 10
  3. Use list_spans to retrieve span information for each trace
  4. Analyze the execution patterns and span durations
  5. Provide performance insights identifying bottlenecks
  6. Suggest optimization opportunities based on the data

Troubleshooting

  1. Verify the API key is valid: echo $ORQ_API_KEY
  2. Check the API key has the necessary permissions
  3. Re-add the MCP with the correct API key
  1. Verify the endpoint URL is correct
  2. Check internet connectivity
  3. Try removing and re-adding the integration
  1. Get MCP server details: claude mcp get orq
  2. Verify the MCP is properly installed: claude mcp list

Plugins

The orq-ai/assistant-plugins marketplace exposes three plugins for Claude Code. Add the marketplace once, then install whichever plugins you need:
orq-skills already bundles the MCP server. Don’t install orq-mcp alongside it or the MCP will be registered twice.

Skills

Skills extend Claude Code with pre-built agentic workflows for the full Build, Evaluate, Optimize lifecycle. See the Skills page for the full reference.

Installation

MCP server only

If you only want the Orq MCP server without the skills bundle, install orq-mcp instead:
This gives Claude access to Orq.ai platform tools (agents, analytics, traces, experiments) without registering the agentic workflows.

Session tracing

The orq-trace plugin captures every Claude Code session as a hierarchical trace in Orq.ai: useful for reviewing past coding sessions, sharing context with teammates, or analyzing tool-call patterns across runs.
Set ORQ_API_KEY in the environment so the plugin can ship spans to the workspace. Sessions appear under Traces once the next session starts.

Commands

Quick slash-command actions available in Claude Code:

Available Skills

Triggered by describing what you need. Claude Code picks the right skill automatically.

Telemetry only

Keep Claude Code on the Anthropic login and still send traces to Orq.ai. Three setups do this; they differ in scope and in what they capture. orq connect and orq launch are documented in the Orq CLI reference, including the credential each resolves and how orq disconnect undoes a connect. Pass --no-otel to orq launch claude when the session should stay uncaptured. Whatever the route, the stored copy is governed by the trace settings rather than the setup: Trace scrubbing selects which fields a stored trace keeps, and Trace data masking masks content through the security parameter.

AI Gateway

Beta Route Claude Code model requests through the Orq.ai AI Gateway. Set an Orq.ai API key as ORQ_API_KEY, then choose the credential to use for model requests.

Use the workspace’s provider configuration

Use the Orq.ai API key as the model client’s credential:
Shell
Windows (PowerShell):
To use an integration other than the workspace default, add x-orq-integration-anthropic with the integration id or its exact display name. See Provider integrations.
Shell
Windows (PowerShell):

Use a Claude subscription

Sign in to Claude Code with a Claude subscription. Keep that login for Anthropic and send the Orq.ai key separately:
Shell
Windows (PowerShell):
Claude Code sends the subscription token and its OAuth beta header. The AI Gateway forwards the credential to Anthropic, so the subscription’s usage limits apply. Orq.ai still records the model request in Traces. For automated sessions, run claude setup-token and set the resulting token as CLAUDE_CODE_OAUTH_TOKEN before launching Claude Code. Keep ANTHROPIC_AUTH_TOKEN and ANTHROPIC_API_KEY unset so they do not replace the subscription credential.

Use an Anthropic API key

Set an Anthropic API key as ANTHROPIC_API_KEY and keep the Orq.ai key in the custom header:
Shell
Windows (PowerShell):
Anthropic bills model usage to the account that owns the API key. Both Anthropic credential options require a public model served directly by Anthropic. See authentication for the header reference.

Route to AWS Bedrock or Vertex AI

Claude Code sends bare model names, such as claude-haiku-5-5, for background tasks and for model aliases without an override. The AI Gateway resolves a bare name to anthropic/… whenever Anthropic serves that model, never to its AWS Bedrock or Vertex AI offering. On a workspace that serves Claude through AWS Bedrock or Vertex AI, pin the main model and every alias to the workspace’s model ids. Otherwise helper requests go to Anthropic, or are rejected when the workspace or API key does not allow Anthropic models. Deploy the configuration as managed settings so it applies to every session and overrides user and project settings. Save it as managed-settings.json in /Library/Application Support/ClaudeCode/ on macOS, /etc/claude-code/ on Linux and WSL, or C:\Program Files\ClaudeCode\ on Windows, or deliver it through an MDM policy. Server-managed settings from the claude.ai console do not apply: Claude Code skips them when ANTHROPIC_BASE_URL points to a gateway. See Anthropic’s managed settings guide.
JSON
  • Model ids: replace the examples with models the workspace enables. For a private model, such as an AWS Bedrock application inference profile, use its <workspace>@aws/<model> id; see Referencing private models. ANTHROPIC_DEFAULT_HAIKU_MODEL also sets the model for background tasks. Set ANTHROPIC_DEFAULT_FABLE_MODEL the same way when the workspace enables a Fable model.
  • Provider variables: never combine CLAUDE_CODE_USE_BEDROCK or CLAUDE_CODE_USE_VERTEX with the AI Gateway. They make Claude Code call AWS Bedrock or Vertex AI in that provider’s own request format, and /v3/anthropic accepts only Anthropic Messages requests. The empty values clear a stale shell export.
  • Tool search: ENABLE_TOOL_SEARCH loads MCP tool definitions on demand; see Enable MCP tool search.
  • Experimental betas: keep CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS out of every settings file and shell profile. It disables tool search and server-side auto mode checks, so Claude Code sends separate, billable classifier requests and shows the classifier billing notice; see Use server-side auto mode checks.
  • Credentials: keep Orq.ai API keys out of the shared file. Each developer sets ANTHROPIC_AUTH_TOKEN and an empty ANTHROPIC_API_KEY as in Use the workspace’s provider configuration.
Run /status in a new session and check that Setting sources lists the managed settings. Claude Code disables MCP tool search by default when ANTHROPIC_BASE_URL points to a custom host. After configuring the AI Gateway, enable tool search to load MCP tool definitions on demand:
Shell
Windows (PowerShell):
Keep experimental betas enabled; disabling them also disables tool search, even with ENABLE_TOOL_SEARCH=true. Use a compatible Claude model and check the Anthropic tool search configuration for model requirements. To persist the setting, merge this env setting into ~/.claude/settings.json, preserving existing entries, then start a new session:
JSON
Use true for on-demand loading. The auto setting loads tool definitions upfront until they reach 10% of the model’s context window, which is 100,000 tokens for a 1M-token model. Remove CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS from shell and settings configuration; contact the administrator if a managed policy sets it. In the new session, run /mcp to confirm the server is connected and /context to inspect MCP tool loading. Ask Claude Code to list the available Orq.ai models, then ask a follow-up that uses the result. Verify the tool search, actual MCP call, and follow-up all succeed; the context display alone does not verify tool execution. Deferred MCP tool loading and execution were validated with Claude Code 2.1.301, using Claude Opus 5.5 through Anthropic and AWS Bedrock. Context savings depend on the configured tools. For Claude Desktop, use the separate Desktop tool-search setting.

Use server-side auto mode checks

In auto mode, Claude Code can request safety checks as part of model responses, avoiding separate, billable classifier requests. Orq.ai forwards these requests and results, including streaming results, through the AI Gateway. Availability of this compatibility depends on the upstream provider, model, region, and credentials. After configuring the gateway connection, remove overrides that disable server-side checks and start a fresh session:
Shell
Remove those disabling overrides from the env settings in ~/.claude/settings.json and project settings too. Contact the administrator if a managed policy sets them. ENABLE_TOOL_SEARCH controls MCP tool loading separately; it does not enable classifier checks. Run /status in the new session and inspect Auto mode server. Enabled means server-side checks are deciding actions; Disabled means the session uses client-side classification. Server-side checks do not change billing for regular model usage. Setting CLAUDE_CODE_AUTO_MODE_SERVER=0 suppresses the eligibility notice by always using client-side classifier requests. This does not remove charges for those requests. See Anthropic’s classifier billing guide.

Choose the prompt cache TTL

Claude Code sets the TTL of the cache breakpoints it sends. Control it in the same env block: A 1-hour cache write costs 2x the input price, against 1.25x for 5 minutes, so the 1-hour TTL pays off only from the third request that reuses the prefix. See Cache TTL.

Add request metadata

Tag requests with ANTHROPIC_CUSTOM_HEADERS. Separate headers with newlines and keep x-orq-api-key when using an Anthropic credential:
Shell
See Request Metadata for the full header reference and precedence rules.
Claude Code requests routed through the AI Gateway appear in Traces.
See the Anthropic Messages API for endpoint details, authentication, and cache_control breakpoints.

Troubleshooting

Follow Enable MCP tool search, then start a new session. Confirm ENABLE_TOOL_SEARCH=true, keep experimental betas enabled, and check that permissions do not deny ToolSearch. Inspect /context with the same model, MCP servers, Skills, and conversation when comparing usage. Tool search reduces MCP schema loading; system prompts, built-in tools, and conversation history also consume context.
Follow Use server-side auto mode checks and start a fresh session. If the notice remains, share the Auto mode server row from /status, the Claude Code version, the exact model ID, the provider region, and the values of CLAUDE_CODE_AUTO_MODE_SERVER and CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS with support, including whether each is unset. Do not include API keys. The notice can indicate either missing gateway pass-through or unavailable upstream checks; a gateway upgrade alone does not establish session eligibility.
A bare model name such as claude-haiku-5-5 resolves to anthropic/…. Pin ANTHROPIC_MODEL, ANTHROPIC_DEFAULT_OPUS_MODEL, ANTHROPIC_DEFAULT_SONNET_MODEL, and ANTHROPIC_DEFAULT_HAIKU_MODEL to the workspace’s model ids, as in Route to AWS Bedrock or Vertex AI.
Set ANTHROPIC_BASE_URL to https://my.orq.ai/v3/anthropic in the terminal that launches Claude Code, then start a new session.
Keep the Orq.ai key in x-orq-api-key. Send the Anthropic credential through the selected setup above. When using a subscription, unset ANTHROPIC_AUTH_TOKEN and ANTHROPIC_API_KEY, then run /status to check the active Claude login.
Verify ANTHROPIC_BASE_URL is set to https://my.orq.ai/v3/anthropic. Requests sent to Anthropic’s own endpoint bypass the AI Gateway and produce no Traces.

Verification

Check a multi-turn session: a single response does not exercise tool search or prompt caching. Start a new Claude Code session with an MCP server connected, such as the Orq MCP:
  1. Send a prompt that needs an MCP tool, such as List the available Orq.ai models. Confirm the transcript shows a tool search followed by the MCP tool call.
  2. Send a follow-up that uses the result, such as Which of those models support tool calling?.
  3. In Traces, check that every request uses a model id set in ANTHROPIC_MODEL or an ANTHROPIC_DEFAULT_*_MODEL variable. On an AWS Bedrock or Vertex AI workspace, an anthropic/… id means a model is not pinned.
  4. Open the follow-up’s trace. A gen_ai.usage.prompt_tokens_details.cached_tokens value greater than zero shows that the follow-up read the prefix cached by the first turn. See Cache usage in Traces.