What is the Orq MCP?
The Orq Model Context Protocol (MCP) server provides AI code assistants with direct access to your Orq.ai workspace. With 21 specialized tools, you can manage experiments, create datasets, configure evaluators, and analyze traces without leaving your IDE.Key Capabilities
Experiment Management
Create and run experiments, configure task columns with prompts or agents, and export results in multiple formats
Dataset Operations
Create synthetic datasets, reshape local data, manage datapoints, and map data to experiments
Analytics & Insights
Query workspace analytics, track performance metrics, and ask natural language questions about your traces
Evaluator Configuration
Create LLM-as-a-Judge evaluators and Python code evaluators for automated quality assessment
Available Tools
The Orq MCP provides 21 tools across 8 categories:| Category | Tool | Description |
|---|---|---|
| Analytics | get_analytics_overview | Get workspace snapshot (requests, cost, tokens, errors, latency, top models) |
| Analytics | query_analytics | Flexible drill-down with filtering and grouping |
| Dataset | create_dataset | Create a new dataset |
| Dataset | list_datapoints | List datapoints in a dataset |
| Dataset | create_datapoints | Create datapoints (max 100) |
| Dataset | update_datapoint | Update a datapoint |
| Dataset | delete_datapoints | Delete datapoints (max 100) |
| Dataset | delete_dataset | Delete a dataset and all datapoints |
| Evaluator | create_llm_eval | Create LLM-as-a-Judge evaluator |
| Evaluator | create_python_eval | Create Python code evaluator |
| Experiment | list_experiment_runs | List runs with cursor pagination |
| Experiment | get_experiment_run | Export run (JSON/JSONL/CSV) |
| Experiment | create_experiment | Create experiment from dataset with optional auto-run |
| Models | list_models | List all available AI models |
| Registry | list_registry_keys | List available attribute keys for filtering traces |
| Registry | list_registry_values | List top values for a specific attribute |
| Search | search_entities | Search projects, datasets, prompts, or experiments |
| Search | search_directories | List directories within a project |
| Traces | list_traces | List traces with filtering and sorting |
| Traces | get_span | Retrieve a single span (compact or full mode) |
| Traces | list_spans | List all spans in a trace |