Prerequisites
- Visual Studio Code 1.99 or later
- GitHub Copilot extension installed and active
- Active Orq.ai account
- Orq.ai API key
Installation
Add MCP Server
- Open the Command Palette with ⌘⇧P (macOS) or Ctrl+Shift+P (Windows/Linux)
- Search for and select MCP: Add Server
- Select HTTP (HTTP or Server-Sent Events) as the server type
- Enter the server URL:
https://my.orq.ai/v2/mcp - Name the server
orqwhen prompted
.vscode/mcp.json file in the workspace root. Open it and replace the contents with:
Verification
- Open Copilot Chat with ⌃⌘I (macOS) or Ctrl+Alt+I (Windows/Linux)
- Ask:
Available Commands
Use natural language to ask Copilot to perform these operations:Agents
Agents
Create an agent with custom instructions and toolsGet agent configuration for [agent-key]Update agent [agent-key] with new instructions or modelConfigure agent with evaluators and guardrailsInvoke agent [agent-key] with input [message]Retrieve agent response [response-id]
Deployments
Deployments
Create a deployment called [deployment-key]Get deployment configuration for [deployment-key]
Skills
Skills
Create a skill called [skill-key]List all skills in my workspaceGet skill [skill-key]Update skill [skill-key]Delete skill [skill-key]
Analytics
Analytics
Get analytics overview for my workspaceShow me workspace metrics for the last 7 daysQuery analytics filtered by deployment ID
Datasets
Datasets
Create a dataset called "customer-queries"List all datapoints in dataset [dataset-key]Add datapoints to dataset [dataset-key]Update datapoint [datapoint-id]Delete specific datapoints in dataset [dataset-key]Delete dataset [dataset-key]
Experiments
Experiments
Create an experiment from dataset [dataset-key]List all experiment runsExport experiment run [run-id] as CSVRun experiment and auto-evaluate results
Evaluators
Evaluators
Get evaluator configuration for [evaluator-key]Create an LLM-as-a-Judge evaluator for toneCreate a Python evaluator to check response lengthAdd evaluator to experiment [experiment-key]Update evaluator [evaluator-key] with a new promptUpdate Python evaluator [evaluator-key] with revised code
Traces
Traces
List traces from the last 24 hoursShow me traces with errorsGet span details for trace [trace-id]Find the slowest traces from todayShow all traces for thread [thread-id]
Models
Models
List all available chat modelsList all available embedding modelsInvoke model [model-id] with prompt [message]
Search
Search
Search for datasets named "customer"Find experiments in project [project-id]List directories in project [project-id]
Documentation
Documentation
Search the Orq.ai docs for [topic]
Managing Entities
Managing Entities
Delete agent [agent-key]Delete experiment [experiment-key]Delete evaluator [evaluator-key]Delete prompt [prompt-key]Delete knowledge base [knowledge-base-key]
delete_dataset to delete a dataset along with all its datapoints.Troubleshooting
Orq MCP Not Responding
Orq MCP Not Responding
- Open the Command Palette and run MCP: List Servers
- Select the
orqserver and choose Show Output to view logs - Restart VS Code and reconnect. VS Code will prompt for the API key again on first connection
Authentication Errors
Authentication Errors
- Confirm the API key is valid in Workspace Settings → API Keys
- Ensure the API key has the necessary permissions
- Try regenerating the API key, then restart VS Code so the input prompt appears again
Tools Not Available
Tools Not Available
- Run MCP: List Servers from the Command Palette and confirm the
orqserver status is active - Ensure GitHub Copilot is signed in and active
- Check network connectivity
- Review the server output log for error details