Monitors

What Are Monitors

A Monitor is a custom dashboard made of widgets. Each widget charts one query over Traces, Metrics, or Logs, so a team can keep the numbers it cares about on a single page: cost per model, p95 latency, evaluator pass rate, and more.

Latency Monitor showing p50 and p95 as line charts and p99 as a bar chart over the last 30 minutes, with Slowest models and Slowest providers top lists below. — A Monitor built from the Latency template, with p50, p95, and p99 latency over the last 30 minutes.

Every Monitor has a scope:

Workspace: charts data across every Project in the workspace. Only workspace admins can create, edit, or delete workspace Monitors.
Project: charts data from a single Project.

The scope is set at creation and cannot be changed afterwards.

Creating a Monitor

AI Studio
API & SDK

New monitor dialog with the name Support overview, a description, the Workspace (all projects) scope, the GenAI overview template, and a Last day default range. — The New monitor dialog, with a template selected.

Open Monitors

Open Monitors in the Observability section of the sidebar, then choose + Monitor.

Name the monitor

Enter a Name and an optional Description of what the Monitor tracks.

Choose the scope

Select Workspace (all projects) or a Project. The dialog opens on the active project. Other members only see the Projects they have access to.

Pick a template

Under Start from template, keep Blank monitor or start from a ready-made dashboard. See Templates.

Set the default range

Default range is the time window the dashboard opens with. Automatic uses the standard range. Ranges beyond the plan’s data retention are not offered.

Create

In the New monitor dialog, choose Create. The dashboard opens straight away.

Every request is authenticated with an API key.

cURL

curl -X POST "https://my.orq.ai/v2/monitors" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "Production cost",
    "description": "Cost tracking for production",
    "project_id": "proj_01HZXW2K7Y8Q9M0N1P2R3S4T5V",
    "default_range": "1d",
    "widgets": [
      {
        "widget_id": "total-cost",
        "type": "bignumber",
        "title": "Total cost",
        "queries": [{ "metric": "genai.cost" }],
        "layout": { "x": 0, "y": 0, "w": 4, "h": 3 }
      }
    ]
  }'

{
  "monitor": {
    "monitor_id": "monitor_01m4cy71kr64ayad4gvmgqtgeg",
    "display_name": "Production cost",
    "description": "Cost tracking for production",
    "project_id": "proj_01HZXW2K7Y8Q9M0N1P2R3S4T5V",
    "widgets": [
      {
        "widget_id": "total-cost",
        "type": "bignumber",
        "title": "Total cost",
        "queries": [
          {
            "source": "MONITOR_TELEMETRY_SOURCE_TRACES",
            "metric": "genai.cost",
            "operation": "sum",
            "group_by": [],
            "filters": []
          }
        ],
        "display": null,
        "layout": { "x": 0, "y": 0, "w": 4, "h": 3 }
      }
    ],
    "default_range": "1d",
    "preset_key": "",
    "workspace_id": "ws_01JQ8Z0MB5W1YTVN3GCK7B0EXA",
    "created_at": "2026-10-08T05:01:43.928Z",
    "updated_at": "2026-10-08T05:01:43.928Z",
    "created_by_id": "01JQ8Z1TF7QK9D2RH4NMV6PY3C",
    "updated_by_id": "01JQ8Z1TF7QK9D2RH4NMV6PY3C"
  }
}

Parameters

Parameter	Type	Required	Description
`display_name`	string	Yes	Workspace-unique name, up to 255 characters.
`description`	string	No	Summary of what the Monitor tracks, up to 1024 characters.
`project_id`	string	No	Owning Project. Omitted creates a workspace-wide Monitor, which requires workspace admin access. Immutable after creation.
`widgets`	array	No	Widgets to render, up to 30. Omitted creates a blank Monitor.
`default_range`	enum	No	`5m`, `15m`, `30m`, `1h`, `6h`, `12h`, `1d`, `7d`, `14d`, `30d`, `60d`, or `90d`.

A widget carries a type of timeseries, toplist, table, bignumber, donut, or breakdown, exactly one queries entry, and a layout grid position (x, y, w, h). A query names the metric and can add group_by, filters, grain, limit, source, and operation. A breakdown widget also takes additional_computes, up to five metrics rendered as columns after the measure. For a genai.* trace metric the server fills source and operation when they are omitted; other datasets require both explicitly.

Templates

Template	What it shows	Default range
GenAI overview	Total requests, cost, tokens, and errors, requests over time by model, p95 latency, and cost and errors by model.	Last day
Cost analysis	Total cost, requests, and tokens, cost over time by model, and cost by model and by Project.	Last 7 days
Latency	p50, p95, and p99 latency over time, and the slowest models and providers at p95.	Last day
Evaluator quality	Evaluator runs, pass rate, guardrails triggered, pass rate and average score over time, and guardrail block rate per evaluator.	Last 7 days

Every widget created from a template can be edited, moved, or deleted like any other.

Using the Dashboard

The dashboard header holds the controls for the whole Monitor:

Time range: pick a preset range or a custom range. The range applies to every widget and is kept in the page URL, so a link to the dashboard opens the same window. Ranges are capped at the plan’s data retention.
Refresh: reloads every widget.
+ Widget: opens the widget editor. See Adding a Widget.
Monitor actions: Settings, Duplicate, and Delete.

Drag a widget by its header to move it, or drag its edge to resize it. Once the layout changes, choose Save layout to keep it, or Cancel to discard it. Open the menu on a widget to:

Edit: reopen the widget in the editor.
Duplicate: add a copy to the same Monitor.
Create alert: open the Alerts form with the widget’s metric, filters, and Project already filled in. A widget’s target line becomes the alert threshold. Available for GenAI metrics from Traces. Alerts have no split, so Split results by is not carried over, and a workspace Monitor needs a Project picked in the form. The alert’s Notifiers are selected in that form.
Delete: remove the widget.

Time series widgets drawn as lines or bars also show a toggle in the header to switch between a line chart and a bar chart. The widget editor keeps the configuration on the left and a live Preview of the widget beside it, using the dashboard’s current time range.

Edit widget page configuring Latency p95 over Traces, with Dataset, Measure, Filter, Split results by, Interval and display options on the left and a preview of the chart on the right. — Editing the Latency p95 widget. The preview updates as the configuration changes.

Choose the data

Dataset: Traces, Metrics, or Logs. Only the datasets available to the workspace are listed.
Measure: the metric to chart, for example Requests, LLM cost, Latency p95, or Evaluator pass rate. See Measures.
Aggregate: how values are combined, such as count, sum, average, or a percentile. The options depend on the measure.
Interval: the bucket size for time series, Auto, Minute, Hour, or Day.

Filter the data

Under Where, choose + Filter to restrict the query, for example to one model, provider, or Deployment. Up to 20 filters per widget. Where the field has known values, the value box suggests them while typing; pick one or enter a value.

Split the results

Under Split results by, select one or more dimensions, such as Model, Provider, Project, Agent, or Tool, to get one series or row per value. Leave it empty to chart all results together.For top lists, tables, donuts, and breakdowns, the same section sets Top or Bottom and the result count. Top lists and donuts also offer Group the rest as Other, which folds the remaining values into one row.

Name and visualize

Enter a Title, then pick a Visualization. Only visualizations that fit the query are listed:

Visualization	When it is available
Line, Bar, Area	Values over time, with or without a split.
Top list, Table	A single value per split value. Requires Split results by.
Donut	A single value per split value, as shares of a total. Requires Split results by.
Breakdown	A single value per split value, with a second dimension nested under the first and up to five more metrics as columns. Requires Split results by.
Big number	A single value for the whole range. No split.

Adjust display options

Display options fine tune the widget. Which options appear depends on the visualization:

Value precision: decimal places, from 0 to 6. Leave blank for automatic.
Show overall: add a total row (top lists, tables, and breakdowns).
Show percentage: show each value’s share (top lists and donuts, with an additive measure).
Target line: a reference value drawn on the chart (time series and big numbers).

Save

Check the Preview, then choose Save. The widget is added to the Monitor.

Measures

The Traces dataset provides the GenAI measures below.

Area	Measures
Usage	`Requests`, `Tokens`
Cost	`LLM cost`
Errors	`Errors`, `Error rate`
Latency	`Latency (avg)`, `Latency p50`, `Latency p95`, `Latency p99`
Streaming	`Time to first token (avg)`, `Time to first token p50`, `Time to first token p95`, `Time to first token runs`
Evaluators	`Evaluator runs`, `Evaluator pass rate`, `Evaluator failure rate`, `Evaluator score (avg)`, `Evaluator errors`, `Evaluator error rate`
Guardrails	`Guardrail runs`, `Guardrails triggered`, `Guardrail block rate`

Managing Monitors

AI Studio
API & SDK

Monitors list with Name, Scope, Widgets, Last update, and Updated by columns, showing two workspace-scoped Monitors with eight and five widgets. — The Monitors list, filtered by scope.

The Monitors list shows each Monitor with its scope, widget count, last update, and who updated it.

Search by name or description, and Filter by Scope.
Open the menu on a row to open, Duplicate, or Delete a Monitor. A duplicate copies every widget.
To rename a Monitor, change its description, or change its default range, open it and choose Settings from the menu.

Operation	Endpoint
Create a Monitor	`POST /v2/monitors`
List Monitors	`GET /v2/monitors`
Retrieve a Monitor	`GET /v2/monitors/{monitor_id}`
Update a Monitor	`PATCH /v2/monitors/{monitor_id}`
Delete a Monitor	`DELETE /v2/monitors/{monitor_id}`

GET /v2/monitors returns the Monitors visible to the caller, newest first, with has_more. A caller filtering by project_id sees that project’s Monitors plus workspace-wide Monitors; a project-scoped API key sees only its own project. Page with starting_after or ending_before set to a monitor_id from the previous page. limit accepts 1 to 200 and defaults to 25. project_id restricts the result to one project, search matches the display name and description case-insensitively, and workspace_wide narrows to workspace-wide or project-scoped Monitors.PATCH /v2/monitors/{monitor_id} replaces only the fields it carries. Protobuf JSON encodes update_mask as a comma-separated string of paths, drawn from display_name, description, widgets, and default_range:

cURL

curl -X PATCH "https://my.orq.ai/v2/monitors/$MONITOR_ID" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "widgets": [],
    "update_mask": "widgets"
  }'

A masked path is replaced outright, so widgets can be cleared. Without a mask, a non-empty widgets array still replaces the whole set and an empty or omitted array keeps the existing widgets, so retrieve the Monitor first when editing a single widget. project_id is immutable. Deleting a Monitor deletes its widgets with it, since they are stored inline.

Permissions

Viewing, creating, editing, and deleting Monitors each depend on the member’s role. Members without edit access see dashboards read only: widgets cannot be moved, resized, added, or edited. Workspace Monitors additionally require a workspace admin to create, edit, or delete them. Create alert on a widget also requires permission to create Alerts. For API keys, Read permission covers list and retrieve, and Write covers create, update, and delete.

Overview

Instrument your app

Explore

Monitor

Export

Review

Data privacy & security

What Are Monitors

Creating a Monitor

Templates

Using the Dashboard

Widget actions

Adding a Widget

Measures

Managing Monitors

Permissions

​What Are Monitors

​Creating a Monitor

​Templates

​Using the Dashboard

​Widget actions

​Adding a Widget

​Measures

​Managing Monitors

​Permissions

What Are Monitors

Creating a Monitor

Templates

Using the Dashboard

Widget actions

Adding a Widget

Measures

Managing Monitors

Permissions