Reporting API

Query AI usage, cost, latency, Evaluator outcomes, and guardrail metrics programmatically. The Reporting API returns time-series data sliced by model, provider, project, identity, and more in a single JSON request.

The Reporting API is deprecated. Existing calls to POST /v2/reporting keep working, but new integrations should use the Telemetry API, and existing ones should migrate to it.To migrate a request:

Send it to POST /v3/telemetry/query with "source": "TELEMETRY_SOURCE_TRACES".
Replace "metric": "genai.cost" with "compute": [{ "metric": "genai.cost", "op": "sum" }]. The genai.* metric names carry over; see Sources and metrics for each metric’s operations. For genai.usage, use "op": "bundle".
Read group instead of dimensions on each row. Metric keys are the metric and operation joined by a colon, for example genai.cost:sum.
In the SDKs, call orq.telemetry.query() instead of orq.reporting.query().

Spend dashboards

Track cost by model, provider, project, or credential type. Build per-customer billing breakdowns.

Performance monitoring

Monitor p50, p95, and p99 latency over time across models and providers.

Evaluator quality

Quantify pass rate, average score, and result distribution per Evaluator and version.

Guardrail enforcement

Measure block rate, triggers, and outcomes by Policy and stage.

Endpoint

POST https://my.orq.ai/v2/reporting

All requests require a Bearer token. See API Keys for how to generate one.
The workspace and project scope are derived from the API key itself.

Trace events flow into reporting in near real time, with a small ingestion delay measured in seconds. Queries against very recent windows may not yet include the latest events.

See the API Reference

Interactive playground and full schema for POST /v2/reporting.

Request

The wire format is snake_case end-to-end. SDKs expose the same fields in each language’s native style (groupBy in TypeScript, group_by in Python).

{
  "metric": "genai.usage",
  "from": "<ISO-8601 start datetime>",
  "to":  "<ISO-8601 end datetime>",
  "grain": "day",
  "group_by": ["model", "provider"],
  "filters": [
    { "field": "project", "op": "eq", "values": ["proj_123"] }
  ],
  "include_totals": true,
  "limit": 1000,
  "time_zone": "UTC"
}

FROM=$(date -u -v-1d +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || date -u -d "1 day ago" +%Y-%m-%dT%H:%M:%SZ)
TO=$(date -u +%Y-%m-%dT%H:%M:%SZ)
curl -X POST "https://my.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"metric\": \"genai.usage\",\"from\": \"$FROM\",\"to\":   \"$TO\",\"grain\": \"day\",\"group_by\": [\"model\", \"provider\"],\"filters\": [{ \"field\": \"project\", \"op\": \"eq\", \"values\": [\"proj_123\"] }],\"include_totals\": true,\"limit\": 1000,\"time_zone\": \"UTC\"}"

import { Orq } from "@orq-ai/node";

const orq = new Orq({ apiKey: process.env.ORQ_API_KEY! });

const report = await orq.reporting.query({
  metric: "genai.usage",
  from: new Date(Date.now() - 24 * 60 * 60 * 1000),
  to: new Date(),
  grain: "day",
  mode: "timeseries",
  sort: "desc",
  groupBy: ["model", "provider"],
  filters: [
    { field: "project", op: "eq", values: ["proj_123"] }
  ],
  includeTotals: true,
  limit: 1000,
  timeZone: "UTC",
});

import os
from datetime import datetime, timedelta, timezone

from orq_ai_sdk import Orq

orq = Orq(api_key=os.environ["ORQ_API_KEY"])

report = orq.reporting.query(
    metric="genai.usage",
    from_=(datetime.now(timezone.utc) - timedelta(days=1)).strftime("%Y-%m-%dT%H:%M:%SZ"),
    to=datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
    grain="day",
    mode="timeseries",
    sort="desc",
    group_by=["model", "provider"],
    filters=[
        {"field": "project", "op": "eq", "values": ["proj_123"]}
    ],
    include_totals=True,
    limit=1000,
    time_zone="UTC",
)

grain sets the time bucket: auto, minute, hour, or day. auto, or an omitted field, lets the server choose the bucket. The response reports the applied bucket in meta.effective_grain.

See the API reference for the full field schema, types, defaults, and validation rules.

Value shaping with mode: the default timeseries mode returns one row per time bucket. Set "mode": "scalar" to aggregate the whole window instead: with group_by the response carries one row per dimension combination ordered by value (a top list, capped by limit; flip the order with "sort": "asc"), and without group_by exactly one row (a single aggregate). Scalar rows omit the timestamp field.

Response

Every successful response follows the same envelope. The data array contains time-ordered buckets. Each bucket carries:

a dimensions map: the group_by values for that row (see Dimensions)
a metrics map: shape depends on the metric requested

Scalar metrics return a single named value per bucket. Bundle metrics (such as genai.usage) return a full set of related fields in one call. totals is present only when include_totals: true is set in the request; otherwise the field is omitted. Metric values are rounded to 10 decimal places server-side.

{
  "object": "report",
  "request": { },
  "data": [
    {
      "timestamp": "2026-05-01T00:00:00Z",
      "dimensions": { "model": "openai/gpt-5.4-mini", "provider": "openai" },
      "metrics": { }
    }
  ],
  "totals": {
    "metrics": { }
  },
  "has_more": false,
  "meta": {
    "effective_grain": "day",
    "row_count": 14,
    "request_id": "req_...",
    "currency": "USD"
  }
}

{
  "object": "report",
  "request": {
    "metric": "genai.cost",
    "from": "2026-05-14T00:00:00Z",
    "to":  "2026-05-16T00:00:00Z",
    "grain": "day",
    "group_by": ["model"],
    "include_totals": false,
    "mode": "timeseries",
    "sort": "desc"
  },
  "data": [
    {
      "timestamp": "2026-05-14T00:00:00Z",
      "dimensions": { "model": "openai/gpt-5.4-mini" },
      "metrics": { "genai.cost": 0.000225 }
    }
  ],
  "has_more": false,
  "meta": {
    "effective_grain": "day",
    "row_count": 1,
    "request_id": "req_...",
    "currency": "USD"
  }
}

{
  "object": "report",
  "request": {
    "metric": "genai.usage",
    "from": "2026-05-14T00:00:00Z",
    "to":  "2026-05-15T00:00:00Z",
    "grain": "day",
    "group_by": ["model"],
    "include_totals": false,
    "mode": "timeseries",
    "sort": "desc"
  },
  "data": [
    {
      "timestamp": "2026-05-14T00:00:00Z",
      "dimensions": { "model": "openai/gpt-5.4-mini" },
      "metrics": {
        "request_count": 44,
        "error_count": 0,
        "input_tokens": 660,
        "output_tokens": 660,
        "total_tokens": 1320,
        "cached_input_tokens": 0,
        "reasoning_tokens": 0,
        "audio_input_tokens": 0,
        "audio_output_tokens": 0,
        "total_cost": 0.000495,
        "input_cost": 0.000099,
        "output_cost": 0.000396,
        "cached_cost": 0,
        "reasoning_cost": 0
      }
    }
  ],
  "has_more": false,
  "meta": {
    "effective_grain": "day",
    "row_count": 1,
    "request_id": "req_...",
    "currency": "USD"
  }
}

Metric catalog

Usage
Evaluator
Guardrail

Metric	Description
`genai.requests`	Number of LLM requests
`genai.tokens`	Total tokens (input + output)
`genai.cost`	Total cost in USD
`genai.errors`	Number of failed requests
`genai.error_rate`	Failed / total requests (ratio in `[0, 1]`)
`genai.latency.p50`	Median request latency (ms)
`genai.latency.p95`	95th percentile latency (ms)
`genai.latency.p99`	99th percentile latency (ms)
`genai.ttft.avg`	Average time to first token (ms, streaming only)
`genai.ttft.p50`	Median time to first token (ms, streaming only)
`genai.ttft.p95`	95th percentile time to first token (ms, streaming only)
`genai.usage`	Bundle of request, token, and cost fields

Quantile metrics (genai.latency.p*, genai.ttft.p*) carry a +/-1-2% error band typical of t-digest estimation at high cardinality. genai.ttft.* metrics cover streaming requests only and do not support entity dimensions such as agent or tool.

Metric	Description
`genai.evaluator.runs`	Total Evaluator runs
`genai.evaluator.pass_rate`	Passed / total (ratio in `[0, 1]`)
`genai.evaluator.score.avg`	Average Evaluator score

Metric	Description
`genai.guardrail.runs`	Total guardrail checks
`genai.guardrail.block_rate`	Blocked / total (ratio in `[0, 1]`)
`genai.guardrail.triggered`	Count of blocked checks

Evaluator and guardrail metrics read from the same source: every Evaluator run is recorded as a guardrail check regardless of the Evaluator implementation type. Pick the metric that matches the question being asked.

Dimensions

Entity dimensions (agent, tool, and so on) are available for usage metrics and break a metric down by a product entity.

billing_billable is available as a filter field for usage metrics, but not as a group_by dimension. To break down Orq.ai-managed versus customer-owned credentials, use credential_type instead. The values orq_managed and customer_byok are derived at query time from the workspace billing setup, not stored on each event.

Usage dimensions
Entity dimensions
Evaluator and guardrail dimensions

Pair these with genai.requests, genai.tokens, genai.cost, genai.errors, genai.latency.*, or genai.usage.

Dimension	Notes
`project`	Workspace project the request is attributed to
`identity`	End-user identifier from Identities
`provider`	For example, `openai`, `anthropic`
`model`	For example, `gpt-5.4-mini`
`product`	Which Orq.ai product surface served the request
`api_key`	API key used to authenticate the request
`status_code`	High-level status (`OK`, `ERROR`)
`http_status_code`	HTTP status from the upstream model
`credential_type`	`orq_managed` (Orq.ai-managed keys) or `customer_byok` (customer BYOK keys)

The status_code values map to the OQL and Telemetry API spellings in Trace status values.

Add an entity dimension to any usage metric to break it down by a product entity.

Entity	Description
`agent`	Agent that issued the request
`tool`	Tool invoked during the request
`deployment`	Deployment ID
`evaluator`	Evaluator that ran on the trace
`policy`	Policy that matched the request
`prompt`	Prompt template
`dataset`	Dataset used in an experiment
`conversation`	Conversation grouping
`thread`	Thread grouping
`memory_store`	Memory Store
`knowledge`	Knowledge base

Two conflicting entity dimensions in the same request (for example, ["agent", "tool"]) return 400. Issue one request per entity dimension.

Queries can also group by a raw dimension, but must include a dimension_type filter such as { "field": "dimension_type", "op": "eq", "values": ["agent"] }. Prefer the entity aliases above for most dashboards.

Pair these with genai.evaluator.* and genai.guardrail.* metrics.

Dimension	Notes
`project`	Workspace project the request is attributed to
`identity`	End-user identifier from Identities
`api_key`	API key used to authenticate the request
`policy`	Policy that matched the request
`evaluator`	Evaluator that produced the result
`evaluator_name`	Human-readable Evaluator name
`evaluator_type`	`llm_eval`, `python_eval`, `function_eval`, and so on
`evaluator_version`	Compare versions over time
`result_type`	`boolean`, `number`, `categorical`, `string`
`result_label`	Bounded result bucket (`pass`, `fail`, `error`, or category name)
`evaluation_stage`	`input` or `output` (aliases: `guardrail_stage`, `evaluator_stage`)
`guardrail_action`	`block` or other
`guardrail_origin`	`rule` or `direct`
`product`	Which Orq.ai product surface served the request
`status_code`	High-level status (`OK`, `ERROR`)

Errors

Errors come back as JSON envelopes with a numeric code, a human-readable message, and an empty details array.

{
  "code": 3,
  "message": "time range exceeds maximum: 11952h0m0s > 90d",
  "details": []
}

HTTP	Code	Reason
400	3	Unknown metric, dimension not allowed, range outside workspace retention, range over 90 days, `from >= to`, grain outside retention, or invalid filter operator.
401	16	Missing or invalid bearer token.
403	7	API key does not have access to the requested project.
500	13	Unexpected backend failure. Forward `meta.request_id` to support.

Every successful response includes meta.request_id. Include it in support tickets for fast log correlation.

Limits

Limit	Value
Query window	Workspace retention period, capped at 90 days
`group_by`	5 columns
`filters`	20 entries
`filter.values`	100 values per filter
`limit`	5000 bucket rows (default 1000)
Per-query timeout	30 seconds

Token and Cost Tracking

How token usage is captured and cost is calculated per LLM request.

Overview

Instrument your app

Explore

Monitor

Export

Review

Data privacy & security

Spend dashboards

Performance monitoring

Evaluator quality

Guardrail enforcement

Endpoint

See the API Reference

Request

Response

Metric catalog

Dimensions

Errors

Limits

Token and Cost Tracking

Spend dashboards

Performance monitoring

Evaluator quality

Guardrail enforcement

​Endpoint

See the API Reference

​Request

​Response

​Metric catalog

​Dimensions

​Errors

​Limits

Token and Cost Tracking

Endpoint

Request

Response

Metric catalog

Dimensions

Errors

Limits