Orq.ai Documentation - AI Gateway & LLM Collaboration Platform

The Reporting API exposes detailed visibility into AI activity on Orq.ai. Break down requests, tokens, cost, latency, evaluator outcomes, and guardrail outcomes by model, provider, project, agent, tool, tag, identity, credential type, and more, all with a single JSON request.

Spend dashboards

Track cost by model, provider, project, or credential type. Build per-customer billing breakdowns.

Performance monitoring

Watch p50, p95, p99 latency over time. Catch regressions before users complain.

Evaluator quality

Quantify pass rate, average score, and result distribution per Evaluator and version.

Guardrail enforcement

Measure block rate, triggers, and outcomes by Policy and stage.

Endpoint

POST https://api.orq.ai/v2/reporting

All requests require a Bearer token. Scope follows the API key: workspace and projects are derived from the token itself and are never passed in the body.

Authorization: Bearer ORQ_API_KEY
Content-Type: application/json

The HTTP JSON wire format is snake_case end-to-end (group_by, include_totals, effective_grain, and so on). SDKs expose the same fields using each language’s normal naming style, for example groupBy in TypeScript and group_by in Python.

Quickstart

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.usage",
    "from": "2026-05-01T00:00:00Z",
    "to":   "2026-05-15T00:00:00Z",
    "grain": "day",
    "group_by": ["model"],
    "include_totals": true
  }'

Request

metric

string

required

Metric to query. Choose one of the supported values in the metric catalogue.

from

string (RFC 3339)

required

Inclusive lower bound, in UTC. Example: "2026-05-01T00:00:00Z".

string (RFC 3339)

required

Exclusive upper bound, in UTC.

grain

string

default:"auto"

Bucket size for the time series.

Show Allowed values

Value	Bucket	Retention
`auto`	server picks	n/a
`minute`	1 minute	14 days
`hour`	1 hour	90 days
`day`	1 day	90 days

Auto rule:

range ≤ 6h: minute
range ≤ 7d: hour
otherwise: day

group_by

string[]

Up to 5 dimensions to break the metric down by. See the dimensions reference. Supported dimensions depend on the selected metric.

filters

object[]

Up to 20 predicates, combined with AND. Each filter is { field, op, values }. Supported filter fields depend on the selected metric and match the dimensions listed below unless noted otherwise.

Show Operators

Op	Meaning
`eq`	`field = values[0]` (one value)
`neq`	`field != values[0]` (one value)
`in`	`field IN (values...)`
`not_in`	`field NOT IN (values...)`

Each filter accepts up to 100 values.

include_totals

boolean

default:"false"

When true, the response includes a totals block aggregated across the whole window.

limit

integer

default:"1000"

Maximum bucket rows returned. Capped at 5000.

time_zone

string

default:"UTC"

IANA time zone applied to bucket boundaries (for example, America/New_York). Use it when day-grain dashboards must align to local midnight.

Response

Every successful response follows the same envelope.

object

string

Type discriminator. Always "report".

request

object

The request payload, echoed back. Use it to correlate the response with a caller-side log entry without re-reading the original request body.

data

DataPoint[]

Time-ordered buckets.

Show DataPoint

timestamp

string (RFC 3339)

Bucket start in UTC.

dimensions

object

Map of group-by column to value for this bucket. Empty values are omitted; the map is {} when no group_by was requested.

metrics

object

Map of metric or field name to numeric value. Single-metric requests carry one entry keyed by the requested metric name. Bundle metrics carry one entry per field. Numeric values are rounded to 10 decimal places.

totals

object | null

Aggregated metrics across the whole window. Present only when include_totals was true.

Show Totals

metrics

object

Same shape as data[].metrics.

has_more

boolean

Pagination contract. Currently always false: the server caps responses at limit.

Scalar metric example

A single-aggregate metric returns one entry in metrics, keyed by the requested metric name.

{
  "object": "report",
  "request": {
    "metric": "genai.cost",
    "from": "2026-05-14T00:00:00Z",
    "to": "2026-05-16T00:00:00Z",
    "grain": "day",
    "group_by": ["model"],
    "include_totals": false
  },
  "data": [
    {
      "timestamp": "2026-05-14T00:00:00Z",
      "dimensions": { "model": "gpt-4o-mini" },
      "metrics": { "genai.cost": 0.000225 }
    }
  ],
  "has_more": false,
  "meta": {
    "effective_grain": "day",
    "row_count": 1,
    "request_id": "req_...",
    "currency": "USD"
  }
}

Bundle metric example

The genai.usage bundle returns multi-field rows so spend dashboards stay consistent across request count, tokens, and cost in a single call.

{
  "object": "report",
  "request": {
    "metric": "genai.usage",
    "from": "2026-05-14T00:00:00Z",
    "to": "2026-05-15T00:00:00Z",
    "grain": "day",
    "group_by": ["model"],
    "include_totals": false
  },
  "data": [
    {
      "timestamp": "2026-05-14T00:00:00Z",
      "dimensions": { "model": "gpt-4o-mini" },
      "metrics": {
        "request_count": 44,
        "error_count": 0,
        "input_tokens": 660,
        "output_tokens": 660,
        "total_tokens": 1320,
        "cached_input_tokens": 0,
        "reasoning_tokens": 0,
        "audio_input_tokens": 0,
        "audio_output_tokens": 0,
        "total_cost": 0.000495,
        "input_cost": 0.000099,
        "output_cost": 0.000396,
        "cached_cost": 0,
        "reasoning_cost": 0
      }
    }
  ],
  "has_more": false,
  "meta": {
    "effective_grain": "day",
    "row_count": 1,
    "request_id": "req_...",
    "currency": "USD"
  }
}

Metric catalogue

Usage
Evaluator
Guardrail

Metric	Description
`genai.requests`	Number of LLM requests
`genai.tokens`	Total tokens (input + output)
`genai.cost`	Total cost in USD
`genai.errors`	Number of failed requests
`genai.latency.p50`	Median request latency (ms)
`genai.latency.p95`	95th percentile latency (ms)
`genai.latency.p99`	99th percentile latency (ms)
`genai.usage`	Bundle of request, token, and cost fields

Metric	Description
`genai.evaluator.runs`	Total Evaluator runs
`genai.evaluator.pass_rate`	Passed / total (ratio in `[0, 1]`)
`genai.evaluator.score.avg`	Average Evaluator score

Metric	Description
`genai.guardrail.runs`	Total guardrail checks
`genai.guardrail.block_rate`	Blocked / total (ratio in `[0, 1]`)
`genai.guardrail.triggered`	Count of blocked checks

Evaluator and guardrail metrics read from the same source: every Evaluator run is recorded as a guardrail check regardless of the Evaluator implementation type. Pick the metric that matches the question being asked.

Dimensions

The API picks the right index for the query automatically. Asking for an entity dimension (agent, tool, tag, and so on) on a usage metric routes the query through entity attribution; everything else stays on the core usage path.

billing_billable is available as a filter field for usage metrics, but not as a group_by dimension. Use credential_type when you want a public breakdown of Orq-managed versus customer-owned credentials.

Usage dimensions
Entity dimensions
Evaluator and guardrail dimensions

Pair these with genai.requests, genai.tokens, genai.cost, genai.errors, genai.latency.*, or genai.usage.

Dimension	Notes
`project`
`identity`	End-user identifier from Identities
`provider`	For example, `openai`, `anthropic`
`model`	For example, `gpt-4o-mini`
`product`	Which Orq.ai product surface served the request
`api_key`	API key used to authenticate the request
`status_code`	High-level status (`OK`, `ERROR`)
`http_status_code`	HTTP status from the upstream model
`credential_type`	`orq_managed` or `customer_byok` (BYOK detection)

Add an entity dimension to any usage metric to break it down by a product entity. The API routes the query to attribution storage automatically.

Entity	Description
`agent`	Agent that issued the request
`tool`	Tool invoked during the request
`deployment`	Deployment ID
`evaluator`	Evaluator that ran on the trace
`policy`	Policy that matched the request
`tag`	Tags attached to the request
`prompt`	Prompt template
`dataset`	Dataset used in an experiment
`conversation`	Conversation grouping
`thread`	Thread grouping
`memory_store`	Memory Store
`knowledge`	Knowledge base
`sheet`	Sheet ID

Two conflicting entity dimensions in the same request (for example, ["agent", "tool"]) return 400. Issue one request per entity dimension.

Advanced attribution queries can group by raw dimension, but must include a dimension_type filter such as { "field": "dimension_type", "op": "eq", "values": ["agent"] }. Prefer the entity aliases above for most dashboards.

Pair these with genai.evaluator.* and genai.guardrail.* metrics.

Dimension	Notes
`project`
`identity`
`api_key`
`policy`
`evaluator`	Evaluator that produced the result
`evaluator_name`	Human-readable Evaluator name
`evaluator_type`	`llm_eval`, `python_eval`, `function_eval`, and so on
`evaluator_version`	Compare versions over time
`result_type`	`boolean`, `number`, `categorical`, `string`
`result_label`	Bounded result bucket (`pass`, `fail`, `error`, or category name)
`evaluation_stage`	`input` or `output` (aliases: `guardrail_stage`, `evaluator_stage`)
`guardrail_action`	`block` or other
`guardrail_origin`	`rule` or `direct`
`product`
`status_code`

Examples by use case

Spend: cost by day (bundle)

Get request count, tokens, and cost in one call. Best for headline dashboards.

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.usage",
    "from": "2026-05-01T00:00:00Z",
    "to":   "2026-05-15T00:00:00Z",
    "grain": "day",
    "include_totals": true
  }'

Spend: cost by model per hour

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.cost",
    "from": "2026-05-13T00:00:00Z",
    "to":   "2026-05-14T00:00:00Z",
    "grain": "hour",
    "group_by": ["model"]
  }'

Spend: BYOK vs orq-managed credentials

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.cost",
    "from": "2026-04-15T00:00:00Z",
    "to":   "2026-05-15T00:00:00Z",
    "grain": "day",
    "group_by": ["credential_type"]
  }'

The response carries dimensions.credential_type as "orq_managed" or "customer_byok".

Attribution: cost by Agent

Grouping by an entity auto-routes the query to attribution storage.

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.cost",
    "from": "2026-05-01T00:00:00Z",
    "to":   "2026-05-15T00:00:00Z",
    "grain": "day",
    "group_by": ["agent"]
  }'

Attribution: tokens by tag, filtered by project

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.tokens",
    "from": "2026-04-15T00:00:00Z",
    "to":   "2026-05-15T00:00:00Z",
    "grain": "day",
    "group_by": ["tag"],
    "filters": [
      { "field": "project", "op": "eq", "values": ["proj_123"] }
    ]
  }'

Attribution: cost by Tool with model filter

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.cost",
    "from": "2026-05-01T00:00:00Z",
    "to":   "2026-05-15T00:00:00Z",
    "grain": "day",
    "group_by": ["tool"],
    "filters": [
      { "field": "model", "op": "in", "values": ["openai/gpt-4o", "openai/gpt-4o-mini"] }
    ]
  }'

Latency: p95 by model

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.latency.p95",
    "from": "2026-05-13T00:00:00Z",
    "to":   "2026-05-14T00:00:00Z",
    "grain": "hour",
    "group_by": ["model"]
  }'

metrics["genai.latency.p95"] is reported in milliseconds.

Latency: p50, p95, p99 in parallel

Quantile metrics are independent calls. Fire them concurrently from the client.

for metric in genai.latency.p50 genai.latency.p95 genai.latency.p99; do
  curl -X POST "https://api.orq.ai/v2/reporting" \
    -H "Authorization: Bearer $ORQ_API_KEY" \
    -H "Content-Type: application/json" \
    -d "{\"metric\":\"$metric\",\"from\":\"2026-05-13T00:00:00Z\",\"to\":\"2026-05-14T00:00:00Z\",\"grain\":\"hour\"}" &
done
wait

Errors: count by provider

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.errors",
    "from": "2026-05-13T00:00:00Z",
    "to":   "2026-05-14T00:00:00Z",
    "grain": "hour",
    "group_by": ["provider"]
  }'

Errors: error rate by model (client-side ratio)

Fetch requests and errors in parallel and divide on the client.

curl -X POST "https://api.orq.ai/v2/reporting" -H "Authorization: Bearer $ORQ_API_KEY" -H "Content-Type: application/json" \
  -d '{"metric":"genai.requests","from":"2026-05-13T00:00:00Z","to":"2026-05-14T00:00:00Z","grain":"hour","group_by":["model"]}' &
curl -X POST "https://api.orq.ai/v2/reporting" -H "Authorization: Bearer $ORQ_API_KEY" -H "Content-Type: application/json" \
  -d '{"metric":"genai.errors","from":"2026-05-13T00:00:00Z","to":"2026-05-14T00:00:00Z","grain":"hour","group_by":["model"]}' &
wait

Evaluator: pass rate by Evaluator and version

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.evaluator.pass_rate",
    "from": "2026-05-01T00:00:00Z",
    "to":   "2026-05-15T00:00:00Z",
    "grain": "day",
    "group_by": ["evaluator", "evaluator_version"]
  }'

Evaluator: average score

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.evaluator.score.avg",
    "from": "2026-05-01T00:00:00Z",
    "to":   "2026-05-15T00:00:00Z",
    "grain": "day",
    "group_by": ["evaluator"]
  }'

Evaluator: runs by result label

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.evaluator.runs",
    "from": "2026-05-01T00:00:00Z",
    "to":   "2026-05-15T00:00:00Z",
    "grain": "day",
    "group_by": ["evaluator", "result_label"]
  }'

Guardrail: block rate by Evaluator and stage

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.guardrail.block_rate",
    "from": "2026-05-01T00:00:00Z",
    "to":   "2026-05-15T00:00:00Z",
    "grain": "day",
    "group_by": ["evaluator", "guardrail_stage"]
  }'

Guardrail: triggers by Policy and stage

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.guardrail.triggered",
    "from": "2026-05-01T00:00:00Z",
    "to":   "2026-05-15T00:00:00Z",
    "grain": "day",
    "group_by": ["policy", "guardrail_stage"]
  }'

Guardrail: filter by origin

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.guardrail.runs",
    "from": "2026-05-01T00:00:00Z",
    "to":   "2026-05-15T00:00:00Z",
    "grain": "day",
    "group_by": ["evaluator"],
    "filters": [
      { "field": "guardrail_origin", "op": "eq", "values": ["rule"] }
    ]
  }'

Multi-tenant: cost per project per day

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.cost",
    "from": "2026-05-01T00:00:00Z",
    "to":   "2026-05-15T00:00:00Z",
    "grain": "day",
    "group_by": ["project"]
  }'

Multi-tenant: usage per API key

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.usage",
    "from": "2026-05-01T00:00:00Z",
    "to":   "2026-05-15T00:00:00Z",
    "grain": "day",
    "group_by": ["api_key"],
    "include_totals": true
  }'

Identity: cost per end-user Identity

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.cost",
    "from": "2026-05-01T00:00:00Z",
    "to":   "2026-05-15T00:00:00Z",
    "grain": "day",
    "group_by": ["identity"]
  }'

Identity: requests filtered to a single end-user

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.requests",
    "from": "2026-05-01T00:00:00Z",
    "to":   "2026-05-15T00:00:00Z",
    "grain": "day",
    "filters": [
      { "field": "identity", "op": "eq", "values": ["user_42"] }
    ]
  }'

Time zone: day buckets in America/New_York

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.cost",
    "from": "2026-05-01T04:00:00Z",
    "to":   "2026-05-15T04:00:00Z",
    "grain": "day",
    "time_zone": "America/New_York"
  }'

Bucket boundaries align to midnight in the requested zone.

High resolution: requests per minute, last hour

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.requests",
    "from": "2026-05-14T11:00:00Z",
    "to":   "2026-05-14T12:00:00Z",
    "grain": "minute"
  }'

Cross-dimension: cost by credential type and model

curl -X POST "https://api.orq.ai/v2/reporting" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metric": "genai.cost",
    "from": "2026-05-01T00:00:00Z",
    "to":   "2026-05-15T00:00:00Z",
    "grain": "day",
    "group_by": ["credential_type", "model"]
  }'

Errors

Errors come back as JSON envelopes with a numeric code, a human-readable message, and an empty details array.

{
  "code": 3,
  "message": "time range exceeds maximum: 11952h0m0s > 90d",
  "details": []
}

HTTP	Code	Reason
400	3	Unknown metric, dimension not allowed, range outside workspace retention, range over 90 days, `from >= to`, grain outside retention, or invalid filter operator.
401	16	Missing or invalid bearer token.
403	7	API key does not have access to the requested project.
500	13	Unexpected backend failure. Forward `meta.request_id` to support.

Every successful response includes meta.request_id. Include it in support tickets for fast log correlation.

Limits

Limit	Value
Query window	Workspace retention period, capped at 90 days
`group_by`	5 columns
`filters`	20 entries
`filter.values`	100 values per filter
`limit`	5000 bucket rows (default 1000)
Per-query timeout	30 seconds

Notes

Trace events flow into reporting in near real time, with a small ingestion delay measured in seconds. Very recent data may take a moment to appear.

Quantile metrics (genai.latency.p*) carry a +/-1-2% error band typical of t-digest estimation at high cardinality.

Numeric responses are rounded to 10 decimal places server-side so dashboards never display floating-point noise like 0.00009900000000000001.

credential_type is a public alias derived from billing setup at query time: orq_managed when Orq.ai keys served the traffic, customer_byok when customer-owned keys did.

Get Started

AI Router

Observability

AI Engineering

Optimize

Manage Prompts

Reporting

Spend dashboards

Performance monitoring

Evaluator quality

Guardrail enforcement

Endpoint

Quickstart

Request

Response

Scalar metric example

Bundle metric example

Metric catalogue

Dimensions

Examples by use case

Errors

Limits

Notes

Get Started

AI Router

Observability

AI Engineering

Optimize

Manage Prompts

Documentation Index

Spend dashboards

Performance monitoring

Evaluator quality

Guardrail enforcement

​Endpoint

​Quickstart

​Request

​Response

​Scalar metric example

​Bundle metric example

​Metric catalogue

​Dimensions

​Examples by use case

​Errors

​Limits

​Notes

Endpoint

Quickstart

Request

Response

Scalar metric example

Bundle metric example

Metric catalogue

Dimensions

Examples by use case

Errors

Limits

Notes