Get a run

Core - Get a run

curl 'https://my.orq.ai/v3/experiments/<experiment_id>/runs/<run_id>' \
  --header 'Authorization: Bearer $ORQ_API_KEY'

import { Orq } from '@orq-ai/node';

const client = new Orq({ apiKey: process.env.ORQ_API_KEY });

const run = await client.experiments.getRun({ experimentId: '<experiment_id>', runId: '<run_id>' });

console.log(run.status, run.cellStatusCounts?.completed);

import os
from orq_ai_sdk import Orq

client = Orq(api_key=os.environ["ORQ_API_KEY"])

run = client.experiments.get_run(experiment_id="<experiment_id>", run_id="<run_id>")

print(run.status, run.cell_status_counts.completed)

{
  "run_id": "<string>",
  "experiment_id": "<string>",
  "key": "<string>",
  "status": "RUN_STATUS_UNSPECIFIED",
  "dataset_id": "<string>",
  "row_count": 123,
  "cell_status_counts": {
    "not_run": 123,
    "queued": 123,
    "running": 123,
    "completed": 123,
    "failed": 123,
    "canceled": 123
  },
  "columns": [
    {
      "column_id": "<string>",
      "key": "<string>",
      "display_name": "<string>",
      "position": 123,
      "active": true,
      "prompt": {
        "prompt_config": {},
        "messages_source": "MESSAGES_SOURCE_UNSPECIFIED",
        "engine": "<string>",
        "tools": [
          {}
        ],
        "knowledge_bases": [
          {}
        ],
        "metadata": {}
      },
      "agent": {
        "agent_key": "<string>",
        "messages_source": "MESSAGES_SOURCE_UNSPECIFIED",
        "messages": [
          {}
        ],
        "engine": "<string>",
        "parent_id": "<string>",
        "agent_tag": "<string>"
      },
      "openresponse": {},
      "evaluator": {
        "evaluator_id": "<string>",
        "evaluator_name": "<string>",
        "evaluator_type": "<string>",
        "evaluator_output_type": "<string>"
      },
      "review": {
        "human_eval_id": "<string>",
        "human_eval_name": "<string>",
        "human_eval_type": "<string>"
      },
      "dataset": {
        "column_type": "<string>"
      },
      "metric": {
        "metric_key": "<string>"
      },
      "job_output": {
        "job_name": "<string>"
      },
      "width": 123
    }
  ],
  "settings": {
    "concurrency": 123,
    "timeout": "<string>",
    "max_turns": 123,
    "hidden_column_ids": [
      "<string>"
    ]
  },
  "started_at": "2023-11-07T05:31:56Z",
  "completed_at": "2023-11-07T05:31:56Z",
  "created_at": "2023-11-07T05:31:56Z",
  "updated_at": "2023-11-07T05:31:56Z",
  "metrics": {},
  "stats": {
    "total_cost": 123,
    "total_errors": 123,
    "total_done": 123
  },
  "created_by_id": "<string>"
}

{
  "message": "<string>",
  "code": 123,
  "details": [
    "<unknown>"
  ]
}

GET

experiments

{experiment_id}

runs

{run_id}

Core - Get a run

curl 'https://my.orq.ai/v3/experiments/<experiment_id>/runs/<run_id>' \
  --header 'Authorization: Bearer $ORQ_API_KEY'

import { Orq } from '@orq-ai/node';

const client = new Orq({ apiKey: process.env.ORQ_API_KEY });

const run = await client.experiments.getRun({ experimentId: '<experiment_id>', runId: '<run_id>' });

console.log(run.status, run.cellStatusCounts?.completed);

import os
from orq_ai_sdk import Orq

client = Orq(api_key=os.environ["ORQ_API_KEY"])

run = client.experiments.get_run(experiment_id="<experiment_id>", run_id="<run_id>")

print(run.status, run.cell_status_counts.completed)

{
  "run_id": "<string>",
  "experiment_id": "<string>",
  "key": "<string>",
  "status": "RUN_STATUS_UNSPECIFIED",
  "dataset_id": "<string>",
  "row_count": 123,
  "cell_status_counts": {
    "not_run": 123,
    "queued": 123,
    "running": 123,
    "completed": 123,
    "failed": 123,
    "canceled": 123
  },
  "columns": [
    {
      "column_id": "<string>",
      "key": "<string>",
      "display_name": "<string>",
      "position": 123,
      "active": true,
      "prompt": {
        "prompt_config": {},
        "messages_source": "MESSAGES_SOURCE_UNSPECIFIED",
        "engine": "<string>",
        "tools": [
          {}
        ],
        "knowledge_bases": [
          {}
        ],
        "metadata": {}
      },
      "agent": {
        "agent_key": "<string>",
        "messages_source": "MESSAGES_SOURCE_UNSPECIFIED",
        "messages": [
          {}
        ],
        "engine": "<string>",
        "parent_id": "<string>",
        "agent_tag": "<string>"
      },
      "openresponse": {},
      "evaluator": {
        "evaluator_id": "<string>",
        "evaluator_name": "<string>",
        "evaluator_type": "<string>",
        "evaluator_output_type": "<string>"
      },
      "review": {
        "human_eval_id": "<string>",
        "human_eval_name": "<string>",
        "human_eval_type": "<string>"
      },
      "dataset": {
        "column_type": "<string>"
      },
      "metric": {
        "metric_key": "<string>"
      },
      "job_output": {
        "job_name": "<string>"
      },
      "width": 123
    }
  ],
  "settings": {
    "concurrency": 123,
    "timeout": "<string>",
    "max_turns": 123,
    "hidden_column_ids": [
      "<string>"
    ]
  },
  "started_at": "2023-11-07T05:31:56Z",
  "completed_at": "2023-11-07T05:31:56Z",
  "created_at": "2023-11-07T05:31:56Z",
  "updated_at": "2023-11-07T05:31:56Z",
  "metrics": {},
  "stats": {
    "total_cost": 123,
    "total_errors": 123,
    "total_done": 123
  },
  "created_by_id": "<string>"
}

{
  "message": "<string>",
  "code": 123,
  "details": [
    "<unknown>"
  ]
}

Related guide: Experiments guide. See the Experiments guide for a walkthrough with examples.

Authorizations

Authorization

string

header

required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Path Parameters

experiment_id

string

required

ID of the experiment.

run_id

string

required

ID of the run.

Response

A run: one pass of the experiment's columns over a copy of its dataset rows.

run_id

string

Run ID (ULID).

experiment_id

string

Experiment the run belongs to.

key

string

Run name. Run 1 for the first run; Run plus the last four characters of the run ID when a run is created without a key.

status

enum<string>

Run state. RUN_STATUS_DRAFT: created, never executed. RUN_STATUS_RUNNING: executing. RUN_STATUS_COMPLETED: no rows left to execute, although individual cells may have failed. RUN_STATUS_FAILED: execution stopped on an internal error. RUN_STATUS_CANCELED: stopped by Cancel a run. RUN_STATUS_QUEUED counts as active, like running.

Available options:

RUN_STATUS_UNSPECIFIED,

RUN_STATUS_DRAFT,

RUN_STATUS_QUEUED,

RUN_STATUS_RUNNING,

RUN_STATUS_COMPLETED,

RUN_STATUS_FAILED,

RUN_STATUS_CANCELED

dataset_id

string

Dataset the rows were copied from, taken from the experiment when the run was created. Empty for evaluatorq uploads.

row_count

integer<int32>

Number of rows in the run.

cell_status_counts

object

Number of cells in each status.

Show child attributes

columns

object[]

Columns of the run, hidden ones included with active false. Get, create and update order them as dataset, task, job output, metric, evaluator and review columns.

Show child attributes

settings

object | null

Null on runs created without settings, such as evaluatorq uploads.

Show child attributes

started_at

string<date-time> | null

Time the latest execution started. Null until the run is executed.

completed_at

string<date-time> | null

Time the run completed or was canceled. Null until then, and kept when the run is executed again.

created_at

string<date-time>

Creation time.

updated_at

string<date-time>

Time of the last change to the run.

metrics

object

Average evaluator results per task column, keyed {task_column_id}_{evaluator_id} (the run ID replaces the task column ID for run-level values). Only returned by List runs with include_metrics; categorical evaluators are left out.

Show child attributes

stats

object

Execution totals of the run.

Show child attributes

created_by_id

string

ID of the user or API key that created the run.

Orq SDK

Terminal & Agents

Terraform

Platform APIs

Management APIs

Authorizations

Path Parameters

Response