Update an Evaluator

curl --request PATCH \
  --url https://api.orq.ai/v2/evaluators/{id} \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "type": "<string>",
  "path": "Default",
  "project_id": "01JMDPA3QW5C1V0NJ1PW34T4E5",
  "key": "<string>",
  "description": "<string>",
  "prompt": "<string>",
  "output_type": "<string>",
  "categories": [
    "<string>"
  ],
  "categorical_labels": [
    {
      "value": "<string>",
      "description": "<string>"
    }
  ],
  "dataset_id": "<string>",
  "repetitions": 123,
  "model": "<string>",
  "schema": "<string>",
  "url": "<string>",
  "method": "<string>",
  "headers": {},
  "payload": {},
  "code": "<string>",
  "guardrail_config": {
    "type": "boolean",
    "value": true,
    "enabled": true,
    "alert_on_failure": false
  },
  "versionDescription": "<string>"
}
'

const options = {
  method: 'PATCH',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({
    type: '<string>',
    path: 'Default',
    project_id: '01JMDPA3QW5C1V0NJ1PW34T4E5',
    key: '<string>',
    description: '<string>',
    prompt: '<string>',
    output_type: '<string>',
    categories: ['<string>'],
    categorical_labels: [{value: '<string>', description: '<string>'}],
    dataset_id: '<string>',
    repetitions: 123,
    model: '<string>',
    schema: '<string>',
    url: '<string>',
    method: '<string>',
    headers: {},
    payload: {},
    code: '<string>',
    guardrail_config: {type: 'boolean', value: true, enabled: true, alert_on_failure: false},
    versionDescription: '<string>'
  })
};

fetch('https://api.orq.ai/v2/evaluators/{id}', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

import requests

url = "https://api.orq.ai/v2/evaluators/{id}"

payload = {
    "type": "<string>",
    "path": "Default",
    "project_id": "01JMDPA3QW5C1V0NJ1PW34T4E5",
    "key": "<string>",
    "description": "<string>",
    "prompt": "<string>",
    "output_type": "<string>",
    "categories": ["<string>"],
    "categorical_labels": [
        {
            "value": "<string>",
            "description": "<string>"
        }
    ],
    "dataset_id": "<string>",
    "repetitions": 123,
    "model": "<string>",
    "schema": "<string>",
    "url": "<string>",
    "method": "<string>",
    "headers": {},
    "payload": {},
    "code": "<string>",
    "guardrail_config": {
        "type": "boolean",
        "value": True,
        "enabled": True,
        "alert_on_failure": False
    },
    "versionDescription": "<string>"
}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.patch(url, json=payload, headers=headers)

print(response.text)

{
  "_id": "<string>",
  "description": "<string>",
  "type": "llm_eval",
  "prompt": "<string>",
  "key": "<string>",
  "created": "2026-08-13T12:46:10.298Z",
  "updated": "2026-08-13T12:46:10.298Z",
  "updated_by_id": "<string>",
  "project_id": "<string>",
  "guardrail_config": {
    "type": "boolean",
    "value": true,
    "enabled": true,
    "alert_on_failure": false
  },
  "repetitions": 2,
  "categories": [
    "<string>"
  ],
  "categorical_labels": [
    {
      "value": "<string>",
      "description": "<string>"
    }
  ],
  "dataset_id": "<string>",
  "model": "<string>",
  "jury": {
    "judges": [
      {
        "model": "<string>",
        "retry": {
          "count": 2,
          "on_codes": [
            429,
            500,
            502,
            503,
            504
          ]
        },
        "fallbacks": []
      }
    ],
    "replacement_judges": [],
    "min_successful_judges": 2,
    "tie_value": "Tie"
  }
}

{
  "message": "<string>"
}

PATCH

evaluators

{id}

Update an Evaluator

curl --request PATCH \
  --url https://api.orq.ai/v2/evaluators/{id} \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "type": "<string>",
  "path": "Default",
  "project_id": "01JMDPA3QW5C1V0NJ1PW34T4E5",
  "key": "<string>",
  "description": "<string>",
  "prompt": "<string>",
  "output_type": "<string>",
  "categories": [
    "<string>"
  ],
  "categorical_labels": [
    {
      "value": "<string>",
      "description": "<string>"
    }
  ],
  "dataset_id": "<string>",
  "repetitions": 123,
  "model": "<string>",
  "schema": "<string>",
  "url": "<string>",
  "method": "<string>",
  "headers": {},
  "payload": {},
  "code": "<string>",
  "guardrail_config": {
    "type": "boolean",
    "value": true,
    "enabled": true,
    "alert_on_failure": false
  },
  "versionDescription": "<string>"
}
'

const options = {
  method: 'PATCH',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({
    type: '<string>',
    path: 'Default',
    project_id: '01JMDPA3QW5C1V0NJ1PW34T4E5',
    key: '<string>',
    description: '<string>',
    prompt: '<string>',
    output_type: '<string>',
    categories: ['<string>'],
    categorical_labels: [{value: '<string>', description: '<string>'}],
    dataset_id: '<string>',
    repetitions: 123,
    model: '<string>',
    schema: '<string>',
    url: '<string>',
    method: '<string>',
    headers: {},
    payload: {},
    code: '<string>',
    guardrail_config: {type: 'boolean', value: true, enabled: true, alert_on_failure: false},
    versionDescription: '<string>'
  })
};

fetch('https://api.orq.ai/v2/evaluators/{id}', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

import requests

url = "https://api.orq.ai/v2/evaluators/{id}"

payload = {
    "type": "<string>",
    "path": "Default",
    "project_id": "01JMDPA3QW5C1V0NJ1PW34T4E5",
    "key": "<string>",
    "description": "<string>",
    "prompt": "<string>",
    "output_type": "<string>",
    "categories": ["<string>"],
    "categorical_labels": [
        {
            "value": "<string>",
            "description": "<string>"
        }
    ],
    "dataset_id": "<string>",
    "repetitions": 123,
    "model": "<string>",
    "schema": "<string>",
    "url": "<string>",
    "method": "<string>",
    "headers": {},
    "payload": {},
    "code": "<string>",
    "guardrail_config": {
        "type": "boolean",
        "value": True,
        "enabled": True,
        "alert_on_failure": False
    },
    "versionDescription": "<string>"
}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.patch(url, json=payload, headers=headers)

print(response.text)

{
  "_id": "<string>",
  "description": "<string>",
  "type": "llm_eval",
  "prompt": "<string>",
  "key": "<string>",
  "created": "2026-08-13T12:46:10.298Z",
  "updated": "2026-08-13T12:46:10.298Z",
  "updated_by_id": "<string>",
  "project_id": "<string>",
  "guardrail_config": {
    "type": "boolean",
    "value": true,
    "enabled": true,
    "alert_on_failure": false
  },
  "repetitions": 2,
  "categories": [
    "<string>"
  ],
  "categorical_labels": [
    {
      "value": "<string>",
      "description": "<string>"
    }
  ],
  "dataset_id": "<string>",
  "model": "<string>",
  "jury": {
    "judges": [
      {
        "model": "<string>",
        "retry": {
          "count": 2,
          "on_codes": [
            429,
            500,
            502,
            503,
            504
          ]
        },
        "fallbacks": []
      }
    ],
    "replacement_judges": [],
    "min_successful_judges": 2,
    "tie_value": "Tie"
  }
}

{
  "message": "<string>"
}

Authorizations

Authorization

string

header

required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Path Parameters

string

required

Body

application/json

type

string

Evaluator type. Optional on update — inferred from existing evaluator.

path

string

Legacy alternative to project_id. Project path. Optional on update — the evaluator keeps its current project when both are omitted. Mutually exclusive with project_id.

Example:

"Default"

project_id

string

Unique identifier of the project that owns the evaluator, as returned by GET /v2/projects. Optional on update — the evaluator keeps its current project when omitted; supplying a different id moves it. Mutually exclusive with path.

Minimum string length: 1

Example:

"01JMDPA3QW5C1V0NJ1PW34T4E5"

key

string

Minimum string length: 1

Pattern: ^[a-zA-Z0-9]([a-zA-Z0-9_-]*[a-zA-Z0-9])?$

description

string

prompt

string

output_type

string

Response

Successfully updated an eval

_id

string

required

description

string

required

type

enum<string>

required

Available options:

llm_eval

prompt

string

required

key

string

required

Minimum string length: 1

Pattern: ^[a-zA-Z0-9]([a-zA-Z0-9_-]*[a-zA-Z0-9])?$

mode

enum<string>

required

Available options:

single,

jury

created

string

default:2026-08-13T12:46:10.298Z

updated

string

default:2026-08-13T12:46:10.298Z

updated_by_id

string | null

project_id

string

Unique identifier of the project owning this evaluator.

guardrail_config

Boolean · object

Boolean
Categorical
Number

Show child attributes

repetitions

integer | null

Required range: 1 <= x <= 3

Orq SDK

Terminal & Agents

Terraform

Platform APIs

Management APIs

Update an Evaluator

Authorizations

Path Parameters

Body

Response