Skip to main content
POST
/
v2
/
evaluators
/
{id}
/
invoke
Invoke a Custom Evaluator
curl --request POST \
  --url https://api.orq.ai/v2/evaluators/{id}/invoke \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "query": "<string>",
  "output": "<string>",
  "reference": "<string>",
  "retrievals": [
    "<string>"
  ],
  "messages": [
    {
      "content": "<string>",
      "tool_calls": [
        {
          "type": "function",
          "function": {
            "name": "<string>",
            "arguments": "<string>"
          },
          "id": "<string>",
          "index": 123
        }
      ],
      "tool_call_id": "<string>"
    }
  ],
  "model": "<string>"
}
'
{
  "type": "string",
  "original_value": "<string>",
  "value": "<string>"
}

Authorizations

​
Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Path Parameters

​
id
string
required

Evaluator ID

Body

application/json
​
query
string

Latest user message

​
output
string

The generated response from the model

​
reference
string

The reference used to compare the output

​
retrievals
string[]

Knowledge base retrievals

​
messages
object[]

The messages used to generate the output, without the last user message

​
model
string

Model to use for LLM-based evaluators (e.g. "openai/gpt-4o")

Response

Returns the result of the evaluator run

​
type
enum<string>
required
Available options:
string
​
original_value
string | null
​
value
string | null