Skip to main content
POST
/
v2
/
evaluators
/
{id}
/
invoke
Invoke a Custom Evaluator
curl --request POST \
  --url https://api.orq.ai/v2/evaluators/{id}/invoke \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "query": "<string>",
  "output": "<string>",
  "reference": "<string>",
  "retrievals": [
    "<string>"
  ],
  "messages": [
    {
      "role": "system",
      "content": "<string>",
      "tool_calls": [
        {
          "type": "function",
          "function": {
            "name": "<string>",
            "arguments": "<string>"
          },
          "id": "<string>",
          "index": 123
        }
      ],
      "tool_call_id": "<string>"
    }
  ]
}
'
{
  "type": "string",
  "original_value": "<string>",
  "value": "<string>"
}

Authorizations

Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Path Parameters

id
string
required

Evaluator ID Evaluator ID

Body

application/json
query
string

Latest user message

output
string

The generated response from the model

reference
string

The reference used to compare the output

retrievals
string[]

Knowledge base retrievals

messages
object[]

The messages used to generate the output, without the last user message

Response

Returns the result of the evaluator run

  • String
  • Number
  • Boolean
  • String Array
  • Rouge N
  • BERT Score
  • LLM
  • HTTP
type
enum<string>
required
Available options:
string
original_value
string | null
value
string | null