Adding Feedback Programmatically

Feedbacks can be issues via the Feedback API and using the Python and Node.js SDK.

Fetching Trace ID

The main parameter needed is a Trace ID. This corresponds to the ID returned by the Get Config or Invoke Deployment calls.

Creating a Feedback

There are multiple Properties usable to create feedback.

Property	Values	type
`rating`	`good`or `bad`	array
`defects`	`grammatical` `spelling` `hallucination` `repetition` `innapropriate` `off_topic` `incompleteness` `ambiguity`	array
`interactions`	`saved` `selected` `deleted` `shared` `copied` `reported`	array
`correction`	Correction Text	string

Examples

Issue a bad rating:

{
  "property": "rating",
  "trace_id": "<trace_id>",
  "value": [
    "bad"
  ]
}

from orq_ai_sdk import Orq
import os


with Orq(
    api_key=os.getenv("ORQ_API_KEY", ""),
) as orq:

    res = orq.feedback.create(field="rating", value=[
        "bad",
    ], trace_id="<trace id>")

    assert res is not None

    # Handle response
    print(res)

import { Orq } from "@orq-ai/node";

const orq = new Orq({
  apiKey: process.env["ORQ_API_KEY"] ?? "",
});

async function run() {
  const result = await orq.feedback.create({
    field: "rating",
    value: [
      "bad",
    ],
    traceId: "<trace id>",
  });

  // Handle the result
  console.log(result);
}

run();

Issue grammatical and ambiguity defects:

{
  "property": "defects",
  "trace_id": "<trace_id>",
  "value": [
    "grammatical",
    "ambiguity"
  ]
}

from orq_ai_sdk import Orq
import os


with Orq(
    api_key=os.getenv("ORQ_API_KEY", ""),
) as orq:

    res = orq.feedback.create(field="defects", value=[
        "grammatical",
        "ambiguity"
    ], trace_id="<trace id>")

    assert res is not None

    # Handle response
    print(res)

import { Orq } from "@orq-ai/node";

const orq = new Orq({
  apiKey: process.env["ORQ_API_KEY"] ?? "",
});

async function run() {
  const result = await orq.feedback.create({
    field: "defects",
    value: [
      "grammatical",
      "ambiguity"
    ],
    traceId: "<trace id>",
  });

  // Handle the result
  console.log(result);
}

run();

Making a correction

You can use the same endpoint/SDK call to issue a correction on a generation.

{
  "property": "corrections",
  "trace_id": "<trace_id>",
  "value": "Correction Text"
}

from orq_ai_sdk import Orq
import os


with Orq(
    api_key=os.getenv("ORQ_API_KEY", ""),
) as orq:

    res = orq.feedback.create(field="correction", 
                              value="Correction Text", 
                              trace_id="<trace id>")

    assert res is not None

    # Handle response
    print(res)

import { Orq } from "@orq-ai/node";

const orq = new Orq({
  apiKey: process.env["ORQ_API_KEY"] ?? "",
});

async function run() {
  const result = await orq.feedback.create({
    field: "correction",
    value: "Correction Text",
    traceId: "<trace id>",
  });

  // Handle the result
  console.log(result);
}

run();