HomepageAdmin PortalSign up
Resources

Feedback via API

Log feedback directly via the API

Suggest Edits

On top of using Feedback in Orq you can choose to integrate feedback within your applications and services. You can rely on the public API and the Orq.ai SDKs to do so. You can automate feedback and corrections directly through those methods.

Prerequisites

Authenticating using the API and SDKs

To get started with submitting feedbacks and corrections, you must first successfully invoke a Deployment and receive a response. To do so, some of your prerequisites are to authenticate with the API and SDKs.

Invoking a Deployment

Once authenticated successfully against the API, you need to be able to invoke a deployment, this will generate a response you can provide feedback and corrections on.

Once the deployment is invoked, you need to store the trace_id that will be returned from the invocation.

Feedback

Once you have the trace_id of the response you want to give feedback to, you can prepare your feedback payload.

There are multiple types of feedback and values available for you to use.

📘

To learn about the feedback types, see Feedback types.

Here are examples of payloads you can prepare for each feedback type, in the following payloads, replace unique-trace-id with the previously acquired trace_id.

Rating

Issuing a good rating on a response.

client.feedback.report(
    property="rating",
    value=["good"],
    trace_id="unique-trace-id"
)

const reportPayload: FeedbackReport = {
  property: 'rating',
  values: ['good'],
  trace_id: 'unique-trace-id',
};

try {
  const response = await client.feedback.report(reportPayload);
  console.log('Feedback report sent:', response);
} catch (error) {
  console.error('Error sending feedback report:', error);
}

curl --request POST \
     --url https://api.orq.ai/v2/feedback \
     --header 'accept: application/json' \
     --header 'authorization: Bearer ACCESS_TOKEN' \
     --header 'content-type: application/json' \
     --data '
{
  "property": "rating",
  "trace_id": "unique-trace-id",
  "value": "good"
}
'

Defects

Issuing a grammatical and hallucination defect feedback on a response.

client.feedback.report(
    property="defects",
    value=["grammatical", "hallucination"],  # Can include multiple defects
    trace_id="unique-trace-id"
)

const reportPayload: FeedbackReport = {
  property: 'defects',
  values: ['grammatical', 'hallucination'],
  trace_id: 'unique-trace-id',
};

try {
  const response = await client.feedback.report(reportPayload);
  console.log('Feedback report sent:', response);
} catch (error) {
  console.error('Error sending feedback report:', error);
}

curl --request POST \
     --url https://api.orq.ai/v2/feedback \
     --header 'accept: application/json' \
     --header 'authorization: Bearer ACCESS_TOKEN' \
     --header 'content-type: application/json' \
     --data '
{
  "property": "defects",
  "trace_id": "unique-trace-id",
  "value": ["grammatical", "hallucination"]
}
'

Interactions

Issuing a saved and copied interaction feedback on a response.

client.feedback.report(
    property="interactions",
    value=["saved", "copied"],  # Can include multiple interactions
    trace_id="unique-trace-id"
)

const reportPayload: FeedbackReport = {
  property: 'interactions',
  values: ['saved', 'copied'],
  trace_id: 'unique-trace-id',
};

try {
  const response = await client.feedback.report(reportPayload);
  console.log('Feedback report sent:', response);
} catch (error) {
  console.error('Error sending feedback report:', error);
}

curl --request POST \
     --url https://api.orq.ai/v2/feedback \
     --header 'accept: application/json' \
     --header 'authorization: Bearer ACCESS_TOKEN' \
     --header 'content-type: application/json' \
     --data '
{
  "property": "interactions",
  "trace_id": "unique-trace-id",
  "value": ["saved", "copied"]
}
'

Corrections

In the same way you can send feedback, you can issue a correction with a new text on a selected response. This is useful when creating curated Datasets.

To send a correction, you also need the previously fetched trace-id.

client.feedback.correct(
    value="This is the corrected text for the AI-generated content.",
    trace_id="unique-trace-id"
)

const correctionPayload: FeedbackCorrection = {
  correction: 'This is the corrected text for the AI-generated content.',
  trace_id: 'unique-trace-id',
};

try {
  const response = await client.feedback.correct(correctionPayload);
  console.log('Correction sent:', response);
} catch (error) {
  console.error('Error sending correction:', error);
}

Updated about 1 month ago