Skip to main content

​
Prerequisites

Before using the Annotations API, ensure you have:
  • Orq.ai API Key: Get your API key from the AI Studio
  • Trace ID and Span ID: From your observability data (deployment invocations, trace queries, or OpenTelemetry instrumentation) or AI Studio
  • Human Reviews: Human Review definitions must exist in the project for the annotation keys you want to use. Each annotation key must match an existing Human Review key. Create Human Reviews in Project Settings > Human Review.

​
API Constraints

  • Batch Limits: Up to 10 annotations per create request, up to 10 keys per delete request
  • String Length: String values are limited to 200 characters maximum
  • Deployment Span Propagation: When annotating a deployment span, the associated log is automatically annotated with the same values
  • Metadata Fields: Optional metadata object supports identityId, source, and reviewerId for tracking and attribution

​
Examples

​
Add a Quality Rating

curl -X POST "https://api.orq.ai/v2/traces/{trace_id}/spans/{span_id}/annotation" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "annotations": [
      {
        "key": "rating",
        "value": "good"
      }
    ]
  }'

​
Add Multiple Defects

curl -X POST "https://api.orq.ai/v2/traces/{trace_id}/spans/{span_id}/annotation" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "annotations": [
      {
        "key": "defects",
        "value": ["grammatical", "spelling", "ambiguity"]
      }
    ]
  }'

​
Add a Numeric Score

curl -X POST "https://api.orq.ai/v2/traces/{trace_id}/spans/{span_id}/annotation" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "annotations": [
      {
        "key": "confidence_score",
        "value": 0.95
      }
    ],
    "metadata": {
      "identityId": "user-123"
    }
  }'

​
Add a Text Correction

curl -X POST "https://api.orq.ai/v2/traces/{trace_id}/spans/{span_id}/annotation" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "annotations": [
      {
        "key": "correction",
        "value": "The correct answer should emphasize scalability and fault tolerance."
      }
    ]
  }'

​
Batch Add Multiple Annotations

curl -X POST "https://api.orq.ai/v2/traces/{trace_id}/spans/{span_id}/annotation" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "annotations": [
      {
        "key": "rating",
        "value": "good"
      },
      {
        "key": "confidence_score",
        "value": 0.92
      },
      {
        "key": "categories",
        "value": ["helpful", "accurate", "concise"]
      }
    ]
  }'

​
Remove Annotations

curl -X DELETE "https://api.orq.ai/v2/traces/{trace_id}/spans/{span_id}/annotation" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "keys": ["rating", "defects"]
  }'

​
Error Handling

Status CodeErrorExample MessageSolution
404Human Review Not FoundThe human review with key "rating" for workspace abc123 was not found.Create a Human Review with the specified key before annotating.
404Span Not FoundSpan with id xyz789 for workspace abc123 was not found.Verify the trace_id and span_id are correct and belong to your workspace.
400Invalid ValueInvalid value: poor. Valid options are: good, bad.Ensure the value matches the options defined in the Human Review.
400Value Out of RangeValue 15 is out of range [0, 10].Provide a number within the defined min/max range for the Human Review.
400String Too LongString value exceeds maximum length of 200 characters.Shorten the string annotation to 200 characters or less.