Skip to main content

​
Prerequisites

Before using the Annotations API, ensure you have:
  • Orq.ai API Key: Get your API key from the AI Studio
  • Trace ID and Span ID: From your observability data (deployment invocations, trace queries, or OpenTelemetry instrumentation) or AI Studio
  • Human Reviews: Create Human Review definitions that define the annotation keys you want to use
Important: You must create Human Reviews before adding annotations. Each annotation key must match an existing Human Review key in your workspace. Create Human Reviews in AI Studio > Human Reviews or via the API.

​
API Constraints

  • Batch Limits: Up to 10 annotations per create request, up to 10 keys per delete request
  • String Length: String values are limited to 200 characters maximum
  • Deployment Span Propagation: When annotating a deployment span, the associated log is automatically annotated with the same values
  • Metadata Fields: Optional metadata object supports identityId, source, and reviewerId for tracking and attribution

​
Examples

​
Add a Quality Rating

curl -X POST "https://api.orq.ai/v2/traces/{trace_id}/spans/{span_id}/annotation" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "annotations": [
      {
        "key": "rating",
        "value": "good"
      }
    ]
  }'

​
Add Multiple Defects

curl -X POST "https://api.orq.ai/v2/traces/{trace_id}/spans/{span_id}/annotation" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "annotations": [
      {
        "key": "defects",
        "value": ["grammatical", "spelling", "ambiguity"]
      }
    ]
  }'

​
Add a Numeric Score

curl -X POST "https://api.orq.ai/v2/traces/{trace_id}/spans/{span_id}/annotation" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "annotations": [
      {
        "key": "confidence_score",
        "value": 0.95
      }
    ],
    "metadata": {
      "identityId": "user-123"
    }
  }'

​
Add a Text Correction

curl -X POST "https://api.orq.ai/v2/traces/{trace_id}/spans/{span_id}/annotation" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "annotations": [
      {
        "key": "correction",
        "value": "The correct answer should emphasize scalability and fault tolerance."
      }
    ]
  }'

​
Batch Add Multiple Annotations

curl -X POST "https://api.orq.ai/v2/traces/{trace_id}/spans/{span_id}/annotation" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "annotations": [
      {
        "key": "rating",
        "value": "good"
      },
      {
        "key": "confidence_score",
        "value": 0.92
      },
      {
        "key": "categories",
        "value": ["helpful", "accurate", "concise"]
      }
    ]
  }'

​
Remove Annotations

curl -X DELETE "https://api.orq.ai/v2/traces/{trace_id}/spans/{span_id}/annotation" \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "keys": ["rating", "defects"]
  }'

​
Error Handling

Status CodeErrorExample MessageSolution
404Human Review Not FoundThe human review with key "rating" for workspace abc123 was not found.Create a Human Review with the specified key before annotating.
404Span Not FoundSpan with id xyz789 for workspace abc123 was not found.Verify the trace_id and span_id are correct and belong to your workspace.
400Invalid ValueInvalid value: poor. Valid options are: good, bad.Ensure the value matches the options defined in the Human Review.
400Value Out of RangeValue 15 is out of range [0, 10].Provide a number within the defined min/max range for the Human Review.
400String Too LongString value exceeds maximum length of 200 characters.Shorten the string annotation to 200 characters or less.