Skip to main content
Annotation Queues are part of the unified Annotations system. They work seamlessly with the Annotations API for programmatic quality tracking and human-in-the-loop workflows.
Annotation queues help you organize and apply Human Reviews effectively to relevant incoming Traces.

​
Creating an Annotation Queue

To create an Annotation Queue, head to the AI Studio > Annotation Queue. Choose Create Annotation Queue. The following fields are configurable:
  • The Name of the queue
  • The Description of the Annotation Queue
  • The Human Reviews that Traces will be reviewed by
Creating Annotation Queue
Annotation queues can be filled with Traces using Trace Automations, which automatically route traces to the appropriate queue based on your configured rules.

​
Using Annotation Queues

When opening an Annotation Queue, you have easy access to all Traces that need feedback.
Annotation Queue
On the right side of the panel, Review and use Feedbacks on the conversation. Use Next to access the next Trace in the queue.
You can add manually any Trace to a Dataset to use in a future Experiment, to ensure further testing on model behavior.