> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orq.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Human Review

> Set up project-level human review workflows to flag LLM outputs for manual inspection. Configure review criteria, thresholds, and annotation queues.

<Info>
  Human Review is now configured at the project level for better organization and control. If you previously configured workspace-level human reviews, they will continue to work but are soon deprecated.
</Info>

<Info>
  **Unified Annotations System:** Human Reviews are part of the [Annotations](/docs/annotations/build) system. Use the [Annotations API](/docs/annotations/build) to programmatically apply human reviews to traces, or use [Annotation Queues](/docs/annotations/build#create-annotation-queues) to manage human review workflows in the UI.
</Info>

## What is a Human Review

Human Reviews let you create custom fields for structured annotations on model generations. Once created, a Human Review is available on all traces and spans within the project. No additional configuration or filtering required.

## Creating a Human Review

To create a Human Review, head to **Project Settings > Human Review** and press the `+` button.

<Frame caption="Customizing a Human Review.">
  <img src="https://mintcdn.com/orqai/6HurnhGELvozB4iC/images/docs/human-review-settings.png?fit=max&auto=format&n=6HurnhGELvozB4iC&q=85&s=0de8b6fbf5c6356d4d6c63891cbbf980" alt="Human Review settings" width="1018" height="994" data-path="images/docs/human-review-settings.png" />
</Frame>

There are 3 human review types available:

* **Categorical:** Create button options with your own labels, such as good/bad or saved/deleted.
* **Range:** Set up a custom scoring slider (for example, a scale from 0 to 100).
* **Open field:** Allow users to enter free-form feedback, perfect for more detailed comments.

## Using Human Review

Once created, a Human Review is available on all traces and spans in the project. Open any trace or span to apply feedback directly.

<img src="https://mintcdn.com/orqai/mhcgQaQdKDq98ARV/images/docs/198c01162597f3e58a2fde19f4ab8525bdb845121cedf0e2b884b153d66ed66e-Screenshot_2025-07-04_at_15.08.54.png?fit=max&auto=format&n=mhcgQaQdKDq98ARV&q=85&s=cebce846244e2c35b2e9cba2905890f4" alt="Using Human Review" width="540" height="1212" data-path="images/docs/198c01162597f3e58a2fde19f4ab8525bdb845121cedf0e2b884b153d66ed66e-Screenshot_2025-07-04_at_15.08.54.png" />

## What's next

<CardGroup cols={2}>
  <Card title="Annotations API" icon="code" href="/docs/annotations/build">
    Apply human reviews programmatically using the REST API or SDKs.
  </Card>

  <Card title="Annotation Queues" icon="list-check" href="/docs/annotations/build#create-annotation-queues">
    Organize human review workflows and efficiently process traces in bulk.
  </Card>
</CardGroup>