> ## Documentation Index > Fetch the complete documentation index at: https://docs.orq.ai/llms.txt > Use this file to discover all available pages before exploring further. > Create a new human review set for manual quality evaluation. Configure review criteria and assign evaluators to assess AI-generated responses. # Create a human review set ## OpenAPI ````yaml post /v2/human-eval-sets openapi: 3.1.0 info: title: orq.ai API version: '2.0' description: orq.ai API documentation servers: - url: https://api.orq.ai security: - ApiKey: [] tags: - name: Guardrail Rules - name: Policies - name: Routing Rules - name: Files description: File upload and retrieval operations. - name: FilesService - name: Projects description: Projects organize resources within a workspace - name: ProjectsService - name: Skills description: >- Skills are modular instructions you can use to codify processes and conventions - name: SkillsService - name: Responses - description: >- Run agents on a cadence — cron, interval, or one-off. Minimum firing interval is 1 hour. name: Agent Schedules - name: Reporting description: >- GenAI reporting API over canonical analytics rollups. Accepts a metric name, time range, grain, group-by, and filters; returns a typed time series and optional totals. - name: ReportingService description: |- ReportingService exposes a single QueryReport RPC that maps allowlisted analytics payloads onto safe rollup queries. Callers never send SQL; the backend picks the rollup family and grain from the metric catalogue, the requested range, and the requested grouping. externalDocs: url: https://docs.orq.ai description: orq.ai Documentation paths: /v2/human-eval-sets: post: tags: - Human Review Sets summary: Create a human review set requestBody: content: application/json: schema: oneOf: - type: object properties: display_name: type: string minLength: 1 maxLength: 255 description: Display name of the human review set description: type: string description: Detailed description of the human review set human_eval_ids: type: array items: type: string format: ulid pattern: ^[0-9A-HJKMNP-TV-Z]{26}$ readOnly: true description: The id of the resource minItems: 1 description: >- Array of human review IDs to be applied to matching spans project_id: anyOf: - type: string format: uuid pattern: >- ^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ readOnly: true description: The id of the resource - type: string format: ulid pattern: ^[0-9A-HJKMNP-TV-Z]{26}$ readOnly: true description: The id of the resource description: >- Optional project ID to scope this human review set to a specific project filter_type: type: string enum: - span_type filter_values: type: array items: type: string minItems: 1 description: Array of span types to match required: - display_name - human_eval_ids - filter_type - filter_values - type: object properties: display_name: type: string minLength: 1 maxLength: 255 description: Display name of the human review set description: type: string description: Detailed description of the human review set human_eval_ids: type: array items: type: string format: ulid pattern: ^[0-9A-HJKMNP-TV-Z]{26}$ readOnly: true description: The id of the resource minItems: 1 description: >- Array of human review IDs to be applied to matching spans project_id: anyOf: - type: string format: uuid pattern: >- ^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ readOnly: true description: The id of the resource - type: string format: ulid pattern: ^[0-9A-HJKMNP-TV-Z]{26}$ readOnly: true description: The id of the resource description: >- Optional project ID to scope this human review set to a specific project filter_type: type: string enum: - name filter_value: type: string minLength: 1 description: Name pattern to match required: - display_name - human_eval_ids - filter_type - filter_value responses: '200': description: Successfully created a human review set content: application/json: schema: oneOf: - type: object properties: _id: type: string format: ulid pattern: ^[0-9A-HJKMNP-TV-Z]{26}$ readOnly: true description: The id of the resource display_name: type: string minLength: 1 maxLength: 255 description: Display name of the human review set description: type: string description: Detailed description of the human review set human_eval_ids: type: array items: type: string format: ulid pattern: ^[0-9A-HJKMNP-TV-Z]{26}$ readOnly: true description: The id of the resource minItems: 1 description: >- Array of human review IDs to be applied to matching spans workspace_id: type: string project_id: anyOf: - type: string format: uuid pattern: >- ^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ readOnly: true description: The id of the resource - type: string format: ulid pattern: ^[0-9A-HJKMNP-TV-Z]{26}$ readOnly: true description: The id of the resource description: >- Optional project ID to scope this human review set to a specific project created_by_id: type: string updated_by_id: type: string created: anyOf: - format: date-time type: string - format: date-time type: string default: '2026-05-14T21:19:47.506Z' readOnly: true format: date-time description: The date and time the resource was created updated: anyOf: - type: string format: date-time default: '2026-05-14T21:19:47.508Z' - format: date-time type: string readOnly: true format: date-time description: The date and time the resource was last updated filter_type: type: string enum: - span_type filter_values: type: array items: type: string minItems: 1 description: Array of span types to match required: - _id - display_name - human_eval_ids - workspace_id - filter_type - filter_values - type: object properties: _id: type: string format: ulid pattern: ^[0-9A-HJKMNP-TV-Z]{26}$ readOnly: true description: The id of the resource display_name: type: string minLength: 1 maxLength: 255 description: Display name of the human review set description: type: string description: Detailed description of the human review set human_eval_ids: type: array items: type: string format: ulid pattern: ^[0-9A-HJKMNP-TV-Z]{26}$ readOnly: true description: The id of the resource minItems: 1 description: >- Array of human review IDs to be applied to matching spans workspace_id: type: string project_id: anyOf: - type: string format: uuid pattern: >- ^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ readOnly: true description: The id of the resource - type: string format: ulid pattern: ^[0-9A-HJKMNP-TV-Z]{26}$ readOnly: true description: The id of the resource description: >- Optional project ID to scope this human review set to a specific project created_by_id: type: string updated_by_id: type: string created: anyOf: - format: date-time type: string - format: date-time type: string default: '2026-05-14T21:19:47.506Z' readOnly: true format: date-time description: The date and time the resource was created updated: anyOf: - type: string format: date-time default: '2026-05-14T21:19:47.508Z' - format: date-time type: string readOnly: true format: date-time description: The date and time the resource was last updated filter_type: type: string enum: - name filter_value: type: string minLength: 1 description: Name pattern to match required: - _id - display_name - human_eval_ids - workspace_id - filter_type - filter_value components: securitySchemes: ApiKey: type: http scheme: bearer bearerFormat: JWT ````