Custom Human Review

Today we're introducing Custom human review, a major update that gives teams full flexibility in collecting and structuring human feedback in Orq.ai.

With this release, you’re no longer limited to predefined review types. Now, you can create your own feedback forms tailored to your workflow and use case. Choose from three flexible input types:

• Categorical: Create button options with your own labels, such as good/bad or saved/deleted.
• Range: Set up a custom scoring slider (for example, a scale from 0 to 100).
• Open field: Allow users to enter free-form feedback, perfect for more detailed comments.

How to collect feedback

There are two ways to collect feedback:

• Orq Studio: Custom human review templates can be used directly within the Orq.ai platform, where colleagues and domain experts review AI outputs and provide structured feedback.
• API: Alternatively, developers can leverage the Orq API to expose these custom feedback forms in their own applications or products, allowing end users to submit feedback seamlessly.

Why this matters

Collecting high-quality, relevant human review is crucial for improving AI features. Teams can structure feedback in a way that makes sense for them, whether for quality control, prompt iteration, or continuous evaluation. All feedback is attached to the log, including prompts and outputs, making it easy for prompt engineers to refine configurations based on real-world usage.

How to create custom Human Review