Evaluators

Evaluators are automated actions performed in an Experiment. They can be used to assess the performance and generated outputs of models, compare outputs with a reference, or perform any automated verification.

There are two types of Evaluators available:

• Standard Evaluators are always available in your orq.ai panel.
• LLM Evaluators are custom ones you can create. They leverage language models to perform any assessment you desire on your experiment.

To learn more about how to use Evaluators, see Evaluators in Experiments.