Creating an Evaluator

Custom Evaluators can be added to your Evaluator Library.

To create an Evaluator, use the + button on a desired Folder within a Project and choose Evaluator.

The following modal opens:

To see details on creating Evaluators:

• see Creating LLM-as-a-judge Evaluator.
• see Creating API Call Evaluator.
• see Creating JSON Schema Evaluator.
• see Creating a Python Evaluator.