Evaluators in Experiments

Overview

Adding Evaluators to your Experiment allows you to quantitatively evaluate the model-generated outputs. Using standard scientific methods or custom LLM-based evaluations, you can automate the scoring of models to quickly detect whether models fit a predefined hypothesis and if they stand out from one another.

Within Experiments, evaluators offer a quick way to validate the behavior of multiple models on a large Dataset.

📘

To get started with Experiments, see Setting up an Experiment

Adding Evaluators to Experiment

To add an Evaluator to an experiment, head to the right of the table and Add new Column > Evaluator

The following panel opens, showing all Evaluators available in your current Project.

To add an Evaluator, enable its toggle. It will appear as a new column in the Report table.

You can also see the Evaluator details by selecting the View button.

📘

To add more Evaluators to your Projects, see Evaluators. You can choose to import Evaluators from our Evaluator Library or create your own Custom LLM Evaluator.

Viewing Evaluator Result

Once an Experiment has been run, you can view the Evaluator results on the Report page. Evaluators will be shown as columns next to the Cost and Latency results. Evaluators display results depending on their configuration.