Running an Experiment

​

Prerequisite

First configure your Experiment, see Creating an Experiment

​

Running an Experiment

Once configured, you can run the Experiment using the Run button. Depending on the Dataset size it may take a few minutes to run all model prompts generations. Once successful your Experiment Run Status will change to Completed. You can then see Experiment Results.

​

Only Evaluating Existing Dataset Outputs

If you want to test evaluators on datasets that already contain generated responses, you can run an evaluation-only experiment:

1. Set up your experiment with the dataset containing existing outputs in the “messages” column
2. Do not select a prompt during experiment setup
3. Add your desired evaluators
4. Run the experiment

This mode will evaluate the existing responses without generating new outputs, allowing you to retroactively score historical responses and conversation chains that are already stored in your dataset.

​

Running a single Prompt

It is often useful to add an extra prompt after running an experiment, to tweak a configuration or try a different version. Once a new Prompt is added, select and choose Run to run on the existing Dataset.

​

Seeing Experiment Results

​

Report

Once an Experiment is ran, its status will change from Running to Completed The right side of the table will be filled with results. Under each Column corresponding to each Prompt you can see results for:

• Latency.
• Costs.
• Evaluators.

​

Comparing Model Performance

Using the Compare tab, visualize multiple model executions.
The variables and expected outputs are now displayed on the left for better context, especially when working with large inputs or detailed test cases. At the bottom of the screen, the evaluators section provides scores and feedback for each result, helping you assess model quality and performance at a glance. You can use this screen to easily apply Feedbacks and Human Review to each output, letting you evaluate and review Experiment results efficiently.

​

Viewing Multiple Experiment Runs

Within the Runs tab, visualize all previous runs for an Experiment. Through this view, all Evaluators results are visible at a glance, making it easy to compare result and see progress between multiple Runs.

​

Compare Results

For easy comparison between models and prompts, click the Compare tab. This opens the redesigned comparison view where you can analyze outputs side by side across multiple models or configurations.
The variables and expected outputs are now displayed on the left for better context, especially when working with large inputs or detailed test cases.
At the bottom of the screen, the evaluators section provides scores and feedback for each result, helping you assess model quality and performance at a glance.

​

Logs

Switching to the Logs tab lets you see the details of each call. Within logs you can process Feedback and build Curated Dataset By hovering a cell you can also directly access the related log using the See log button.

​

Export

You can export datasets and results in CSV, JSON and JSONL formats using the Export button.