Setting up an Experiment
Follow these steps to set up your first Experiment
Experiments offer ways to quickly test a number of scenarios against multiple models at once. By configuring Datasets, Variables, and Evaluators, you can create complex output and validation for a wide variety of cases, validating your hypotheses before moving to Deployments.
Creating an Experiment
To create an experiment:
- Login to your Admin Panel.
- Go to the Experiments tab.
- Select Create Experiment
- Choose a Title and Project and select Create.
Configuring your Dataset
Head to the Datasets tab where you can configure the prompts that will be tested as well as their references.
To learn more about Datasets configuration see Datasets.
Configuring Variables
Inputs are variable text that can be templated into your prompt. You can then use any number of value for Variables.
To configure Inputs head to the Variable Collections tab.
To learn more about Inputs configuration see Variables.
Configuring Models
Next you can decide which models will be tested and compared in the experiments. You can choose any number of models that are available within your Model Garden.
To learn more about Model configuration, see Model Configuration.
Configuring Evaluators
Evaluators are automated ways to review the results of generations made by the models within the Experiments.
There are many Standard Evaluators available, you can also create Custom LLM Evaluators.
To learn more about Evaluators Configuration, see Evaluators in Experiments.
Running the Experiment
Once your experiment configuration is ready, you can decide to run it by selecting Run Experiment.
Your experiment will run in the background, all results will appear within the Report tab.
Once the experiment is finished, you can review all outputs generated as well as Evaluators results, giving you insights into the success of the experiment.
Exporting Results
You can export experiment results by pressing the ... at the top-right of the screen then choosing Export.
You can choose to export in CSV, JSON Lines, or JSON format.
Updated 2 months ago