Setting up an Experiment

Experiments offer ways to quickly test a number of scenarios against multiple models at once. By configuring prompts, Datasets , Inputs and Evaluators you can create complex output and validation for a wide variety of cases, validating your hypotheses before moving to Deployments.

Creating an Experiment

To create an experiment:

• Login to your Admin Panel.
• Go to the Experiments tab.
• Select Create Experiment
• Choose a Title and Domain and select Create.

Configuring your Dataset

Head to the Datasets tab where you can configure the prompts that will be tested as well as their references.

📘

To learn more about Datasets configuration see Datasets.

Configuring Inputs

Inputs are variable text that can be templated into your prompt. You can then use any number of value for Inputs.

To configure Inputs head to the Inputs tab.

📘

To learn more about Inputs configuration see Inputs.

Configuring Models

Next you can decide which models will be tested and compared in the experiments. You can choose any number of models that are available within your Model Garden.

📘

To learn more about Model configuration, see Model Configuration.

Configuring Evaluators

Evaluators are automated ways to review the results of generations made by the models within the Experiments.

There are many Standard Evaluators available, you can also create Custom LLM Evaluators.

📘

To learn more about Evaluators Configuration, see Evaluators in Experiments.

Running the Experiment

Once your experiment configuration is ready, you can decide to run it by selecting Run Experiment.

Your experiment will run in the background, all results will appear within the Report tab.

Once the experiment is finished, you can review all outputs generated as well as Evaluators results, giving you insights into the success of the experiment.