Dataset

Overview

Datasets hold Messages, Inputs and Expected Outputs to test models at scale.

• Inputs – Variables that can be used in the prompt message, e.g., {{firstname}}.
• Messages – The prompt template, structured with system, user, and assistant roles.
• Expected Outputs – Reference responses that evaluators use to compare against newly generated outputs.

Datasets hold entries that let you test your models to make sure their performances are up to expectations, whether it be on quality of output, cost or latency. With datasets, you can test models at each iterations to be sure your updates are free of regressions and incrementally improve the user experience.

Datasets are better used within Experiments, make sure you have models available by adding them to your Model Garden.

Note: You don’t need to include all three entities when uploading a dataset. Depending on your experiment, you can choose to include only inputs, messages, or expected outputs as needed. For example, you can create a dataset with just inputs.

Prerequisites

To get started with Datasets, make sure you have access to a Projects and the sufficient Permissions within the workspace.

To get started, see Creating a Dataset.