Evaluator
Evaluators automate the performance assessment of outputs in Experiments and Deployments.
Evaluators are automated tools designed to assess the performance and outputs of models within an Experiment or Deployment.
Evaluators can verify outputs against reference data, ensure compliance with specific criteria, and perform various automated validations.
By utilizing Evaluators, teams can automate the validation process, maintain high-quality outputs, and ensure that their AI systems operate within desired parameters.
To create started:
- Create an Evaluator
- Using Evaluator in Experiment
- Evaluators & Guardrails in Deployments
- Import existing Evaluators from Evaluator Library in the Hub
- Learn more about our different Evaluator types:
- Function Evaluator readily available in the Hub
- HTTP Evaluator
- JSON Evaluator
- LLM Evaluator readily available in the Hub
- Python Evaluator
- Ragas Evaluator readily available in the Hub
Updated 13 days ago
What’s Next