Evaluator
Evaluators automate the performance assessment of outputs in Experiments and Deployments.
Evaluators are automated tools designed to assess the performance and outputs of models within an Experiment or Deployment.
Evaluators can verify outputs against reference data, ensure compliance with specific criteria, and perform various automated validations.
By utilizing Evaluators, teams can automate the validation process, maintain high-quality outputs, and ensure that their AI systems operate within desired parameters.
To create started:
Create an Evaluator
Use an Evaluator
Evaluator Library
- Import existing Evaluators from Evaluator Library in the Hub
- Use the Evaluator Library via the API
- Learn more about our different Evaluator types:
- Function Evaluator readily available in the Hub
- HTTP Evaluator
- JSON Evaluator
- LLM Evaluator readily available in the Hub
- Python Evaluator
- Ragas Evaluator readily available in the Hub
Updated 10 days ago
What’s Next