Create an Evaluator
Use an Evaluator
Evaluator Library
- Import existing Evaluators from Evaluator Library
- Use the Evaluator Library via the API
- Learn more about our different Evaluator types:
Action required: Entity management is moving from Workspaces to Projects. Migrate your entities now. See how to move entities in our guide.
Evaluator
Evaluators | Automate LLM Output Assessment
Automate LLM output evaluation with custom evaluators. Use LLM-as-judge, Python code, HTTP endpoints, or RAGAS metrics to validate model responses in experiments and production.
Evaluators are automated tools designed to assess the performance and outputs of models within an Experiment, Deployment, and Agent.
Evaluators can verify outputs against reference data, ensure compliance with specific criteria, and perform various automated validations.
By utilizing Evaluators, teams can automate the validation process, maintain high-quality outputs, and ensure that their AI systems operate within desired parameters.
To get started:
Was this page helpful?
Integrate and use Deployments via the API | SDK & API Setup Guide
Previous
Create Evaluators | LLM Output Assessment
Next
⌘I