added
Online Evaluators in Live Deployments
22 days ago by Kyra Dresen
You can now configure Evaluators after you have added them to your Library directly in Deployments > Settings for both input and output, giving you full control over live deployments.
Why Are Evaluators Important?
- Quality and Safety: Evaluators mitigate LLM risks by monitoring and ensuring output quality and safety.
- Performance Insights: They provide clear feedback on updates, helping teams identify improvements or regressions and refine iterations effectively.
What Are Evaluators?
Evaluators are tools designed to assess the quality, relevance, and safety of AI inputs or outputs, ensuring reliable and effective system performance. Off the shelf, we offer several options, including LLMs as a Judge, RAGAS, Function, HTTP, and JSON evaluators. Additionally, the evaluators allow for the following configuration:
- Sample Rate Option:Run evaluations on a subset of calls (e.g., 10%) to balance coverage and efficiency.
- Cost Management:Reduces additional expenses associated with running evaluators, especially LLM-based ones.
- Latency Control: Minimizes added response time by limiting the number of calls evaluated.
If you want to set up the evals in your own deployment, you can find more information here.
