HTTP
HTTP Evaluators lets you run custom evaluation on an endpoint of your choice.
What is an HTTP Evaluator?
The API Call evaluator allows users to set up a custom evaluation by calling an external API, enabling flexible and tailored assessments of generated responses.
This approach lets users leverage their own or third-party APIs to perform specific checks, such as custom quality scoring, compliance verification, or domain-specific validations that align with their unique requirements.
Example
Suppose a customer wants to evaluate AI-generated responses for compliance with industry-specific language.
They set up the API Call evaluator to call their custom compliance-check API, which analyzes each response and returns a score.
If a response does not meet the compliance standards, the evaluator flags it, allowing the customer to ensure that generated content adheres to their specific regulatory guidelines.
Updated 8 days ago