What is an HTTP Evaluator?
The HTTP evaluator allows users to set up a custom evaluation by calling an external API, enabling flexible and tailored assessments of generated responses. This approach lets users leverage their own or third-party APIs to perform specific checks, such as custom quality scoring, compliance verification, or domain-specific validations that align with their unique requirements.Example
Suppose a customer wants to evaluate AI-generated responses for compliance with industry-specific language. They set up the HTTP evaluator to call their custom compliance-check API, which analyzes each response and returns a score. If a response does not meet the compliance standards, the evaluator flags it, allowing the customer to ensure that generated content adheres to their specific regulatory guidelines.What’s Next