JSON Evaluator

What is a JSON Evaluator

JSON Evaluators are ways to validate payloads against a JSON Schema for LLM generations.

Why use JSON Evaluators?

Using a JSON Evaluator lets us ensure that payloads correspond to what is expected, contain all fields needed for a generation to function correctly. They especially work well in conjunction with Tools to validate generated payloads before they are used with third-party systems.

📘

JSON Evaluators can be used as Guardrails to block invalid Inputs/Outputs from a Deployment, to learn more, see Evaluators & Guardrails in Deployments.

To get started, see Creating a JSON Evaluator.