> ## Documentation Index > Fetch the complete documentation index at: https://docs.orq.ai/llms.txt > Use this file to discover all available pages before exploring further. # Retrieve an Evaluator > Retrieve a single evaluator by its unique identifier. Returns the evaluator exactly as stored, including its type-specific configuration — prompt and model for LLM evaluators, source code for Python and TypeScript evaluators, the JSON Schema for schema evaluators, and so on. Use this when you already know the evaluator id (for example to refresh the state of a resource you manage declaratively). To discover evaluator ids, list them with `GET /v2/evaluators`. This endpoint returns the stored record, which carries more detail than the representation `GET /v2/evaluators` returns: `display_name` rather than `key`, `model` as an object rather than a provider-qualified string, plus the `owner`, `domain_id`, `metadata`, `enabled` and `output_type` fields. ## OpenAPI ````yaml get /v2/evaluators/{id} openapi: 3.1.0 info: title: orq.ai API version: '2.0' description: orq.ai API documentation servers: - url: https://api.orq.ai security: - ApiKey: [] tags: - description: List models available through the AI Router. name: Models - name: Guardrail Rules - name: Policies - name: Routing Rules - name: Activities description: Activity timeline entries for comments and entity versions. - name: Alerts description: >- Alerts evaluate a Reporting API metric on a fixed interval and fire notifications through notifiers when the value breaches a threshold. Each breach opens a trigger that tracks the incident until the value recovers. - name: API keys description: >- API keys authenticate programmatic access to the workspace. The unified key model exposes opaque tokens, per-domain access grants, and budget / rate-limit constraints (see ADR 0001 and ADR 0002). - name: Budgets description: >- Budgets govern spend, token usage, and request rate across six scopes: workspace, project, identity, api-key, provider, and model. A budget is hierarchical and defense-in-depth — every applicable budget is a hard gate, and the most restrictive one wins per dimension (see ADR 0007). - name: Documentation description: >- Search the orq.ai documentation. Proxies the workspace's query to the hosted docs search index. - name: FeaturePreviews description: >- Internal: list and toggle per-user feature previews (beta features users can opt into). - name: Files description: File upload and retrieval operations. - name: Finder description: >- Finder entities back the studio navigation tree, project switcher, and cross-project resource references. - name: Identities description: >- Identities represent end users from your system for usage and engagement tracking. - name: Management keys description: >- Management keys are workspace-scoped credentials that authenticate programmatic access to workspace administration surfaces (API keys, budgets). Unlike project-scoped API keys, a management key always operates at the workspace level. - name: MCP Gateway description: >- Manage upstream MCP servers and client-facing MCP gateways. Create server connections, discover tools, assemble gateways, and review call logs. - name: Model Sharing description: >- Model sharing controls which projects a workspace model is available to. Workspace admins configure a model's Sharing (all projects / selected projects); the config ships inline on the models list response, so there is no separate read endpoint. - name: Monitors description: >- Monitors are custom observability dashboards over GenAI trace metrics. Each monitor holds a set of widgets whose queries run against the Reporting API metric catalogue. - name: Notifiers description: Notifier destinations used to send delivery and workflow notifications. - name: People description: >- People are workspace members and invited users. Manage roles, groups, and invitations for your team. - name: Projects description: Projects organize resources within a workspace - name: Skills description: >- Skills are modular instructions you can use to codify processes and conventions - name: Smart Routers description: >- Smart Routers distribute an ordered pool of eligible language models across V2 routing bands. - name: Webhooks description: >- Create and manage webhooks that deliver workspace events to external HTTPS endpoints. - name: Workspace Settings description: >- Workspace-level settings managed with a workspace credential. A workspace is the tenant, so these settings are a singleton — there is nothing to create or delete, only read and update. - name: Responses - description: >- Run agents on a cadence — cron, interval, or one-off. Minimum firing interval is 1 hour. name: Agent Schedules - name: Embeddings - name: Logs description: >- OpenTelemetry log query API. Search, filter, aggregate, and facet log records ingested via OTLP. - name: Reporting description: >- GenAI reporting API over canonical analytics rollups. Accepts a metric name, time range, grain, group-by, and filters; returns a typed time series and optional totals. - name: Traces description: >- Query and inspect ingested trace data: search trace summaries, aggregate metrics, and read individual traces and their spans. externalDocs: url: https://docs.orq.ai description: orq.ai Documentation paths: /v2/evaluators/{id}: get: tags: - Evals summary: Retrieve an Evaluator description: >- Retrieve a single evaluator by its unique identifier. Returns the evaluator exactly as stored, including its type-specific configuration — prompt and model for LLM evaluators, source code for Python and TypeScript evaluators, the JSON Schema for schema evaluators, and so on. Use this when you already know the evaluator id (for example to refresh the state of a resource you manage declaratively). To discover evaluator ids, list them with `GET /v2/evaluators`. This endpoint returns the stored record, which carries more detail than the representation `GET /v2/evaluators` returns: `display_name` rather than `key`, `model` as an object rather than a provider-qualified string, plus the `owner`, `domain_id`, `metadata`, `enabled` and `output_type` fields. operationId: GetEval parameters: - schema: type: string description: >- Unique identifier of the evaluator, as returned in the `_id` field by `GET /v2/evaluators`. example: 01JMDPA3QW5C1V0NJ1PW34T4E5 required: true description: >- Unique identifier of the evaluator, as returned in the `_id` field by `GET /v2/evaluators`. name: id in: path responses: '200': description: The evaluator. The response shape depends on the evaluator `type`. content: application/json: schema: oneOf: - type: object properties: enabled: type: boolean default: true metadata: type: object properties: required_model_with_tools_support: type: boolean required_retrieval_context: type: boolean required_expected_output: type: boolean supported_on_input_type: type: boolean supported_on_output_type: type: boolean support_use_as_guardrail: type: boolean _id: type: string display_name: type: string description: type: string owner: type: string created: type: string default: '2026-08-04T15:14:03.981Z' updated: type: string default: '2026-08-04T15:14:03.981Z' created_by_id: type: - string - 'null' updated_by_id: type: - string - 'null' domain_id: type: string guardrail_config: anyOf: - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - boolean value: type: boolean required: - type - value title: Boolean - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - categorical values: type: array items: type: string minLength: 1 required: - type - values title: Categorical - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - number value: type: number operator: type: string enum: - eq - ne - gt - gte - lt - lte required: - type - value - operator title: Number - type: 'null' x-speakeasy-type-override: any output_type: type: string enum: - boolean - categorical - number - string description: The type of output expected from the evaluator type: type: string enum: - llm_eval mode: type: - string - 'null' enum: - single - jury - null repetitions: type: - integer - 'null' minimum: 1 maximum: 3 model: type: object properties: id: type: string integration_id: type: - string - 'null' model_parameters: type: object properties: temperature: type: number max_tokens: type: number top_k: type: number top_p: type: number frequency_penalty: type: number presence_penalty: type: number reasoning_effort: type: string budget_tokens: type: number required: - id jury: type: - object - 'null' properties: judges: type: array items: type: object properties: model: type: object properties: id: type: string integration_id: type: - string - 'null' model_parameters: type: object properties: temperature: type: number max_tokens: type: number top_k: type: number top_p: type: number frequency_penalty: type: number presence_penalty: type: number reasoning_effort: type: string budget_tokens: type: number required: - id retry: type: object properties: count: type: integer minimum: 1 maximum: 5 default: 2 on_codes: type: array items: type: integer minimum: 100 maximum: 599 minItems: 1 default: - 429 - 500 - 502 - 503 - 504 fallbacks: type: array items: type: object properties: model: type: object properties: id: type: string integration_id: type: - string - 'null' model_parameters: type: object properties: temperature: type: number max_tokens: type: number top_k: type: number top_p: type: number frequency_penalty: type: number presence_penalty: type: number reasoning_effort: type: string budget_tokens: type: number required: - id required: - model default: [] required: - model minItems: 2 replacement_judges: type: array items: type: object properties: model: type: object properties: id: type: string integration_id: type: - string - 'null' model_parameters: type: object properties: temperature: type: number max_tokens: type: number top_k: type: number top_p: type: number frequency_penalty: type: number presence_penalty: type: number reasoning_effort: type: string budget_tokens: type: number required: - id retry: type: object properties: count: type: integer minimum: 1 maximum: 5 default: 2 on_codes: type: array items: type: integer minimum: 100 maximum: 599 minItems: 1 default: - 429 - 500 - 502 - 503 - 504 fallbacks: type: array items: type: object properties: model: type: object properties: id: type: string integration_id: type: - string - 'null' model_parameters: type: object properties: temperature: type: number max_tokens: type: number top_k: type: number top_p: type: number frequency_penalty: type: number presence_penalty: type: number reasoning_effort: type: string budget_tokens: type: number required: - id required: - model default: [] required: - model default: [] min_successful_judges: type: integer minimum: 2 default: 2 tie_value: type: string enum: - Tie default: Tie required: - judges prompt: type: string categories: type: - array - 'null' items: type: string categorical_labels: type: - array - 'null' items: type: object properties: value: type: string minLength: 1 description: type: string required: - value dataset_id: type: - string - 'null' required: - metadata - _id - display_name - description - owner - domain_id - type - prompt title: LLM - type: object properties: enabled: type: boolean default: true metadata: type: object properties: required_model_with_tools_support: type: boolean required_retrieval_context: type: boolean required_expected_output: type: boolean supported_on_input_type: type: boolean supported_on_output_type: type: boolean support_use_as_guardrail: type: boolean _id: type: string display_name: type: string description: type: string owner: type: string created: type: string default: '2026-08-04T15:14:03.981Z' updated: type: string default: '2026-08-04T15:14:03.981Z' created_by_id: type: - string - 'null' updated_by_id: type: - string - 'null' domain_id: type: string guardrail_config: anyOf: - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - boolean value: type: boolean required: - type - value title: Boolean - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - categorical values: type: array items: type: string minLength: 1 required: - type - values title: Categorical - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - number value: type: number operator: type: string enum: - eq - ne - gt - gte - lt - lte required: - type - value - operator title: Number - type: 'null' x-speakeasy-type-override: any output_type: type: string enum: - boolean - categorical - number - string type: type: string enum: - function_eval function_params: oneOf: - type: object properties: type: type: string enum: - contains value: type: string required: - type - value - type: object properties: type: type: string enum: - contains_none keywords: type: array items: type: string required: - type - keywords - type: object properties: type: type: string enum: - contains_all keywords: type: array items: type: string required: - type - keywords - type: object properties: type: type: string enum: - contains_any keywords: type: array items: type: string required: - type - keywords - type: object properties: type: type: string enum: - contains_email required: - type - type: object properties: type: type: string enum: - contains_url required: - type - type: object properties: type: type: string enum: - contains_valid_link required: - type - type: object properties: type: type: string enum: - start_with value: type: string required: - type - value - type: object properties: type: type: string enum: - ends_with value: type: string required: - type - value - type: object properties: type: type: string enum: - exact_match required: - type - type: object properties: type: type: string enum: - length_less_than value: type: number required: - type - value - type: object properties: type: type: string enum: - length_between min: type: number max: type: number required: - type - min - max - type: object properties: type: type: string enum: - length_greater_than value: type: number required: - type - value - type: object properties: type: type: string enum: - one_line required: - type - type: object properties: type: type: string enum: - regex pattern: type: string required: - type - pattern - type: object properties: type: type: string enum: - is_valid_json required: - type - type: object properties: type: type: string enum: - moderations_openai required: - type - type: object properties: type: type: string enum: - moderations_google required: - type - type: object properties: type: type: string enum: - bert_score required: - type - type: object properties: type: type: string enum: - bleu_score required: - type - type: object properties: type: type: string enum: - rouge_n required: - type - type: object properties: type: type: string enum: - meteor_score required: - type - type: object properties: type: type: string enum: - cosine_similarity required: - type - type: object properties: type: type: string enum: - levenshtein_distance required: - type - type: object properties: type: type: string enum: - keywords_match keywords: type: array items: type: string required: - type - keywords - type: object properties: type: type: string enum: - most_repeated_words required: - type - type: object properties: type: type: string enum: - flesch_reading_ease required: - type - type: object properties: type: type: string enum: - gse_english_level required: - type - type: object properties: type: type: string enum: - words_count required: - type - type: object properties: type: type: string enum: - sentences_count required: - type - type: object properties: type: type: string enum: - lexical_repetition required: - type - type: object properties: type: type: string enum: - grammar_diversity required: - type required: - metadata - _id - display_name - description - owner - domain_id - output_type - type - function_params title: Function - type: object properties: enabled: type: boolean default: true metadata: type: object properties: required_model_with_tools_support: type: boolean required_retrieval_context: type: boolean required_expected_output: type: boolean supported_on_input_type: type: boolean supported_on_output_type: type: boolean support_use_as_guardrail: type: boolean _id: type: string display_name: type: string description: type: string owner: type: string created: type: string default: '2026-08-04T15:14:03.981Z' updated: type: string default: '2026-08-04T15:14:03.981Z' created_by_id: type: - string - 'null' updated_by_id: type: - string - 'null' domain_id: type: string guardrail_config: anyOf: - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - boolean value: type: boolean required: - type - value title: Boolean - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - categorical values: type: array items: type: string minLength: 1 required: - type - values title: Categorical - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - number value: type: number operator: type: string enum: - eq - ne - gt - gte - lt - lte required: - type - value - operator title: Number - type: 'null' x-speakeasy-type-override: any output_type: type: string enum: - number type: type: string enum: - ragas model: type: object properties: id: type: string integration_id: type: - string - 'null' required: - id ragas_metric: type: string enum: - context_precision - context_recall - context_entities_recall - harmfulness - maliciousness - coherence - correctness - conciseness - response_relevancy - faithfulness - summarization - noise_sensitivity required: - metadata - _id - display_name - description - owner - domain_id - type - model - ragas_metric title: Ragas - type: object properties: enabled: type: boolean default: true metadata: type: object properties: required_model_with_tools_support: type: boolean required_retrieval_context: type: boolean required_expected_output: type: boolean supported_on_input_type: type: boolean supported_on_output_type: type: boolean support_use_as_guardrail: type: boolean _id: type: string display_name: type: string description: type: string owner: type: string created: type: string default: '2026-08-04T15:14:03.981Z' updated: type: string default: '2026-08-04T15:14:03.981Z' created_by_id: type: - string - 'null' updated_by_id: type: - string - 'null' domain_id: type: string guardrail_config: anyOf: - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - boolean value: type: boolean required: - type - value title: Boolean - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - categorical values: type: array items: type: string minLength: 1 required: - type - values title: Categorical - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - number value: type: number operator: type: string enum: - eq - ne - gt - gte - lt - lte required: - type - value - operator title: Number - type: 'null' x-speakeasy-type-override: any output_type: type: string enum: - boolean type: type: string enum: - json_schema schema: type: string required: - metadata - _id - display_name - description - owner - domain_id - type - schema title: JSON - type: object properties: enabled: type: boolean default: true metadata: type: object properties: required_model_with_tools_support: type: boolean required_retrieval_context: type: boolean required_expected_output: type: boolean supported_on_input_type: type: boolean supported_on_output_type: type: boolean support_use_as_guardrail: type: boolean _id: type: string display_name: type: string description: type: string owner: type: string created: type: string default: '2026-08-04T15:14:03.981Z' updated: type: string default: '2026-08-04T15:14:03.981Z' created_by_id: type: - string - 'null' updated_by_id: type: - string - 'null' domain_id: type: string guardrail_config: anyOf: - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - boolean value: type: boolean required: - type - value title: Boolean - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - categorical values: type: array items: type: string minLength: 1 required: - type - values title: Categorical - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - number value: type: number operator: type: string enum: - eq - ne - gt - gte - lt - lte required: - type - value - operator title: Number - type: 'null' x-speakeasy-type-override: any output_type: type: string enum: - boolean - categorical - number - string description: The type of output expected from the evaluator type: type: string enum: - http_eval url: type: string method: type: string enum: - GET - POST headers: type: object additionalProperties: type: string payload: type: object additionalProperties: {} required: - metadata - _id - display_name - description - owner - domain_id - type - url - method - headers - payload title: HTTP - type: object properties: enabled: type: boolean default: true metadata: type: object properties: required_model_with_tools_support: type: boolean required_retrieval_context: type: boolean required_expected_output: type: boolean supported_on_input_type: type: boolean supported_on_output_type: type: boolean support_use_as_guardrail: type: boolean _id: type: string display_name: type: string description: type: string owner: type: string created: type: string default: '2026-08-04T15:14:03.981Z' updated: type: string default: '2026-08-04T15:14:03.981Z' created_by_id: type: - string - 'null' updated_by_id: type: - string - 'null' domain_id: type: string guardrail_config: anyOf: - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - boolean value: type: boolean required: - type - value title: Boolean - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - categorical values: type: array items: type: string minLength: 1 required: - type - values title: Categorical - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - number value: type: number operator: type: string enum: - eq - ne - gt - gte - lt - lte required: - type - value - operator title: Number - type: 'null' x-speakeasy-type-override: any output_type: type: string enum: - boolean - number code: type: string type: type: string enum: - python_eval required: - metadata - _id - display_name - description - owner - domain_id - code - type title: Python - type: object properties: enabled: type: boolean default: true metadata: type: object properties: required_model_with_tools_support: type: boolean required_retrieval_context: type: boolean required_expected_output: type: boolean supported_on_input_type: type: boolean supported_on_output_type: type: boolean support_use_as_guardrail: type: boolean _id: type: string display_name: type: string description: type: string owner: type: string created: type: string default: '2026-08-04T15:14:03.981Z' updated: type: string default: '2026-08-04T15:14:03.981Z' created_by_id: type: - string - 'null' updated_by_id: type: - string - 'null' domain_id: type: string guardrail_config: anyOf: - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - boolean value: type: boolean required: - type - value title: Boolean - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - categorical values: type: array items: type: string minLength: 1 required: - type - values title: Categorical - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - number value: type: number operator: type: string enum: - eq - ne - gt - gte - lt - lte required: - type - value - operator title: Number - type: 'null' x-speakeasy-type-override: any output_type: type: string enum: - boolean - categorical - number - string description: The type of output expected from the evaluator code: type: string type: type: string enum: - typescript_eval required: - metadata - _id - display_name - description - owner - domain_id - code - type title: Typescript - type: object properties: enabled: type: boolean default: true metadata: type: object properties: required_model_with_tools_support: type: boolean required_retrieval_context: type: boolean required_expected_output: type: boolean supported_on_input_type: type: boolean supported_on_output_type: type: boolean support_use_as_guardrail: type: boolean _id: type: string display_name: type: string description: type: string owner: type: string created: type: string default: '2026-08-04T15:14:03.981Z' updated: type: string default: '2026-08-04T15:14:03.981Z' created_by_id: type: - string - 'null' updated_by_id: type: - string - 'null' domain_id: type: string guardrail_config: anyOf: - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - boolean value: type: boolean required: - type - value title: Boolean - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - categorical values: type: array items: type: string minLength: 1 required: - type - values title: Categorical - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - number value: type: number operator: type: string enum: - eq - ne - gt - gte - lt - lte required: - type - value - operator title: Number - type: 'null' x-speakeasy-type-override: any output_type: type: string enum: - boolean type: type: string enum: - bedrock_eval guardrail_identifier: type: string minLength: 1 guardrail_version: type: string minLength: 1 default: DRAFT guardrail_region: type: string minLength: 1 integration_id: type: - string - 'null' required: - metadata - _id - display_name - description - owner - domain_id - type - guardrail_identifier title: Bedrock Guardrail '404': description: >- No evaluator with this id exists in the authenticated workspace, or the request carries no workspace. content: application/json: schema: type: object properties: message: type: string example: Evaluator not found required: - message components: securitySchemes: ApiKey: type: http scheme: bearer bearerFormat: JWT ````