> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orq.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Guardrails

> Create LLM-as-a-Judge and Python guardrails in the AI Gateway to validate requests and responses and block non-compliant generations.

**Guardrails** in the **AI Gateway** are LLM-as-a-Judge and Python evaluators that validate requests and responses passing through the gateway. Once created, they can be attached to [Guardrail Rules](/ai-gateway/configuration/guardrail-rules) to block non-compliant generations before they reach the caller.

<CardGroup cols={2}>
  <Card title="LLM Guardrail" icon="robot" href="#llm-guardrail">
    Use a model to judge outputs against any criteria defined in a prompt.
  </Card>

  <Card title="Python Guardrail" icon="python" href="#python-guardrail">
    Write custom Python code for full flexibility: regex checks, length validation, HTTP calls, or JSON schema validation.
  </Card>
</CardGroup>

## Execution behavior

Set each Guardrail to **Input**, **Output**, or **Both** when adding it to a [Guardrail Rule](/ai-gateway/configuration/guardrail-rules).

Input Guardrails, including Secret Detection, run synchronously on the complete input before the **AI Gateway** sends the request to the model. Streaming does not change this behavior.

Output Guardrails run after generation and before the response for non-streaming requests.

<Warning>
  Output Guardrails do not run on streaming responses. A secret can span multiple chunks, so a chunk-by-chunk check cannot inspect the complete output. Use a non-streaming request when output enforcement is required.
</Warning>

## Timeout

On the [Responses endpoint](/reference/sdk/responses), each entry in a request's `guardrails[]` array can set its own `timeout` in milliseconds. It bounds that one entry's execution and is independent of the request-level `call_timeout`, which bounds the model call only. See [Request timeouts](/ai-gateway/features/timeouts).

An entry runs as a blocking Guardrail when `is_guardrail` is `true`, and as a monitoring Evaluator otherwise.

| Field          | Type    | Required | Description                                                                                                                             |
| -------------- | ------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `id`           | string  | Yes      | Key of a published Guardrail or Evaluator.                                                                                              |
| `execute_on`   | string  | Yes      | `input`, `output`, or `both`. An entry with an unrecognized value is skipped without an error.                                          |
| `is_guardrail` | boolean | No       | `true` runs the check as a blocking Guardrail, `false` as a monitoring Evaluator. Defaults to `false`.                                  |
| `sample_rate`  | number  | No       | Fraction in `[0,1]` of matching requests to run on. Ignored for a Guardrail, which always runs; a monitoring Evaluator defaults to `0`. |
| `timeout`      | integer | No       | Execution limit in milliseconds. Defaults to `60000` (60 seconds); values above `600000` (10 minutes) are capped at 10 minutes.         |
| `options`      | object  | No       | Guardrail-specific configuration, for example `language` and `threshold` for PII Detection.                                             |

```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "model": "openai/gpt-5.6-sol",
  "timeout": { "call_timeout": 30000 },
  "guardrails": [
    { "id": "orq_pii_detection", "execute_on": "input", "is_guardrail": true, "timeout": 5000 },
    { "id": "response-quality", "execute_on": "output", "is_guardrail": true, "timeout": 15000 }
  ]
}
```

On the Responses endpoint and the OpenAI-compatible chat completions endpoint, a Guardrail that exceeds its `timeout` fails the request with HTTP `408` and `error.code` `guardrail_timeout`, alongside the other Guardrail failures. See [Guardrail Error Response](/ai-studio/optimize/evaluators#guardrail-error-response).

The OpenAI-compatible chat completions endpoint accepts the same `timeout` on each entry of its `guardrails[]` array, but rejects values outside `1000`-`600000` milliseconds with `400` rather than using them verbatim.

<Note>
  Guardrails attached through a [Guardrail Rule](/ai-gateway/configuration/guardrail-rules) cannot set a per-guardrail timeout and always run with the 60 second default.
</Note>

## LLM Guardrail

LLM Guardrails use a model to judge requests or responses against criteria defined in a prompt.

Navigate to **Guardrails** in the **AI Gateway** sidebar, click <kbd className="key">+ Guardrail</kbd>, and select **LLM**. Fill in the following fields:

| Field           | Description                                                                                               |
| --------------- | --------------------------------------------------------------------------------------------------------- |
| **Key**         | Unique identifier for the guardrail                                                                       |
| **Description** | Optional context                                                                                          |
| **Model**       | The model used as judge. Any model enabled in the [AI Gateway](/ai-gateway/using-the-router) is available |

### Configure Prompt

Reference the evaluated run with the following **string** variables. Type `{{` in the prompt editor to pick one from the full list.

| Variable                        | Description                                                            |
| ------------------------------- | ---------------------------------------------------------------------- |
| `{{input.user_query}}`          | The last message sent to the model                                     |
| `{{input.all_messages}}`        | The full conversation, including the graded turn                       |
| `{{input.system_instructions}}` | The system prompt used for the run                                     |
| `{{input.retrievals}}`          | [Knowledge Base](/ai-studio/ai-engineering/knowledge-bases) retrievals |
| `{{input.expected_output}}`     | The reference used to compare output                                   |
| `{{output.response}}`           | The output response generated by the evaluated model                   |
| `{{output.tools_called}}`       | The tool calls made during the run, with their results                 |

See [Structured variable shapes](/ai-studio/optimize/evaluators#structured-variable-shapes) for the fields available when indexing messages and tool calls.

<Accordion title="Legacy log variables" icon="clock-rotate-left">
  The `log.*` variables remain supported, so existing Guardrails keep working. Prefer the variables above for new ones.

  | Variable             | Description                                                                      |
  | -------------------- | -------------------------------------------------------------------------------- |
  | `{{log.input}}`      | Same as `{{input.user_query}}`                                                   |
  | `{{log.output}}`     | Same as `{{output.response}}`                                                    |
  | `{{log.retrievals}}` | Same as `{{input.retrievals}}`                                                   |
  | `{{log.reference}}`  | Same as `{{input.expected_output}}`                                              |
  | `{{log.tool_calls}}` | Same as `{{output.tools_called}}`                                                |
  | `{{log.messages}}`   | The conversation **without** the graded turn. `{{input.all_messages}}` keeps it. |
</Accordion>

### Output and Guardrail Configuration

Select the output type and set the pass condition. The **Guardrail configuration** panel is visible directly in the settings.

<Tabs>
  <Tab title="Boolean" icon="bars">
    The model returns a **True** or **False** response. Use for binary pass/fail checks.

    **Pass condition**: Select **True** or **False**. The guardrail passes when the model returns the selected value.
  </Tab>

  <Tab title="Number" icon="hashtag">
    The model returns a numeric score. Use any scale that fits the use case (e.g. 1-5, 0-100).

    **Pass condition**: Enter a threshold in **Pass if greater or equal than**. The guardrail passes when the score meets or exceeds the threshold.
  </Tab>

  <Tab title="Categorical" icon="grid-2">
    The model classifies the output into one of the predefined labels.

    When **Categorical** is selected, a label editor appears. Add one label per row: enter a **Value** (the exact string the model must return) and an optional **Description** to guide the model.

    **Pass condition**: Select one or more values in **Pass if output is one of**. The guardrail passes when the model's output matches any selected label.
  </Tab>

  <Tab title="String" icon="font">
    The model returns a free-form string response. Not available as a guardrail pass condition.
  </Tab>
</Tabs>

### Testing

The **Playground** panel provides an **Editor** for testing. Fill the payload manually:

```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "messages": [],
  "input": "",
  "retrievals": [],
  "output": "",
  "reference": ""
}
```

Click **Run test** to execute the guardrail. The result appears in the **Response** field.

<Note>
  The Dataset tab is not available for Guardrails.
</Note>

## Python Guardrail

Python Guardrails accept custom **Python code** for full evaluation flexibility. The UI has three panels: **Settings**, **Code**, and **Playground**.

<Note>
  Python code is limited to 1 MB (1,048,576 bytes) per guardrail: roughly 1 million characters, or about 20,000 lines of typical Python. Larger code returns a `Code exceeds maximum size` error and does not run.
</Note>

Navigate to **Guardrails** in the **AI Gateway** sidebar, click <kbd className="key">+ Guardrail</kbd>, and select **Python**.

The evaluation function receives a `log` object with the following fields:

* `log["input"]` `<str>`: the last message sent to generate the output
* `log["output"]` `<str>`: the generated response from the model
* `log["reference"]` `<str>`: the reference used to compare the output
* `log["messages"]` `list<str>`: all previous messages sent to the model
* `log["retrievals"]` `list<str>`: all [Knowledge Base](/ai-studio/ai-engineering/knowledge-bases) retrievals

The function must return a **Boolean** or **Number**:

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
def evaluate(log):
    return True
```

<Info>
  Define multiple helper functions if needed. The last defined function is the entry point when the guardrail runs.
</Info>

### Environment and Libraries

The Python Guardrail runs in **Python 3.12** with the following preloaded libraries:

```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
numpy==1.26.4
nltk==3.9.1
requests
pydantic
json
re
```

### Guardrail Configuration

Set the pass condition based on the return type:

* **Boolean**: select **True** or **False**. The guardrail passes when the function returns the selected value.
* **Number**: enter a score threshold. The guardrail passes when the return value is greater than or equal to the threshold.

### Testing

The **Playground** panel provides an **Editor** for testing. Fill the payload manually and click **Run test** to execute the guardrail.

<Note>
  The Dataset tab is not available for Guardrails.
</Note>

## Versions

Click <kbd className="key">Publish</kbd> to save changes. Choose a version bump:

* **Patch** (e.g. `v1.0.0` → `v1.0.1`): small fixes, no behavior change
* **Minor** (e.g. `v1.0.0` → `v1.1.0`): new functionality, backwards compatible
* **Major** (e.g. `v1.0.0` → `v2.0.0`): breaking change or significant rework

The **Versions** tab shows the full history with author and publish timestamp for each version.

### Restoring a Version

Click the compare icon on any version in the **Versions** tab to open **Compare**, then click <kbd className="key">Restore</kbd> next to an older version to load it into the current working draft.

Restore does not publish automatically: the guardrail is loaded into the draft as unpublished changes on the **Settings** tab, and <kbd className="key">Publish</kbd> still needs to be clicked for it to become a real version. Earlier versions are never deleted, so restoring is always reversible.

<Note>
  If there are unpublished changes already, a confirmation dialog asks for confirmation before overwriting them.
</Note>

## Using Guardrails

Guardrails created here are available for selection when configuring [Guardrail Rules](/ai-gateway/configuration/guardrail-rules). To block on a decision from a policy engine or a third-party provider such as Guardrails AI or Lakera, see [External Guardrails](/ai-gateway/configuration/external-guardrails).