Deployments

1. If an error code above is returned and retries are configured (retry count > 0), Orq.ai retries the Primary Model.
2. If all retry attempts fail (or no retries are configured) AND a Fallback Model is configured, Orq.ai routes to the Fallback Model.
3. If the Fallback Model also fails, the error is returned to the calling application.

• JSON Mode: the model automatically returns a valid JSON object for every generation.
• JSON Schema: define a schema that explicitly describes the fields, types, and structure of the model output.

• Text (default): variables use {{double_braces}} syntax.
• Jinja: full templating with conditionals, loops, filters, and more.
• Mustache: logic-less templating with sections.

Jinja example

1

Prompt template

Jinja

You are a support assistant for {{company_name}}.

{% if user_tier == "premium" %}
{{customer_name}} is a premium customer. Greet them by name and let them know they have priority support with a 2-hour response SLA.
{% else %}
{{customer_name}} is on the free plan. Let them know the standard response time is 24 hours.
{% endif %}

2

Template in the Studio

3

Call the deployment

Python

TypeScript

response = client.deployments.invoke(
    key="support-bot",
    inputs={
        "company_name": "Acme",
        "customer_name": "Sarah",
        "user_tier": "premium",
    }
)

4

Trace

Mustache example

1

Prompt template

Mustache

You are a support assistant for {{company_name}}.

{{! Pass is_premium: true for premium customers, false for free plan }}
{{# is_premium}}
{{customer_name}} is a premium customer. Greet them by name with priority support and a 2-hour SLA.
{{/ is_premium}}
{{^ is_premium}}
{{customer_name}} is on the free plan. Standard response time is 24 hours.
{{/ is_premium}}

2

Template in the Studio

3

Call the deployment

Python

TypeScript

response = client.deployments.invoke(
    key="support-bot",
    inputs={
        "company_name": "Acme",
        "customer_name": "Sarah",
        "is_premium": True,
    }
)

4

Trace

• Last User Message: the user’s latest message is automatically used as a query to retrieve relevant chunks.
• Query: a predefined query is used to retrieve chunks. Use Input Variables like {{query}} to make it dynamic at runtime.

• Create a new Tool: define a custom function directly within the deployment.
• Import an existing Tool: select a previously created Function tool from the resource library.

• Input evaluator: runs evaluation on the input sent to the model.
• Output evaluator: runs evaluation on the output generated by the model.

• Input Guardrail: runs before the input is sent to the model.
• Output Guardrail: runs after generation, before client response.

Each row represents a single Variant. Each column represents a single Context field. Each cell represents a Value for a Context field to be matched with a Variant.

1. Open a Deployment and go to the Routing page.
2. Right-click the target Variant and select Generate Code Snippet.

1. Send PDFs directly to the model in the invocation payload.
2. Attach a Knowledge Base to the Deployment.

curl --request POST \
     --url https://api.orq.ai/v2/deployments/invoke \
     --header 'accept: application/json' \
     --header 'authorization: Bearer <orq-api-key>' \
     --header 'content-type: application/json' \
     --data '
{
  "key": "key",
  "messages": [
    {
      "role": "user",
      "content": [
        { "type": "text", "text": "prompt" },
        {
          "type": "file",
          "file": {
            "file_data": "data:application/pdf;base64,<base64-encoded-data>"
          }
        }
      ]
    }
  ]
}
'

• Variant: select a single Variant to filter logs.
• Evaluation: Matched (a routing rule was matched) or Default Matched (no routing rule matched, default Variant was used).
• Source: API, SDK, or Simulator.