> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orq.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Updates from 2025

> Orq.ai product updates announced in 2025.

**August 26th, 2025**

## Introducing Evaluatorq

![Evaluatorq library interface for running parallel evaluation jobs with progress tracking.](https://mintcdn.com/orqai/W8gaV2S7D8ydczdf/images/95a2594e467cbb9ef8d9ed1f4673a4617bd8c8a238b156f0a340bf7ddcbc3320-image.png?fit=max\&auto=format\&n=W8gaV2S7D8ydczdf\&q=85\&s=991735290e75e25a942b6ea4083a88f3)

Today, we are proud to introduce Evaluatorq

An evaluation framework library that provides a flexible way to run parallel evaluations and optionally integrates with the Orq AI platform.

**Features**

* Parallel Execution: Run multiple evaluation jobs concurrently with progress tracking
* Flexible Data Sources: Support for inline data, promises, and Orq platform datasets
* Type-safe: Fully written in TypeScript
* Orq Platform Integration: Seamlessly fetch and evaluate datasets from Orq AI (optional)

**Installation**

`npm install @orq-ai/evaluatorq`

**Basic Usage**

```typescript Typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import { evaluatorq, job } from "@orq-ai/evaluatorq";

const textAnalyzer = job("text-analyzer", async (data) => {
  const text = data.inputs.text;
  const analysis = {
    length: text.length,
    wordCount: text.split(" ").length,
    uppercase: text.toUpperCase(),
  };

  return analysis;
});

await evaluatorq("text-analysis", {
  data: [
    { inputs: { text: "Hello world" } },
    { inputs: { text: "Testing evaluation" } },
  ],
  jobs: [textAnalyzer],
  evaluators: [
    {
      name: "length-check",
      scorer: async ({ output }) => {
        const passesCheck = output.length > 10;
        return {
          value: passesCheck ? 1 : 0,
          explanation: passesCheck
            ? "Output length is sufficient"
            : `Output too short (${output.length} chars, need >10)`,
        };
      },
    },
  ],
});

```

Repository: [https://github.com/orq-ai/orqkit](https://github.com/orq-ai/orqkit)

**August 20th, 2025**

## Native support for Gemini models - URL Context Tool

The new URL context tool enables models to access content directly from provided URLs, enhancing responses with external information for tasks like extracting data, comparing documents, synthesizing content, and analyzing code or documentation.

**Key Features**

* Two-step retrieval: internal index cache first, live fetch if unavailable.
* Returns url\_context\_metadata with retrieval details and safety checks.
* Supports up to 20 URLs per request, max 34MB per URL.
* Retrieved content counts toward input tokens (tracked in usage\_metadata).

**Supported Models**

* gemini-2.5-pro
* gemini-2.5-flash
* gemini-2.5-flash-lite

**Best practices**

* Use direct, specific, fully qualified URLs (https\://...).
* Avoid login-restricted or paywalled content.
* Combine with Google Search grounding for broader workflows.

**August 18th, 2025**

## OpenTelemetry Tracing Support

![OpenTelemetry tracing support showing how spans are exported to the Orq.ai OTLP collector using standard OTel SDKs.](https://mintcdn.com/orqai/W8gaV2S7D8ydczdf/images/bee740abe3e2aab4c4f38c4acf5310e79b3dfd6a2cfbef6eff9060bb2fb071d3-image.png?fit=max\&auto=format\&n=W8gaV2S7D8ydczdf\&q=85\&s=835aef9129a43ac194327e04c93e2e79)

**Key Features**

* Native OpenTelemetry (OTel) export for traces.
* Send spans directly to our OTLP/HTTP collector at `https://my.orq.ai/v2/otel`.
* Works with standard OTel SDKs and exporters (Python, Node.js).
* Supports custom headers (e.g., bearer tokens) and resource attributes (e.g., service.name).

**Quick Start**

Environment based

```
# Common env vars (both Python & Node.js)
export OTEL_EXPORTER_OTLP_ENDPOINT="https://my.orq.ai/v2/otel"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer <ORQ_API_KEY>"
export OTEL_RESOURCE_ATTRIBUTES="service.name=my-service,service.version=1.0.0"
```

Python

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
# pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http opentelemetry-instrumentation

from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

resource = Resource.create({
    "service.name": "my-python-service",
    "service.version": "1.0.0"
})

provider = TracerProvider(resource=resource)
trace.set_tracer_provider(provider)

exporter = OTLPSpanExporter(
    endpoint="https://my.orq.ai/v2/otel",
    # Optional if your setup requires auth
    headers={"Authorization": "Bearer <ORQ_API_KEY>"}
)

provider.add_span_processor(BatchSpanProcessor(exporter))

tracer = trace.get_tracer(__name__)

with tracer.start_as_current_span("demo-span"):
    # your app logic here
    pass
```

NodeJS

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import { NodeSDK } from '@opentelemetry/sdk-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { Resource } from '@opentelemetry/resources';

const traceExporter = new OTLPTraceExporter({
  url: 'https://my.orq.ai/v2/otel',
  // Optional if your setup requires auth
  headers: { Authorization: 'Bearer <ORQ_API_KEY>' },
});

const sdk = new NodeSDK({
  resource: new Resource({
    'service.name': 'my-node-service',
    'service.version': '1.0.0',
  }),
  traceExporter,
});

sdk.start();

// Example span
import { context, trace } from '@opentelemetry/api';
const tracer = trace.getTracer('example');

const span = tracer.startSpan('demo-span');
try {
  // your app logic here
} finally {
  span.end();
}
```

**August 18th, 2025**

## Knowledge API: New Parameters

New `search_type` field with options:

* `vector_search`: Semantic similarity search
* `keyword_search`: Exact text matching
* `hybrid_search`: Combined approach (default)
* Separated Top-K Controls:
  * `top_k`: Controls initial document retrieval (1-100)
  * `rerank_config.top_k`: Controls final result count after reranking (1-100)
* New agentic\_rag\_config parameter to customize AI-driven search behavior per request

**Benefits**

* Flexible Search Strategy: Choose optimal search method per query
* Fine-tuned Relevance: Retrieve broadly, return precisely with independent top-k settings
* Performance Optimization: Better control over retrieval vs. quality trade-offs

**Impact**

* Users can now choose specific search strategies instead of relying on knowledge base defaults
* Better control over the retrieval-rerank pipeline allows for more relevant results by retrieving more documents initially but returning fewer, higher-quality results

You can find more info in the [knowledge base search reference](https://docs.orq.ai/reference/knowledge-bases/search-knowledge-base.md)

**August 18th, 2025**

## Support for OpenAI's image edit and variation APIs

Image Edit and Image Variation are now supported in our [AI Proxy](https://docs.orq.ai/ai-gateway/using-the-router.md). This comes with complete trace monitoring and cost tracking.

Also, all the images are stored in our Orq with the same expirations as the retention period of your workspace.

```bash cURL theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl --location 'https://my.orq.ai/v3/router/images/edits' \
--header "Authorization: Bearer $ORQ_API_KEY" \
--form 'model="openai/gpt-image-1"' \
--form 'image=@"/path/to/file"' \
--form 'image=@"/path/to/file"' \
--form 'prompt="Add the style of card with the gradient style of the first image"'
```

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import base64
from openai import OpenAI

client = OpenAI(base_url="https://my.orq.ai/v3/router", api_key="ORQ_API_KEY")

prompt = """
Generate a photorealistic image of a gift basket on a white background
labeled 'Relax & Unwind' with a ribbon and handwriting-like font,
containing all the items in the reference pictures.
"""

result = client.images.edit(
    model="gpt-image-1",
    image=[
        open("body-lotion.png", "rb"),
        open("bath-bomb.png", "rb"),
        open("incense-kit.png", "rb"),
        open("soap.png", "rb"),
    ],
    prompt=prompt,
)

image_base64 = result.data[0].b64_json
image_bytes = base64.b64decode(image_base64)

# Save the image to a file
with open("gift-basket.png", "wb") as f:
    f.write(image_bytes)

```

```typescript Typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import fs from "fs";
import OpenAI, { toFile } from "openai";

const client = new OpenAI({
  baseURL: "https://my.orq.ai/v3/router",
  apiKey: process.env.ORQ_API_KEY
});

const imageFiles = [
    "bath-bomb.png",
    "body-lotion.png",
    "incense-kit.png",
    "soap.png",
];

const images = await Promise.all(
    imageFiles.map(async (file) =>
        await toFile(fs.createReadStream(file), null, {
            type: "image/png",
        })
    ),
);

const rsp = await client.images.edit({
    model: "gpt-image-1",
    image: images,
    prompt: "Create a lovely gift basket with these four items in it",
});

// Save the image to a file
const image_base64 = rsp.data[0].b64_json;
const image_bytes = Buffer.from(image_base64, "base64");
fs.writeFileSync("basket.png", image_bytes);

```

**August 18th, 2025**

## File Uploads for Anthropic Models

PDF support is currently supported via direct API access and Google Vertex AI on:

* Claude Opus 4.1 (claude-opus-4-1-20250805)
* Claude Opus 4 (claude-opus-4-20250514)
* Claude Sonnet 4 (claude-sonnet-4-20250514)
* Claude Sonnet 3.7 (claude-3-7-sonnet-20250219)
* Claude Haiku 3.5 (claude-3-5-haiku-20241022)

You can find more information on [attaching files](https://docs.orq.ai/ai-studio/ai-engineering/run-agents.md#attach-files)

**August 18th, 2025**

## Comprehensive Budget Controls & Cost Management

![Budget management dashboard showing workspace and contact-level spend limits with configurable reset cycles.](https://mintcdn.com/orqai/1clR3dRdy0GtV4iZ/images/fcf5aff38411309481571a64888af08d6db0dcc9af2584b32abacbc7865f852e-image.png?fit=max\&auto=format\&n=1clR3dRdy0GtV4iZ\&q=85\&s=df221065ce0171edcf693b9d540eb758)

Take full control of your AI spending with granular budget management across your entire workspace. Set limits, track consumption, and prevent cost overruns with real-time enforcement.

* Configure budgets with daily, weekly, monthly, or yearly reset cycles
* Automatic period boundary calculation with UTC timezone standardization
* Period-aligned cache TTL for optimal performance
* New Budgets API to create budgets programmatically

<video src="https://mintcdn.com/orqai/1oqtKOJj_dB5Fg2y/videos/clipboard-20250818-211353-665.mp4?fit=max&auto=format&n=1oqtKOJj_dB5Fg2y&q=85&s=5e8f5d7bd25bcfdf001b3f87477031c4" autoPlay muted loop playsInline>
  Animation showing how to configure a budget with daily, weekly, monthly, or yearly reset cycles for AI spend control.
</video>

Use Cases:

* Prevent individual users from exceeding allocated AI budgets
* Set department-level spending limits for enterprise teams
* Control costs across multiple AI models and services

> [!NOTE]
>
> For more on using **Contacts**, see [Creating a Contact](https://docs.orq.ai/ai-studio/observability/identities.md#creating-an-identity).

**August 8th, 2025**

## GPT-5 available in the AI Router

Today we’re introducing GPT‑5⁠, OpenAI’s smartest, fastest, most useful model yet in the **AI Router**.

![GPT-5 now available in the AI Router model selection.](https://mintcdn.com/orqai/kH7ELCU0voLJRri0/images/24876acb976eb7dc3bfe88a0106fb95bf4e5e87799c28886bb547ee8cb74d567-image.png?fit=max\&auto=format\&n=kH7ELCU0voLJRri0\&q=85\&s=82a5207eed4a8348f9b8a20412c7426c)

**July 10th, 2025**

## Preview Model Support

We will no longer support experimental and preview models by default in the **AI Router**.

Going forward, access to preview or experimental models will only be granted upon request. Many of these models are unreliable in production, they regularly fail or become unavailable without notice, which leads to a poor experience and extra maintenance overhead.

If you have a use case that specifically requires an experimental or preview model, please reach out to [support@orq.ai](mailto:support@orq.ai)

**July 9th, 2025**

## Three new RAGAS Evaluators for enhanced RAG Experiments

We’ve added three new RAGAS evaluators to the Orq.ai platform, giving you deeper insights into your retrieval-augmented generation (RAG) workflows. These evaluators help you understand not just if the right information was retrieved, but how well your system handles noisy data and captures key context.

**Here’s what’s new:**

### 1. Context Recall

Measure how much of the relevant reference information your retrieval pipeline actually brings into context. This evaluator compares the *retrieved text* to the *reference* for each user query, helping you identify if important facts are missing. Use context recall to improve your retrieval strategy and ensure your LLM responses are always well-supported.

**Entities used:**

* Reference (ground truth)
* Retrieved text

**When to use:**

* When you want to know if your retrieval step is actually surfacing all necessary information for the LLM to answer correctly.

[Context Recall metric docs](https://docs.ragas.io/en/latest/concepts/metrics/available_metrics/context_recall/)

***

### 2. Noise Sensitivity

Understand how your RAG system performs when irrelevant or “noisy” information is mixed into the retrieved context. This evaluator tests the robustness of your model by introducing noise and measuring the impact on your outputs. It’s a great way to benchmark reliability in real-world scenarios where retrieval isn’t always perfect.

**Entities used:**

* User message
* Reference (ground truth)
* Retrieved text (with added noise)

**When to use:**

* When you want to see if your LLM can still provide accurate answers, even when extra, unrelated information appears in the context.

[Noise Sensitivity metric docs](https://docs.ragas.io/en/latest/concepts/metrics/available_metrics/noise_sensitivity/)

***

### 3. Context Entities Recall

Check whether the most important entities (names, places, organizations, etc.) present in your *reference* are also present in the *retrieved context*. This evaluator goes beyond text overlap and focuses on the coverage of key information units, helping you catch subtle gaps in your retrieval pipeline.

**Entities used:**

* Reference (ground truth)
* Retrieved text

**When to use:**

* When you want to be sure your retrieval is capturing and surfacing all the essential entities needed to answer user queries correctly.

[Context Entities Recall metric docs](https://docs.ragas.io/en/latest/concepts/metrics/available_metrics/context_entities_recall/)

***

Use these new RAGAS evaluators to systematically assess and improve the quality, completeness, and reliability of your RAG pipelines right from within Orq.ai.

**July 9th, 2025**

## Flexible Prompt Configuration for Experiments

Configuring Experiments just got more flexible.

Previously, when setting up an Experiment, you could only use the prompt from your dataset (the `message` column), or you had to set up your prompt manually in the `prompt template`. With this update, you can now combine both approaches, giving you far more control over your Experiments.

This is especially useful if your application logs real user input from your AI features in production. Those logs often contain valuable user messages you want to test against different system prompts. With this update, you can easily combine actual user messages from your dataset with new or imported system prompts directly in your prompt template configuration. This allows you to experiment and see which prompt produces the best response, without needing to restructure your data.

This improved flexibility makes it much easier to:

* Systematically validate regression, improvements, and guardrail compliance
* Compare outputs for cost, latency, and consistency
* Run more realistic and effective experiments based on actual user input

Start combining user messages and system prompts in your Experiments to get more actionable results and deeper insights.

![Experiment configuration combining user messages from a dataset with a system prompt template for flexible testing.](https://mintcdn.com/orqai/W8gaV2S7D8ydczdf/images/9032d51aa1b4f6e431bb30d91202ed30951f88b28bf1d3fdf23adbb6bf47d134-image.png?fit=max\&auto=format\&n=W8gaV2S7D8ydczdf\&q=85\&s=02d57f33c8da275818550d38a93948fd)

**July 4th, 2025**

## Chunking API

The Chunking API is now live - the final building block for fully programmatic knowledge base workflows. With this new API, you can automatically split (“chunk”) your text data in the way that best fits your needs, all without leaving your workflow.

The Chunking API supports six different chunking strategies, each suited to a different kind of content:

* Token chunker: Splits your text based on token count. Great for keeping chunks small enough for LLMs and for consistent embedding sizes.
* Sentence chunker: Breaks your text at sentence boundaries, so each chunk stays readable and sentences remain intact.
* Recursive chunker: Chunks text by working down a hierarchy (paragraphs, then sentences, then words) to maintain document structure.
* Semantic chunker: Groups together sentences that are topically related, so each chunk makes sense on its own.
* SDPM chunker: Uses advanced skip-gram patterns to find natural split points, especially helpful for technical or structured documents.
* Agentic chunker: Uses an LLM to determine the best split points, ideal for complex documents that need intelligent segmentation.

With this release, our RAG-as-a-service stack is complete. You can create knowledge bases, upload and chunk your data (or send pre-chunked data), search and filter content with chunk-level metadata, turn on agentic RAG, and apply reranker models. The knowledge base tester in the UI lets you experiment with search settings, chunk sizes, and thresholds to optimize retrieval for your use case.

> 📘 Try the Chunking API out for yourself: the [Chunking API reference](https://docs.orq.ai/reference/chunking/parse-text.md)

**June 19th, 2025**

## Evaluators API

We’ve expanded evaluator support with two new APIs, enabling much more flexible, code-driven workflows:

* **Evaluators Hub API:** Run any evaluator on demand via API, without needing to attach it to an experiment or deployment. Ideal for integrating evaluator runs into your own pipelines or automated tools.
* **Evaluators API:** Create, retrieve, update, and delete evaluators directly via API. This allows you to fully manage your evaluators programmatically, instead of only through the UI.

**You can use both APIs together** to build and manage evaluators entirely through code, combining on-demand runs with dynamic evaluator management. This enables teams to automate their evaluation workflows, integrate with CI/CD, and adapt evaluators to changing requirements—all without leaving their own codebase.

**Key benefits:**

* **Decoupled Evaluation:** Run and manage evaluators without relying on experiments or deployments.
* **Full API Control:** Create, update, and remove evaluators as your evaluation needs evolve.
* **Pipeline Integration:** Integrate both evaluator management and execution into your own data pipelines or CI/CD workflows.
* **Faster Iteration:** Quickly test, refine, and iterate on evaluators using code.
* **Automation:** Scale evaluations and management as needed for batch operations or continuous quality monitoring.

See an example on how the new Evaluator API works with RAGAS faithfulness below

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl --request POST \
     --url https://my.orq.ai/v2/evaluators/ragas_faithfulness \
     --header 'accept: application/json' \
     --header "Authorization: Bearer $ORQ_API_KEY" \
     --header 'content-type: application/json' \
     --data '
{
  "model": "openai/gpt-4.1",
  "query": "How large can my carry on bag be?",
  "output": "Your carry-on bag must not exceed 22 x 14 x 9 inches (56 x 36 x 23 cm), including handles and wheels. There is no weight limit for carry-on bags, but you must be able to lift it into the overhead bin without assistance.",
  "retrievals": [
    "{ \"question\": \"What are the size and weight limits for carry-on bags?\", \"answer\": \"Your carry-on bag must not exceed 22 x 14 x 9 inches (56 x 36 x 23 cm), including handles and wheels. There is no weight limit for carry-on bags, but you must be able to lift it into the overhead bin without assistance.\" }",
    "{ \"question\": \"Are there bag or seasonal bag limitations?\", \"answer\": \"Learn about checked bag limitations on our website.\" }",
    "{ \"question\": \"What are your liability limitations for lost bags?\", \"answer\": \"Learn about our liability limitations and see items excluded from liability on our website.\" }",
    "{ \"question\": \"What should I do about delayed or damaged bags?\", \"answer\": \"Find out what to do about delayed or damaged bags on our website.\" }"
  ]
}
'
```

**June 19th, 2025**

## Connect LiteLLM models to Orq

You can now connect your LiteLLM-managed models directly to Orq.ai, making it even easier for teams already using LiteLLM to get started with Orq—without having to change your LLM gateway.

**Who is this for?**\
This integration is ideal for teams who have standardized on LiteLLM for unified API access, flexible model management, or cost control—but want to benefit from Orq’s full platform for everything beyond just model access.

**How to get started:**

1. Go to the **Integrations** tab, click **LiteLLM**, and enter your base URL and API keys.
2. In the **AI Router**, click **Add Model**, select **Import from LiteLLM**, and choose your model(s).

**Why use this?**

* **No migration pain:** Instantly connect your existing LiteLLM models, no rework needed.
* **Keep your infra, add new superpowers:** Keep LiteLLM as your gateway while gaining everything Orq offers for GenAI ops.
* **Best of both worlds:** For teams already invested in LiteLLM, this means you get prompt management, deployments, evaluation, and observability—on top of your existing setup.

Orq’s own gateway remains the best option for teams wanting a deeply integrated, managed experience. But for teams committed to LiteLLM, you can now take full advantage of Orq’s broader capabilities—no compromises.

![LiteLLM integration configuration in Orq.ai showing how to import models from a LiteLLM instance into the AI Router.](https://mintcdn.com/orqai/kH7ELCU0voLJRri0/images/178199123363f32b3b50d6794eb7583bc62ea8bad82106a5733fb21998b654bf-image.png?fit=max\&auto=format\&n=kH7ELCU0voLJRri0\&q=85\&s=5be2d575e5d4582e2c3619db1f3a5058)

**June 19th, 2025**

## Self onboard Vertex AI models

You can now onboard your own Vertex AI models, including fine-tuned and private models.

Go to **AI Router** > Add model > Vertex AI

![Vertex AI model onboarding form in the AI Router for adding private and fine-tuned models.](https://mintcdn.com/orqai/W8gaV2S7D8ydczdf/images/82d6c0c71eddba60bed1236ab6e57b52f18683523268a32b32e5e8103216547e-image.png?fit=max\&auto=format\&n=W8gaV2S7D8ydczdf\&q=85\&s=b91d61b7c8d1b3bd2784c59be855d172)

**June 19th, 2025**

## Custom Human Review

Today we're introducing **Custom human review**, a major update that gives teams full flexibility in collecting and structuring human feedback in Orq.ai.

With this release, you’re no longer limited to predefined review types. Now, you can **create your own feedback forms** tailored to your workflow and use case. Choose from three flexible input types:

* **Categorical:** Create button options with your own labels, such as good/bad or saved/deleted.
* **Range:** Set up a custom scoring slider (for example, a scale from 0 to 100).
* **Open field:** Allow users to enter free-form feedback, perfect for more detailed comments.

### How to collect feedback

There are two ways to collect feedback:

* Orq Studio: Custom human review templates can be used directly within the Orq.ai platform, where colleagues and domain experts review AI outputs and provide structured feedback.
* API: Alternatively, developers can leverage the Orq API to expose these custom feedback forms in their own applications or products, allowing end users to submit feedback seamlessly.

### Why this matters

Collecting high-quality, relevant human review is crucial for improving AI features. Teams can structure feedback in a way that makes sense for them, whether for quality control, prompt iteration, or continuous evaluation. All feedback is attached to the log, including prompts and outputs, making it easy for prompt engineers to refine configurations based on real-world usage.

![Custom Human Review form with Categorical, Range, and Open field input types for collecting structured feedback.](https://mintcdn.com/orqai/W8gaV2S7D8ydczdf/images/d3f08142d9153a21c01579ef6e39c0487cac5bcda059aba155bc79a683500c20-newfeedback.png?fit=max\&auto=format\&n=W8gaV2S7D8ydczdf\&q=85\&s=10e54506c22f1e9788c84982ec355a2d)

### How to create custom Human Review

<video src="https://mintcdn.com/orqai/1oqtKOJj_dB5Fg2y/videos/humanreviews.mp4?fit=max&auto=format&n=1oqtKOJj_dB5Fg2y&q=85&s=9c78c226f16905430ae6a8d654be1a05" autoPlay muted loop playsInline>
  Animation showing how to create a custom Human Review template in Orq.ai.
</video>

**June 19th, 2025**

## Evaluator playground

We’ve added a new way to quickly test your evaluators directly within the evaluator configuration screen.

Previously, testing an evaluator meant attaching it to a deployment or running an experiment, often more work than needed for a quick check. With the new Test Your Evaluator playground, you can now try out your evaluator on the spot, using custom queries and responses.

This makes it easier to iterate, validate, and fine-tune your evaluators, without leaving the configuration screen or disrupting your workflows.

<video src="https://mintcdn.com/orqai/1oqtKOJj_dB5Fg2y/videos/evalplayground2.mp4?fit=max&auto=format&n=1oqtKOJj_dB5Fg2y&q=85&s=ecf801f4025536870edcfaca76427392" autoPlay muted loop playsInline>
  Evaluator playground showing a live test run with custom query and response inputs directly in the evaluator configuration screen.
</video>

**June 2nd, 2025**

## Attach files directly to the model

With our latest update, you can now attach PDF files directly to Gemini and OpenAI models. Simply include the encoded file in your deployment invocation, and the model will natively process the PDF - no more manual file uploads or bloated prompts.

Why this matters

* **Better File Understanding**: Models can directly analyze the content and structure of your PDFs—including text, images, tables, and diagrams - improving extraction quality and structured outputs.
* **Cleaner Prompts**: File data is no longer dumped into your system or user message, resulting in faster, less error-prone requests.

The `file_data` should be using the standard data URI scheme (data:content/type;base64). See an example API call below:

```bash cURL theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl --request POST \
     --url https://my.orq.ai/v2/deployments/invoke \
     --header 'accept: application/json' \
     --header 'authorization: Bearer <your_orq_key>' \
     --header 'content-type: application/json' \
     --data '
{
  "key": "key",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "prompt"
        },
        {
          "type": "file",
          "file": {
						"filename" : "<filename>",
            "file_data": "data:application/pdf;base64,<base64-encoded-data>"
          }
        }
      ]
    }
  ]
}
'
```

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
generation = client.deployments.invoke(
    key="deployment_key",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "prompt"
                },
                {
                    "type": "file",
                    "file": {
                        "file_data": "data:application/pdf;base64,<base64-encoded-data>",
                        "filename": "filename"
                    }
                }
            ]
        }
    ],
    metadata={
        "user_id": "123",
        "session_id": "456",
    }
)
```

```typescript Typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
const generation = await client.deployments.invoke({
  key: 'deployment_key',
  messages: [
    {
      role: 'user',
      content: [
        {
          type: 'text',
          text: 'prompt'
        },
        {
          type: 'file',
          file: {
            fileData: 'data:application/pdf;base64,<base64-encoded-data>',
            filename: 'filename.pdf'
          }
        }
      ]
    }
  ],
  metadata: {
    user_id: '123',
    session_id: '456'
  }
});
```

**June 2nd, 2025**

## Evaluator runs without new generation

You can now evaluate existing LLM outputs without generating new responses. Previously, running an evaluator required creating a new output every time. With this update, you can retroactively score any response already stored in your dataset.

Why this matters

* Quality: Evaluate historical responses to ensure you’re assessing the true **source of truth**, not just new outputs.
* Flexibility: Apply evaluations to both **single responses and full conversation chains**, adapting to your specific review needs.

How it works

1. Prepare your dataset including LLM outputs in the messages column.
2. Set up an experiment and select one or more evaluators - do not select a prompt.
3. The evaluator will analyze the responses already present in the “messages” column of your dataset.

![Experiment setup showing an evaluator run on existing LLM outputs without generating new responses.](https://mintcdn.com/orqai/W8gaV2S7D8ydczdf/images/89f0d65b52c7c52375a99b63d209eccffcee95fdf42b4fb4c8533c2d315a7f34-image.png?fit=max\&auto=format\&n=W8gaV2S7D8ydczdf\&q=85\&s=435103379bfce6f5825e629912858d9c)

**June 2nd, 2025**

## Reasoning field now included in the API response

With the latest update to our reasoning models, API responses now include the `reasoning` and `reasoning_signature` fields. These additions provide more transparency into how the model arrives at its answers.

See an example below:

```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
    "id": "01JWQZ1JB6PSJ1F8KSWE5771GR",
    "created": "2025-06-02T09:19:59.308Z",
    "object": "vision",
    "model": "claude-sonnet-4-20250514",
    "provider": "anthropic",
    "is_final": true,
    "finalized": "2025-06-02T09:20:05.615Z",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "",
                "tool_calls": [
                    {
                        "type": "function",
                        "id": "toolu_01A57N7TgYAqcFmiMY9Q7TT4",
                        "index": 0,
                        "function": {
                            "name": "AA_tool",
                            "arguments": "{\"question\":\"How large can my carry on bag be?\",\"answer\":\"Your carry-on bag must not exceed 22 x 14 x 9 inches (56 x 36 x 23 cm), including handles and wheels. There is no weight limit for carry-on bags, but you must be able to lift it into the overhead bin without assistance.\"}"
                        }
                    }
                ],
                "reasoning": "The user is asking about carry-on bag size limits for American Airlines. Let me check the knowledge base for relevant information.\n\nLooking at the knowledge base, I can see there's a question that directly addresses this: \"What are the size and weight limits for carry-on bags?\" with the answer: \"Your carry-on bag must not exceed 22 x 14 x 9 inches (56 x 36 x 23 cm), including handles and wheels. There is no weight limit for carry-on bags, but you must be able to lift it into the overhead bin without assistance.\"\n\nThis question is clearly related to American Airlines travel and I have the relevant information in the knowledge base to answer it. I should use the AA_tool to structure my response.",
                "reasoning_signature": "Euxxx",
                "type": "tool_calls"
            },
            "finish_reason": "tool_use"
        }
    ],
    "provider_response": {
        "id": "msg_01VDbNW9N2khGgx16x5texpt",
        "type": "message",
        "role": "assistant",
        "model": "claude-sonnet-4-20250514",
        "content": [
            {
                "type": "thinking",
                "thinking": "The user is asking about carry-on bag size limits for American Airlines. Let me check the knowledge base for relevant information.\n\nLooking at the knowledge base, I can see there's a question that directly addresses this: \"What are the size and weight limits for carry-on bags?\" with the answer: \"Your carry-on bag must not exceed 22 x 14 x 9 inches (56 x 36 x 23 cm), including handles and wheels. There is no weight limit for carry-on bags, but you must be able to lift it into the overhead bin without assistance.\"\n\nThis question is clearly related to American Airlines travel and I have the relevant information in the knowledge base to answer it. I should use the AA_tool to structure my response.",
                "signature": "Euxxx"
            },
            {
                "type": "tool_use",
                "id": "toolu_01A57N7TgYAqcFmiMY9Q7TT4",
                "name": "AA_tool",
                "input": {
                    "question": "How large can my carry on bag be?",
                    "answer": "Your carry-on bag must not exceed 22 x 14 x 9 inches (56 x 36 x 23 cm), including handles and wheels. There is no weight limit for carry-on bags, but you must be able to lift it into the overhead bin without assistance."
                }
            }
        ],
        "stop_reason": "tool_use",
        "stop_sequence": null,
        "usage": {
            "input_tokens": 984,
            "cache_creation_input_tokens": 0,
            "cache_read_input_tokens": 0,
            "output_tokens": 312,
            "service_tier": "standard"
        }
    }
}
```

**May 30th, 2025**

## Experiment Runs overview

You can now **compare different Experiment runs in a single overview** using the new Runs tab.

Previously, reviewing the results of multiple experiment runs required clicking into each run individually and manually comparing scores. This made it harder to spot improvements or regressions at a glance.

With the new Runs overview, you get a clear, side-by-side comparison of all your experiment runs, including all your evaluator scores. This makes it much easier to see if recent changes to your prompt or model setup have led to measurable improvements, or unexpected regressions.

Example:\
In the screenshot below, you can see that both Experiment Run #1 and #2 achieved a perfect score on the JSON Schema Evaluator (100), while Cosine Similarity improved from 0.65 to 0.69.

No more clicking back and forth between runs, get instant insight into your progress in one place.

**May 30th, 2025**

## Improved Experiment configuration panel

Configuring your experiments just got a lot easier. The new side panel gives you a clear and organized overview of everything in your experiment, making setup and adjustments faster and more intuitive.

With the improved configuration panel, you can instantly see:\\

* Which prompts, models, and evaluators are included in your experiment
* The status and details of each prompt and evaluator
* A simple way to add or edit prompts and evaluators directly from the side panel

This new layout means you no longer have to dig through multiple screens or remember which components are active. Everything you need is visible at a glance, helping you stay organized and making it easier to experiment with different configurations.

**May 28th, 2025**

## Metadata filter on Knowledge Base Chunks

Knowledge Base chunks can now be enriched with metadata during creation via the Chunks API. When searching your Knowledge Base, you can filter results by metadata to target only the most relevant chunks—improving both accuracy and efficiency in retrieval.

Highlights:

* Add custom metadata to each Knowledge Base chunk at creation.
* Search using metadata filters to refine results.

## Why this matters

As your Knowledge Base grows, searching through every chunk for every query can slow things down and introduce noise in responses. By tagging each chunk with metadata—such as document type, source, department, date, or custom tags—you gain precise control over what information gets retrieved. This ensures users and AI models only access the most relevant information, leading to faster, higher-quality answers and improved user experience.

### Example

If your company stores support articles in the Knowledge Base for multiple products, you can now instantly filter retrieval to only include articles for a specific product, version, or language—removing irrelevant matches and streamlining customer support.

```bash cURL theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl --location 'https://my.orq.ai/v2/knowledge/{knowledge_id}/search' \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $ORQ_API_KEY" \
--data '{
    "query": "What were the top editions of science fiction books",
    "filter": {
        "edition": {
            "gte": 2020
        }
    },
    "search_options":{
        "include_metadata": true,
        "include_vectors": true,
        "include_scores": true
    }
}'
```

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from orq_ai_sdk import Orq
import os

client = Orq(api_key=os.getenv("ORQ_API_KEY"))

client.knowledge.search(
    knowledge_id="unique_knowledge_id",
    query={"edition": {"$gte": 2020}},
    search_options={
        "include_metadata": True,
        "include_vectors": True,
        "include_scores": True,
    },
)
```

```typescript Typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import { Orq } from '@orq-ai/node';

const orq = new Orq({
  apiKey: 'ORQ_API_KEY',
});

orq.knowledge.search({
  knowledgeId: 'unique_knowledge_id',
  query: { edition: { $gte: 2020 } },
  searchOptions: {
    includeMetadata: true,
    includeVectors: true,
    includeScores: true,
  },
});

```

> 📘 To learn more about search, see [Searching a knowledge base](https://docs.orq.ai/ai-studio/ai-engineering/knowledge-bases.md) and the related [API](https://docs.orq.ai/reference/knowledge-bases/search-knowledge-base.md).

**May 27th, 2025**

## Claude Sonnet 4 & Opus 4

With the launch of Claude Sonnet 4 and Opus 4, Anthropic has introduced its most advanced AI models yet.

Both models share the same large 200k token context window and up-to-date training data (March 2025), but they are designed for slightly different needs.

| Feature | Claude Sonnet 4 | Claude Opus 4 |
| - | - | - |
| **Context Window** | 200k tokens | 200k tokens |
| **Training Cutoff** | March 2025 | March 2025 |
| **Max Output Tokens** | 64k | 32k |
| **Input Price** | \$3 / million tokens | \$15 / million tokens |
| **Output Price** | \$15 / million tokens | \$75 / million tokens |
| **Performance Summary** | Improved reasoning and intelligence vs Sonnet 3.7 | Most capable model with superior reasoning |
| **Latency** | Moderate | Slow |
| **When to use?** | Best price/ value and capable enough for 95% use cases | Best for complex tasks needing deep analysis where speed is not important |

Claude Sonnet 4 is the best choice for most users. It offers excellent reasoning, strong tool use, and great overall performance at a much lower price. For 95% of typical use cases—including content generation, Q\&A, code review, and day-to-day tasks—Sonnet provides the best balance of speed, capability, and cost.

Claude Opus 4 is Anthropic's top-tier model. It delivers the highest level of reasoning and intelligence and outperforms other models on complex or nuanced tasks. Opus is ideal for scenarios where accuracy and depth matter more than speed or cost, such as advanced research, detailed data analysis, and orchestrating multi-step workflows. However, it is slower and significantly more expensive.

Both models represent a significant leap over previous versions, with better reasoning, improved tool use, and more robust handling of longer or more complex inputs. Choose Sonnet for value and everyday performance, and Opus for your most demanding AI workloads.

**May 8th, 2025**

## Tool response in Playground

We've enhanced the Playground experience by introducing interactive tool responses. When you're using tools (also known as function calls) within your workflow, you can now seamlessly provide the tool response back to the model, enabling continuous and efficient testing of your workflows.

**How it works**:

**Invoke Tools**: When your model calls a tool (e.g., `get_weather`), it provides the tool name and relevant arguments.

**Interactive Response**: Previously, your workflow paused here, requiring manual intervention. Now, you can directly give the tool response back to the model or use the handy `Tab` shortcut to auto-generate a suggested response.

**Continue Workflow**: With the tool response provided, the model can continue, generating a final, context-aware response.

This new capability simplifies debugging and allows you to rapidly prototype sophisticated workflows without interruption. See an example below.

**April 18th, 2025**

## OpenAI's new o3, o4-mini, GPT-4.1 models

OpenAI's latest models can be found in the **AI Router**.

## When to use what model

* o-series models are purpose-built for reasoning and tool-use. Choose o3 when raw capability beats everything else; use o4-mini when you still need strong reasoning but want 10× cheaper tokens and snappier replies.
* GPT-4.1 family is the evolution of the “generalist” GPT line. The full model gives you 1 M-token context with strong instruction following; mini trims cost/latency for mainstream workloads; nano pushes pricing and speed to the floor for simple or massive-scale tasks.

| Feature | GPT-4.1 | GPT-4.1 mini | GPT-4.1 nano | o3 | o4 mini |
| :- | :- | :- | :- | :- | :- |
| Training cutoff date | May 2024 | May 2024 | May 2024 | May 2024 | May 2024 |
| Context window | 1M | 1M | 1M | 200k | 200k |
| Max output tokens | 32,768 | 32,768 | 32,768 | 100k | 100k |
| Input price | \$2/M Tokens | \$0.40/M Tokens | \$0.10/M Tokens | \$10/M tokens | \$1.10/M tokens |
| Output price | \$8/M Tokens | \$1.60/M Tokens | \$0.40/M Tokens | \$40/M tokens | \$4.40/M tokens |
| Latency | Moderate | Fast | Very fast | Slow | Moderate |
| When to use? | Long-context chat, knowledge-work, multimodal apps needing top GPT quality | Everyday product features, prototyping & chat where speed and price count | Real-time, latency-critical or large-scale batch jobs on a tight budget | Deep research, complex multi-step reasoning, high-stakes coding/science tasks | Reasoning workloads where cost & throughput matter; API agents, math & data-science |

**April 18th, 2025**

## Rate Limits

To ensure fair usage and consistent performance, we’ve introduced rate limits for all Orq.ai APIs on a per-account basis. This helps prevent server overload, reduces risk of abuse, and keeps costs manageable.

If your account exceeds its rate limit, you’ll receive a `429 Too Many Requests` response.

Rate limits vary by subscription tier:

| Subscription | Rate Limit for Deployment API calls | Rate Limit for other API calls | Log retention |
| :- | :- | :- | :- |
| Sandbox | 100 API calls/minute | 20 API calls/minute | 3 days |
| Team (Legacy) | 1000 API calls/minute | 50 API calls/minute | 14 days |
| Pro | 2000 API calls/minute | 100 API calls/minute | 30 days |
| Enterprise | Custom | Custom | Custom |

For more details, refer to our [pricing page](https://orq.ai/pricing).

**April 18th, 2025**

## Agentic RAG

You can now enable **Agentic RAG** in your deployments to improve the relevance of retrieved context and the overall output quality.

Once toggled on, simply select a model to act as the agent. The agent will automatically check if the retrieved knowledge chunks are relevant to the user query. If they aren’t, it rewrites the query — preserving the original intent — to improve retrieval results.

This iterative refinement loop increases the chance of surfacing useful context from your Knowledge Base, giving the language model better grounding to generate high-quality, reliable responses. The setup includes two key components:

* **Document Grading Agent** – determines if relevant chunks were retrieved.
* **Query Refinement Agent** – rewrites the query if needed.

See the screenshot below on how the input query gets refined. Input query: *'is my suitcase too big?'* is reformulated to > *'luggage size requirements and restrictions for carry-on and checked baggage'.*

![Agentic RAG showing input query refinement from a vague question to a more specific retrieval query.](https://mintcdn.com/orqai/1clR3dRdy0GtV4iZ/images/e322cdafbeb496599a9744d8b464414d043ddbeed4a3aff05ad34c5f57e4a333-agenticRAGg.jpg?fit=max\&auto=format\&n=1clR3dRdy0GtV4iZ\&q=85\&s=e068228449e1329d5190a25625559649)

How to enable this? Just toggle the Agentic RAG feature on and select your model.

![Agentic RAG toggle showing model selection for the Document Grading and Query Refinement agents.](https://mintcdn.com/orqai/kH7ELCU0voLJRri0/images/2756820a1c6e316324d29574a5854a47477417d375bd7b9b8f8182cf579e8d0c-image.png?fit=max\&auto=format\&n=kH7ELCU0voLJRri0\&q=85\&s=a8707f2716ae6c62018b01a4506f5bbc)

This feature is part of our ongoing effort to help you ship more robust AI features, let us know if you have any feedback!

**April 18th, 2025**

## Threads

We’ve added a new Threads view to help you make sense of **multi-step conversations** at a glance.

Each thread captures the full **back-and-forth between the user and the assistant**, showing just the **inputs and outputs per step**—without the technical breakdowns like embeddings, rerankers, or guardrails. It’s ideal for reviewing how the conversation unfolds across multiple steps.

The **Thread Overview** includes:

* Start time and last update
* Total duration
* Number of traces (steps)
* Total cost and tokens used
* Project name and session ID
* Custom tags to help categorize threads

You can **customize** which columns are visible to tailor the overview to your needs.

To narrow things down, you can **filter threads** by:

* Project
* Tags
* Start date

This view complements the existing Traces tab, which remains the place for inspecting each individual LLM call in detail.

> 📘 To start using the Threads function add a `thread id` during the [deployment invocation](https://docs.orq.ai/reference/deployments/invoke.md), to learn more, see [Threads](https://docs.orq.ai/ai-gateway/thread-management.md).

**April 9th, 2025**

## Jina AI - new embedding and rerank models

We’ve added Jina AI as a new provider, bringing a suite of high-performance models to the platform:

* **Multilingual Embedding Models**: Use Jina AI’s embedding models—like jina-embeddings-v3, supporting 89 languages and sequences up to 8,192 tokens.
* **Efficient Rerank Models**: Choose from three reranker variations:
  * Tiny – optimized for speed and low memory use.
  * Turbo – a balance between speed and accuracy.
  * Base – best-in-class accuracy for critical use cases.

These models strengthen both multilingual coverage and reranking performance across a wide range of applications.

![Jina AI embedding and reranker model options available in Orq.ai for multilingual knowledge base search.](https://mintcdn.com/orqai/kH7ELCU0voLJRri0/images/567b835fd1771279dae11632e4b41755a38fb690d31ef8d14236c594031e7e86-jinaai.jpg?fit=max\&auto=format\&n=kH7ELCU0voLJRri0\&q=85\&s=29a0effb80c9ca04bf53861e929c0159)

**April 9th, 2025**

## Knowledge Base API

Previously, creating and managing a knowledge base required using the Orq UI. Users had to manually create a knowledge base, upload one file at a time, and define a chunking strategy per datasource. While simple, this process was time-consuming, especially for developers looking to scale or automate their workflows.

With the new Knowledge Base API, **all major operations are now available programmatically**. You can create and manage knowledge bases, upload datasources, generate or manage chunks, and perform retrieval-related actions, all through code.

This opens up much more flexibility, especially for teams working on complex chunks. You can now:\\

* **Perform chunking on your own side**, tailored to your data structure
* **Bring your own chunks**—and even include your own embeddings
* Let Orq embed the chunks if embeddings aren’t provided, using the model specified in your API call

Note: Attaching custom metadata to chunks isn’t supported yet but will be added soon.

Available operations:\\

* **Knowledge bases**: create, list, retrieve, update, delete
* **Datasources**: create, list, retrieve, update, delete
* **Chunks**: create, list, retrieve, update, delete
* **Search**: inspect what’s being retrieved

Whether you’re working with a single file or a dynamic content pipeline, this update makes the knowledge base workflow faster, more flexible, and developer-friendly.

> 📘 Read more on the Knowledge API in [our docs](https://docs.orq.ai/ai-studio/ai-engineering/knowledge-bases.md) or in the [SDKs](https://docs.orq.ai/reference/client-libraries.md)

![Knowledge Base API overview showing available programmatic operations for knowledge bases, datasources, chunks, and search.](https://mintcdn.com/orqai/1clR3dRdy0GtV4iZ/images/f759321f32c743a0b55ea56649308c8118d21a497ba437a1dc8054ccb0db94b9-image.png?fit=max\&auto=format\&n=1clR3dRdy0GtV4iZ\&q=85\&s=3ed80a9786a322b7e0a2bf31069e3b30)

**April 7th, 2025**

## Hub

We’ve added a new feature called the Hub — a library where you can browse and reuse common evaluators and prompts for your projects.

Instead of starting from a blank screen, you can now add ready-made components with one click. Some can be fully customized after adding them to your project (like prompts or classification evaluators), while others — such as traditional NLP metrics like cosine similarity — are fixed by design.

The goal: make it easier to start with off-the-shelf evals and prompts.

At launch, the Hub includes prompts and evaluators. Soon, you’ll also find datasets, tools, and other entities here to further speed up your workflow.

You can find the Hub in the left-hand menu of your workspace.

![The Hub library showing browsable ready-made evaluators and prompts available to add to a project with one click.](https://mintcdn.com/orqai/W8gaV2S7D8ydczdf/images/728fbf5559b403585f39a0d5db68d530b1d7883d643dd117a259a08ab9684e82-hubb.jpg?fit=max\&auto=format\&n=W8gaV2S7D8ydczdf\&q=85\&s=d98cb9654cbf0e3e73c0c7a4176eaaa9)

**March 18th, 2025**

## Tracing

AI workflows can feel like a black box—when something goes wrong, it’s hard to know why. **Tracing** changes that by giving you full visibility into every step of your workflow. Instead of guessing why an LLM output is wrong, you can quickly check **every step in the workflow**—saving time and reducing frustration.

With this release, you can inspect **all events** that occur within a trace, including:

* **Retrieval** – See which knowledge chunks were fetched.
* **Embedding & Reranking** – Understand how inputs are processed and prioritized.
* **LLM Calls** – Track prompts, responses, and latency.
* **Evaluation & Guardrails** – Ensure quality control in real time.
* **Cache Usage** – Spot inefficiencies in repeated queries.
* **Fallbacks & Retries** – Detect when your system auto-recovers from failures.

This level of observability helps teams **debug faster, optimize workflows, and make data-driven improvements**.

***

***

## Billing impact - Event count

With the introduction of Traces, all the events seen in the overview will count towards the number of events. This has direct impact on the billing.

An example: A chat request with 2 evaluators was historically counted as 1 request but will now be counted as 3 events.

**March 18th, 2025**

## Python Evaluators

When building AI features, ensuring high-quality and reliable outputs is crucial. Orq.ai allows you to implement custom evaluators in Python, giving you full control over how AI-generated content is assessed and validated.

## Benefits of Using Python for Evaluators

* **Flexibility & Customization**

Python enables you to define evaluation logic that precisely matches your needs, whether it’s scoring relevance, detecting biases, or enforcing style guidelines.

* **Seamless Integration**

Orq.ai supports direct integration of Python-based evaluators, allowing you to run checks on AI outputs within your workflow without extra tooling.

* **Preloaded with NumPy**

Your Python evaluators can leverage NumPy (v1.26.4) for numerical computations, making it easier to apply statistical methods, calculate custom scores, or analyze AI responses efficiently.

* **Automated & Scalable**

Python evaluators run automatically on AI responses, ensuring continuous quality control and reducing manual review efforts.

> 📘 Read here how to set up a custom Python Evaluator, see [our docs](https://docs.orq.ai/ai-studio/optimize/evaluators.md#python-evaluator)

**March 18th, 2025**

## Move entities

Previously, once an entity was created, it was locked in place—you couldn’t move it to another project or directory. Now, you finally can.

With this new feature, users can seamlessly move entities and directories to different projects and directories, ensuring better organization and flexibility in managing data.

## Key Details:

* Move entities and directories freely between projects and directories.
* Maintain project-level permissions—entities will only be visible to the selected teams assigned to that project.
* Improve workflow efficiency by keeping related entities together where they belong.

## How It Works

<video src="https://mintcdn.com/orqai/1oqtKOJj_dB5Fg2y/videos/move.mp4?fit=max&auto=format&n=1oqtKOJj_dB5Fg2y&q=85&s=bc380f83e9a0fdb77b3ded4d3aafde1f" autoPlay muted loop playsInline>
  Animation showing how to move entities and directories between projects in Orq.ai.
</video>

This update gives you more control over your workspace, making it easier to structure your data the way you need it.

Try it out and optimize your project structure today!

**February 28th, 2025**

## GPT 4.5

OpenAI has unveiled GPT-4.5, its latest and most advanced AI language model to date. Building upon the foundation of GPT-4o, GPT-4.5 offers enhanced pattern recognition, deeper world knowledge, and a more refined conversational experience. This release aims to provide users with a more intuitive and reliable AI assistant for a variety of applications.

## Why It Matters

GPT-4.5 represents a significant step forward in AI-human interaction. By reducing instances of AI "hallucinations" and improving emotional intelligence, users can expect more accurate and contextually appropriate responses. These enhancements are particularly beneficial for tasks requiring creativity, empathy, and nuanced understanding, such as content creation, customer support, and personal assistance.

## What’s New?

**Enhanced Knowledge**: GPT-4.5 has been trained on a broader dataset, resulting in a more comprehensive understanding of various topics and improved factual accuracy.

**Improved Conversational Abilities**: The model now exhibits a more natural and emotionally nuanced conversational style, making interactions feel more like engaging with a thoughtful human.

**Reduced Hallucinations**: Through advanced training techniques, GPT-4.5 significantly lowers the occurrence of generating incorrect or fabricated information, enhancing user trust.

### Additional Enhancements

* **Training Innovations**: GPT-4.5 combines traditional supervised fine-tuning with scalable alignment, where smaller models generate high-quality training data for larger models. This approach accelerates training and improves the model's ability to follow nuanced instructions.
* **Multilingual and Multimodal Support**: The model shows moderate gains in understanding multiple languages and processing various data types, making it versatile for global applications.

## Performance & Analysis Tools

* **Benchmark Evaluations**: GPT-4.5 demonstrates solid improvements over GPT-4o, especially in math (+27.4%) and science (+17.8%), making it more reliable for factual reasoning.
* **User Feedback Integration**: OpenAI encourages users to engage with GPT-4.5 and provide feedback to further refine its capabilities and address any limitations.
* **Complex reasoning**: Even though GPT-4.5 is an improvement over GPT-4.o, for more complex reasoning tasks, you should use o3-mini.

  ![GPT-4.5 model now available in the AI Router, with benchmark improvements over GPT-4o in math and science.](https://mintcdn.com/orqai/W8gaV2S7D8ydczdf/images/b84100f5a12a66614ad1acf881083cb35860c3adbed1a6b3d0dd9ef91dc5d478-image.png?fit=max\&auto=format\&n=W8gaV2S7D8ydczdf\&q=85\&s=f51791811dd8d85de8f82fb003bf9a85)

We invite all users to explore GPT-4.5 and experience its enhanced features firsthand. Your interactions and feedback are invaluable as we continue to evolve and improve our AI models. Dive in and discover how GPT-4.5 can assist you in achieving your goals more effectively.

**February 25th, 2025**

## Claude 3.7 Sonnet

Claude 3.7 Sonnet is Anthropic’s most intelligent AI model yet, introducing hybrid reasoning, fine-tuned for business use cases, and an extended 200K context window.

> 🚧 We don't return the thinking message yet, only the final output will be shown

> 📘 The model is available through Anthropic and Vertex AI

> 🚧 In AWS, the model is only available in the US region

## Thinking - Budget tokens Parameter

With the introduction of the new model, we also introduce a new parameter with it: **Thinking - budget tokens**

Using this parameter users can enable/ disable the models reasoning capabilities. Additionally, you can set how many tokens should be dedicated for its 'thinking' step. We do recommend a minimum of 1024 budget tokens. A value of 0 means the thinking is disabled. The budget tokens should never exceed the Max Tokens parameter because it still needs tokens to generate a final answer.

## Key highlights

🧠 Hybrid Reasoning Model

* First-of-its-kind AI that can provide instant responses or extended, step-by-step reasoning based on user preference.
* API users can control “thinking time” to balance speed, accuracy, and cost.

📊 Advanced Visual Data Extraction

* Improved ability to analyze charts, graphs, and complex diagrams, making it ideal for data science, analytics, and business intelligence.

🗣️ More Human-Like Chatbots

* Enhanced reasoning and natural conversation flow make it ideal for interactive AI assistants that connect data, make decisions, and take actions.

🤖 Robotic Process Automation

* Automate complex workflows and repetitive tasks with industry-leading instruction following and tool integration.

🏗️ Expanded Capabilities for Developers

* Supports a 200K context window for deep comprehension of long documents, knowledge bases, and large datasets.
* 128K output tokens (beta)—15x longer responses, enabling extensive code generation, planning, and detailed analysis.

## Benchmarks and performance

![Claude 3.7 Sonnet performance benchmark comparison chart across multiple AI evaluations.](https://mintcdn.com/orqai/kH7ELCU0voLJRri0/images/4f54f12b1df5d731e27a7979f09665775e1f80837a82442e8bbd8892b0f282a3-image.png?fit=max\&auto=format\&n=kH7ELCU0voLJRri0\&q=85\&s=8ff7550f82b923cd33fa72179ffcd85e)

Claude 3.7 Sonnet is designed for developers, businesses, and AI enthusiasts looking for an all-in-one model that balances speed, reasoning, and precision.

**February 12th, 2025**

## Increased fallbacks

We’ve enhanced our fallback and retry system to provide even greater reliability and flexibility in production use cases. Previously, Orq.ai allowed users to define a single fallback model if the primary model failed to generate a satisfactory output. Now, we’ve increased the number of fallback models to **five**, giving users even more control over model orchestration without additional coding.

## What’s New?

* Up to **Five Fallback Models** – Previously limited to one fallback model, users can now configure up to five fallback models for increased reliability.
* Flexible Fallback Configuration – The ability to define multiple fallback models allows for more advanced use cases, such as distributing API requests across different endpoints. For example:
  * Azure provides different regional deployments for models like GPT-4o. If a user is making a large number of API calls through the Sweden Central region, they can configure their first fallback to West Europe, and subsequent fallbacks to other locations if rate limits are hit.
  * If a primary model encounters rate limits, Orq can seamlessly transition requests to alternative endpoints, ensuring uninterrupted performance.

This update doesn’t change the core retry and fallback functionality that users already love—it simply expands its capabilities, making Orq.ai’s automated failover system even more powerful.

Try out the new expanded fallbacks and let us know how they enhance your workflow!

![Deployment configuration showing up to five fallback models for increased reliability.](https://mintcdn.com/orqai/kH7ELCU0voLJRri0/images/6ff469fdbb8e9669b8b82e328bfef9929ffcdc51ce1d6695d9a66cbde6870aec-image.png?fit=max\&auto=format\&n=kH7ELCU0voLJRri0\&q=85\&s=10915d73538f1793394587b34671367a)

**February 12th, 2025**

## Importing prompts

We’ve introduced a new feature that makes it easier than ever to reuse your prompts within Orq.ai. With the import functionality, you can seamlessly integrate your already created, version-controlled prompts into your Playgrounds, Experiments, and Deployments.

## What’s New?

* Seamless Importing – Users can now import their saved prompts directly into their workspace.
* Version Control – Select the specific version of a prompt to use.
* Model Configuration – You can import your prompt including the optional model config.

This feature eliminates the need to manually recreate prompts or copy paste them, making it easier to experiment and deploy with confidence.

For a quick walkthrough, check out the video below, which visually demonstrates how to import and use your prompts.

<video src="https://mintcdn.com/orqai/1oqtKOJj_dB5Fg2y/videos/import_prompts.mp4?fit=max&auto=format&n=1oqtKOJj_dB5Fg2y&q=85&s=8c44b9849f91e2146ba6fd8a720157f2" autoPlay muted loop playsInline>
  Animation showing how to import a saved prompt with version selection and model configuration into a Deployment.
</video>

**February 12th, 2025**

## Datasets v2

We've made a significant update to how datasets work in Orq.ai with the release of Datasets v2. This update **merges variable collections into datasets**, streamlining the structure and eliminating confusion between the two concepts.

## What’s Changed?

Previously, variable collections only contained input variables, while Datasets v1 stored message arrays and optional references. With Datasets v2, we now combine everything into a single entity. This provides a single source of truth, making it clearer what constitutes a dataset and simplifying how datasets are used.

## Why This Change?

* **Eliminates confusion** between datasets and variable collections.
* **Creates a unified structure** for better dataset management.
* **Ensures consistency** for experiments and historical logging.

## Why use Datasets v2?

* Upload a dataset to evaluate your use case in an [Experiment](https://docs.orq.ai/ai-studio/optimize/experiments.md)
* Save **human-curated** logs into **curated datasets**, which you can use to fine-tune your model.
* Save **bad-rated** logs, crucial for prompt engineers looking to **refine prompts and improve output quality**.

See an example of how to create a dataset below:

<video src="https://mintcdn.com/orqai/1oqtKOJj_dB5Fg2y/videos/datasetsv2.mp4?fit=max&auto=format&n=1oqtKOJj_dB5Fg2y&q=85&s=66bcd596bb1f2b84a789753bf972bef0" autoPlay muted loop playsInline>
  Animation showing how to create a Dataset in Orq.ai combining messages, input variables, and expected outputs.
</video>

**February 7th, 2025**

## Experiments v2 - Evaluate your LLM Config

We’re excited to introduce Experiments V2, a major upgrade to our Experiments module that makes testing, evaluating, and benchmarking models and prompts more intuitive and flexible than ever before.

## Why Experiments Matter

The Experiments module allows users to analyze model performance systematically without manually reviewing logs one by one. It leverages **evaluators**—such as JSON schema validators, LLM-as-a-judge, and functions like cosine similarity —to **automate the scoring process** and **assess output quality**. This ensures users can confidently measure effectiveness and make data-driven improvements.

Additionally, Experiments enable benchmarking through:

* A/B testing – Compare different prompts or model configurations to determine what works best.
* Regression testing – Detect unintended changes in outputs after modifying prompts or configurations.
* Backtesting – Assess how new setups would have performed on historical data.

## What’s New in Experiments V2?

We listened to your feedback and made significant improvements:

* **Compare Prompts Side by Side** – Users can now directly compare multiple prompts within an experiment, making it easier to test variations and find the most effective approach.
* **Intuitive and Flexible UI** – The new interface is streamlined for ease of use, making configuration and analysis smoother than ever.
* **Merged Data Sets and Variable Collections** – Previously separate, these have now been combined to simplify workflows. We appreciate all the feedback on this and have made it clearer: when setting up an experiment, users now upload a data set containing input variables, messages (prompts), and expected outputs (if evaluators require a reference).

[How to evaluate LLMs? Experiment on Orq.ai](https://www.youtube.com/watch?v=jSYIO4wuhrs)

**February 6th, 2025**

## DeepSeek Models Now Available: 67B Chat, R1, and V3

We are excited to announce the integration of DeepSeek’s latest AI models—67B Chat, R1, and V3—into our platform.

## What model should I use?

* Choose 67B Chat for fast everyday conversations
* Pick R1 when you need deeper thinking and logic for complex tasks
* Use V3 as a starting point to create your own specialized AI model

  ![DeepSeek 67B Chat, R1, and V3 models available in the Orq.ai AI Router.](https://mintcdn.com/orqai/W8gaV2S7D8ydczdf/images/b3759cf9616fa243c60b7f539cc782613153bf3e87e63fcace540ecd326e19ae-deepseek.jpg?fit=max\&auto=format\&n=W8gaV2S7D8ydczdf\&q=85\&s=2132ffc45d8ce48a518ecb4cc65b27ed)

**February 5th, 2025**

## OpenAI’s Latest Small Reasoning Model – o3-mini

Start using OpenAI’s newest and most advanced ‘small’ reasoning model: o3-mini.

![OpenAI o3-mini reasoning model now available in the Orq.ai AI Router.](https://mintcdn.com/orqai/W8gaV2S7D8ydczdf/images/814bfd14e5fd993604a020be259781f656fa096f9e89497c6f535b12c52e9c71-o3.jpg?fit=max\&auto=format\&n=W8gaV2S7D8ydczdf\&q=85\&s=edb62b9c412d65ff6af74a93e772d058)

**January 16th, 2025**

## Llama 3.3 70b & Llama Guard 3 are now available through Together AI

Experience the power of the latest Llama 3.3 70b and Llama Guard 3 models on Orq, integrated via Together AI.

## Together AI

With the deprecation of Anyscale, we've been looking for a good alternative, and that's where Together AI comes in. Together AI allows you to start using the latest Llama models with more open source models to come.

## Llama Guard 8b

In addition to our latest big release featuring [Evaluators & Guardrails](https://docs.orq.ai/ai-studio/ai-engineering/deployments.md#evaluators-and-guardrails), we also added the Llama Guard model to make your AI feature even more secure.

Llama Guard 3 is built to protect AI systems from common prompt hijacking attacks and misuse of LLMs.

* How it works: It evaluates inputs and outputs for safety, returning a simple `safe` or `unsafe` verdict.
* Key feature: When flagged as unsafe, it identifies the violated content categories, offering clear insights.
* Why use it: Ideal for organizations exposing AI features to external users, it prevents misuse and ensures secure, compliant interactions.

This model is an essential tool for maintaining the integrity of your AI systems.

## Llama 3.3 70b

For users building with Llama 3.1, you can now switch to the improved Llama 3.3 70b. You can use the same prompt as with Llama 3.1 since the prompt format remains the same. Also, the 3.3 70b model is similar in performance to the 3.1 405b model, meaning you can save cost and time by switching to the newer model.

**January 15th, 2025**

## New Layout with Project Structure

We’re introducing a new project structure UI to help you organize and manage your resources more effectively. With projects, you can group your work by use case, environment, or any logical structure that suits your needs.

## What are the benefits of the new Project Structure UI?

* Easier grouping of resources that you're working on as a team
* Less clutter when working on multiple use cases
* Better incorporation of teams and roles within a project

## Where are my old resources?

Your old resources are automatically grouped per entity of your project. For example: all your deployments within the same project are moved to the deployments folder of that project.

![New project structure layout showing resources grouped by entity type within a project folder.](https://mintcdn.com/orqai/kH7ELCU0voLJRri0/images/00ec8bb44173761f4dfeeb655e29de0327b92f18dfb77ee7ea0dfa8dae20d4ea-image.png?fit=max\&auto=format\&n=kH7ELCU0voLJRri0\&q=85\&s=0e961d571e4757d798453912f2ce9af6)

## Example structure

In the example image below, you can see that you'll have the ability to group multiple entities within the same folder. This allows you to have a production folder of a specific use case, and keep your other projects separate.

## Can I move my entities to a different project or directory?

Unfortunately not yet. We're working on this feature, but until then, make sure you are working in the right place.

## Search your entities

To find the right entity you want to work on, we added a global search function.

You can filter on project and creator, as seen in the gif below.

> 📘 To read more about our new project structure, please visit [our documentation](https://docs.orq.ai/ai-studio/get-started/projects.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.