Embeddings

Use Cases

Semantic search and retrieval over a collection of documents.
Clustering, classification, and anomaly detection on text.
Ingesting vectors into a custom or third-party vector database.
Building RAG pipelines where retrieval happens outside the gateway.

Overview

The AI Gateway exposes POST /embeddings on the OpenAI-compatible API base URL (https://my.orq.ai/v3/router). Send text and receive vector embeddings from any enabled embedding model, with the same request and response format as the OpenAI Embeddings API. Fallbacks, budgets, caching, and observability apply to embedding calls exactly as they do to chat completions.

Quick Start

curl -X POST https://my.orq.ai/v3/router/embeddings \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/text-embedding-3-small",
    "input": "The food was delicious"
  }'

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.ORQ_API_KEY,
  baseURL: "https://my.orq.ai/v3/router",
});

const embedding = await client.embeddings.create({
  model: "openai/text-embedding-3-small",
  input: "The food was delicious",
});

console.log(embedding.data[0].embedding);

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ.get("ORQ_API_KEY"),
    base_url="https://my.orq.ai/v3/router",
)

embedding = client.embeddings.create(
    model="openai/text-embedding-3-small",
    input="The food was delicious",
)

print(embedding.data[0].embedding)

Providers and Models

Providers that support embeddings include AWS Bedrock, Azure, Cohere, Vertex AI, Google AI, Jina AI, Mistral, Nebius, OpenAI, Scaleway, Tencent, and TensorX. See Supported Models for the full embedding model catalog.

Model Selection and Dimensions

Choose a model based on which languages it supports, cost, and output quality. For content in multiple languages, use a multilingual model such as cohere/embed-multilingual-v3.0 or jina/jina-embeddings-v3. For English-only content, openai/text-embedding-3-small is smaller and cheaper. Pass dimensions to request a specific number of output dimensions:

{
  "model": "openai/text-embedding-3-small",
  "input": "The food was delicious",
  "dimensions": 256
}

Models that support configurable output sizes (for example openai/text-embedding-3-small and openai/text-embedding-3-large) return vectors of the requested size, which reduces storage and compute cost with a small loss in retrieval quality. Models with a fixed output size return vectors of their fixed dimension.

Set encoding_format to base64 to receive embeddings as base64-encoded strings instead of JSON float arrays, which makes the response smaller:

{
  "model": "openai/text-embedding-3-small",
  "input": "The food was delicious",
  "encoding_format": "base64"
}

Batching and Token Usage

The input field accepts a single string, an array of strings, or token arrays. Batch related texts in one request instead of sending one request per text. One round trip is faster than many:

curl -X POST https://my.orq.ai/v3/router/embeddings \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/text-embedding-3-small",
    "input": [
      "The food was delicious",
      "And the waiter was friendly"
    ]
  }'

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.ORQ_API_KEY,
  baseURL: "https://my.orq.ai/v3/router",
});

const batch = await client.embeddings.create({
  model: "openai/text-embedding-3-small",
  input: [
    "The food was delicious",
    "And the waiter was friendly",
  ],
});

console.log(batch.data.map((item) => item.embedding.length));

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ.get("ORQ_API_KEY"),
    base_url="https://my.orq.ai/v3/router",
)

batch = client.embeddings.create(
    model="openai/text-embedding-3-small",
    input=[
        "The food was delicious",
        "And the waiter was friendly",
    ],
)

print([len(item.embedding) for item in batch.data])

The response returns one embedding per input in the data array, ordered to match the input. Token usage follows the OpenAI format: usage.prompt_tokens reports the input tokens consumed and usage.total_tokens the total for the request.

Embeddings and Knowledge Bases

Knowledge Bases handle embedding and retrieval internally: chunk the source documents, embed them with a configured embedding model, and inject the retrieved context into the model call. Prefer a Knowledge Base when the goal is RAG inside the gateway, because it handles ingestion and retrieval automatically. Call the embeddings endpoint directly when the pipeline needs control that a Knowledge Base does not offer:

Storing vectors in an external vector database such as Pinecone or a custom vector DB.
Using custom chunking, a custom embedding model, or embeddings for non-retrieval tasks such as clustering and classification.
Running retrieval outside the gateway while keeping generation inside it.

Fallbacks, Caching, Budgets, and Observability

Embedding calls support the same gateway controls as chat completions:

Fallbacks: pass an array of fallbacks to route to backup models when the primary model fails. See Fallbacks.
Caching: identical embedding requests can be served from cache. See Cache.
Budgets: embedding traffic counts against Budgets, with limits applied per workspace, API key, or provider.
Observability: embedding calls appear in Traces with model, token usage, and latency. Pass the top-level name field, for example "name": "semantic-search-embed", to label the call on the trace.

Overview

APIs

MCP Portal

Routing

Security & Privacy

Model Capabilities

Server Tools

Common Scenarios

Overview

Quick Start

Providers and Models

Model Selection and Dimensions

Batching and Token Usage

Embeddings and Knowledge Bases

Fallbacks, Caching, Budgets, and Observability

​Overview

​Quick Start

​Providers and Models

​Model Selection and Dimensions

​Batching and Token Usage

​Embeddings and Knowledge Bases

​Fallbacks, Caching, Budgets, and Observability

Overview

Quick Start

Providers and Models

Model Selection and Dimensions

Batching and Token Usage

Embeddings and Knowledge Bases

Fallbacks, Caching, Budgets, and Observability