Orq.ai Documentation - AI Gateway & LLM Collaboration Platform

Create completion

curl --request POST \
  --url https://api.orq.ai/v2/router/completions \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "model": "<string>",
  "prompt": "<string>",
  "echo": false,
  "frequency_penalty": 0,
  "max_tokens": 16,
  "presence_penalty": 0,
  "seed": 123,
  "stop": "<string>",
  "temperature": 1,
  "top_p": 1,
  "n": 1,
  "user": "<string>",
  "name": "<string>",
  "fallbacks": [
    {
      "model": "<string>"
    }
  ],
  "retry": {
    "count": 3,
    "on_codes": [
      429,
      500,
      502,
      503,
      504
    ]
  },
  "cache": {
    "type": "exact_match",
    "ttl": 3600
  },
  "load_balancer": {
    "type": "weight_based",
    "models": [
      {
        "model": "openai/gpt-4o",
        "weight": 0.7
      }
    ]
  },
  "timeout": {
    "call_timeout": 30000
  },
  "orq": {
    "retry": {
      "count": 3,
      "on_codes": [
        429,
        500,
        502
      ]
    },
    "fallbacks": [
      {
        "model": "openai/gpt-5"
      },
      {
        "model": "anthropic/claude-4-opus"
      }
    ],
    "identity": {
      "id": "identity_01ARZ3NDEKTSV4RRFFQ69G5FAV",
      "display_name": "Jane Doe",
      "email": "jane.doe@example.com"
    },
    "thread": {
      "id": "thread_01ARZ3NDEKTSV4RRFFQ69G5FAV",
      "tags": [
        "customer-support"
      ]
    },
    "inputs": {
      "customer_name": "John Smith",
      "issue_type": "billing"
    },
    "cache": {
      "ttl": 3600,
      "type": "exact_match"
    },
    "knowledge_bases": [
      {
        "knowledge_id": "knowledge_01ARZ3NDEKTSV4RRFFQ69G5FAV",
        "top_k": 5
      }
    ],
    "timeout": {
      "call_timeout": 30000
    }
  },
  "stream": false
}
'

{
  "id": "<string>",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 123,
      "text": "<string>"
    }
  ],
  "model": "<string>",
  "object": "<string>",
  "created": "<string>",
  "system_fingerprint": "<string>",
  "usage": {
    "completion_tokens": 123,
    "prompt_tokens": 123,
    "total_tokens": 123,
    "prompt_tokens_details": {
      "cached_tokens": 123,
      "cache_creation_tokens": 123,
      "audio_tokens": 123
    },
    "completion_tokens_details": {
      "reasoning_tokens": 123,
      "accepted_prediction_tokens": 123,
      "rejected_prediction_tokens": 123,
      "audio_tokens": 123
    }
  }
}

POST

router

completions

Create completion

curl --request POST \
  --url https://api.orq.ai/v2/router/completions \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "model": "<string>",
  "prompt": "<string>",
  "echo": false,
  "frequency_penalty": 0,
  "max_tokens": 16,
  "presence_penalty": 0,
  "seed": 123,
  "stop": "<string>",
  "temperature": 1,
  "top_p": 1,
  "n": 1,
  "user": "<string>",
  "name": "<string>",
  "fallbacks": [
    {
      "model": "<string>"
    }
  ],
  "retry": {
    "count": 3,
    "on_codes": [
      429,
      500,
      502,
      503,
      504
    ]
  },
  "cache": {
    "type": "exact_match",
    "ttl": 3600
  },
  "load_balancer": {
    "type": "weight_based",
    "models": [
      {
        "model": "openai/gpt-4o",
        "weight": 0.7
      }
    ]
  },
  "timeout": {
    "call_timeout": 30000
  },
  "orq": {
    "retry": {
      "count": 3,
      "on_codes": [
        429,
        500,
        502
      ]
    },
    "fallbacks": [
      {
        "model": "openai/gpt-5"
      },
      {
        "model": "anthropic/claude-4-opus"
      }
    ],
    "identity": {
      "id": "identity_01ARZ3NDEKTSV4RRFFQ69G5FAV",
      "display_name": "Jane Doe",
      "email": "jane.doe@example.com"
    },
    "thread": {
      "id": "thread_01ARZ3NDEKTSV4RRFFQ69G5FAV",
      "tags": [
        "customer-support"
      ]
    },
    "inputs": {
      "customer_name": "John Smith",
      "issue_type": "billing"
    },
    "cache": {
      "ttl": 3600,
      "type": "exact_match"
    },
    "knowledge_bases": [
      {
        "knowledge_id": "knowledge_01ARZ3NDEKTSV4RRFFQ69G5FAV",
        "top_k": 5
      }
    ],
    "timeout": {
      "call_timeout": 30000
    }
  },
  "stream": false
}
'

{
  "id": "<string>",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 123,
      "text": "<string>"
    }
  ],
  "model": "<string>",
  "object": "<string>",
  "created": "<string>",
  "system_fingerprint": "<string>",
  "usage": {
    "completion_tokens": 123,
    "prompt_tokens": 123,
    "total_tokens": 123,
    "prompt_tokens_details": {
      "cached_tokens": 123,
      "cache_creation_tokens": 123,
      "audio_tokens": 123
    },
    "completion_tokens_details": {
      "reasoning_tokens": 123,
      "accepted_prediction_tokens": 123,
      "rejected_prediction_tokens": 123,
      "audio_tokens": 123
    }
  }
}

Authorizations

Authorization

string

header

required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json

model

string

required

ID of the model to use

prompt

string

required

The prompt(s) to generate completions for, encoded as a string, array of strings, array of tokens, or array of token arrays.

echo

boolean | null

default:false

Echo back the prompt in addition to the completion

frequency_penalty

number | null

default:0

Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

max_tokens

number | null

default:16

The maximum number of tokens that can be generated in the completion.

presence_penalty

number | null

default:0

Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

Required range: -2 <= x <= 2

seed

number | null

If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result.

stop

Up to 4 sequences where the API will stop generating further tokens. The returned text will not contain the stop sequence.

temperature

number | null

default:1

What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.

top_p

number | null

default:1

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

number | null

default:1

How many completions to generate for each prompt. Note: Because this parameter generates many completions, it can quickly consume your token quota.

user

string

A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse.

name

string

The name to display on the trace. If not specified, the default system name will be used.

fallbacks

object[]

Array of fallback models to use if primary model fails

Show child attributes

retry

object

Retry configuration for the request

Show child attributes

cache

object

Cache configuration for the request.

Show child attributes

load_balancer

object

Load balancer configuration for the request.

Show child attributes

timeout

object

Timeout configuration to apply to the request. If the request exceeds the timeout, it will be retried or fallback to the next model if configured.

Show child attributes

orq

object

deprecated

Leverage Orq's intelligent routing capabilities to enhance your AI application with enterprise-grade reliability and observability. Orq provides automatic request management including retries on failures, model fallbacks for high availability, identity-level analytics tracking, conversation threading, and dynamic prompt templating with variable substitution.

Show child attributes

Example:

{
  "retry": { "count": 3, "on_codes": [429, 500, 502] },
  "fallbacks": [
    { "model": "openai/gpt-5" },
    { "model": "anthropic/claude-4-opus" }
  ],
  "identity": {
    "id": "identity_01ARZ3NDEKTSV4RRFFQ69G5FAV",
    "display_name": "Jane Doe",
    "email": "jane.doe@example.com"
  },
  "thread": {
    "id": "thread_01ARZ3NDEKTSV4RRFFQ69G5FAV",
    "tags": ["customer-support"]
  },
  "inputs": {
    "customer_name": "John Smith",
    "issue_type": "billing"
  },
  "cache": { "ttl": 3600, "type": "exact_match" },
  "knowledge_bases": [
    {
      "knowledge_id": "knowledge_01ARZ3NDEKTSV4RRFFQ69G5FAV",
      "top_k": 5
    }
  ],
  "timeout": { "call_timeout": 30000 }
}

stream

boolean

default:false

Response

Represents a completion response from the API.

string

required

A unique identifier for the completion.

choices

object[]

required

The list of completion choices the model generated for the input prompt.

Show child attributes

model

string

required

The model used for the chat completion.

object

string

required

The object type

created

The Unix timestamp (in seconds) of when the chat completion was created.

system_fingerprint

string

This fingerprint represents the backend configuration that the model runs with.

usage

object

Usage statistics for the completion request.

Show child attributes

Create response

Create embeddings

AI & Execution

Access & Security

AI Router Features

API Reference

Create completion

Authorizations

Body

Response