> ## Documentation Index > Fetch the complete documentation index at: https://docs.orq.ai/llms.txt > Use this file to discover all available pages before exploring further. # Migrate to Orq.ai from OpenAI, OpenRouter, or LiteLLM > Move existing LLM traffic to the Orq.ai AI Gateway from the OpenAI SDK, OpenRouter, or LiteLLM by changing the base URL, the API key, and the model name. The **AI Gateway** uses the OpenAI API format. Migrating an existing application means changing three values: the base URL, the API key, and the model name. Request bodies, response bodies, and streaming stay the same. Tool calling is unchanged for most models, with one exception noted below. ## What changes | Item | Change to | | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Base URL | `https://api.orq.ai/v3/router`, the same for every organization, workspace, and project | | Authorization header | `Bearer $ORQ_API_KEY` | | Model name | `provider/model`, for example `openai/gpt-5.6-sol`. Always include the prefix: some models are served by more than one provider, and the prefix selects between them | The base URL does not vary by region or by workspace. Data stored in the **Orq.ai** platform resides in the European Union, while the region a model is served from depends on the model and is filterable on the **Models** page. See [Sovereign AI](/docs/enterprise/sovereign-ai). A private model carries its workspace in the model name as `@/` rather than in the URL. See [Private Models](/docs/ai-gateway/private-models). ## What works immediately Once those three values are in place, the following need no further code: * **Cost per call**: every request records input, output, and total cost. * **Traces**: every request is recorded with its latency, token counts, and the model that answered. * **Model access**: every model enabled on the [Models page](/docs/ai-gateway/using-the-router) is reachable from the same client, whichever provider serves it. Browse the catalog in [Supported Models](/docs/ai-gateway/supported-models). * **Routing Rules**: route or restrict traffic from the **AI Gateway** interface, without touching the application. See [Routing Rules](/docs/ai-gateway/configuration/routing-rules). **Fallbacks**, **Retries**, **Cache**, and **Guardrails** are opt-in fields on the request body, covered in [What Orq.ai adds](#what-orq-ai-adds). ## What does not carry over Two areas do not transfer. | Area | Detail | | ----------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Assistants API and fine-tuning | Not available. `/assistants` and `/fine_tuning/jobs` return `404`. Applications built on those endpoints cannot migrate as they are. See [OpenAI-Compatible API](/docs/ai-gateway/features/openai-compatible-api) for the supported endpoint list. | | Tool calling with OpenAI reasoning models | Sending `tools` to `/chat/completions` with a reasoning model such as `openai/gpt-5.6-sol` returns `400`. Use `/responses` instead. Setting `reasoning_effort` to `none` also unblocks `/chat/completions`, at the cost of the reasoning the model would otherwise do. Other models, including `openai/gpt-4o-mini` and `anthropic/claude-sonnet-5`, accept `tools` on `/chat/completions` normally. | ## Before starting Complete the [Quick Start](/docs/ai-gateway/get-started/introduction) first. A migration needs an API key, and every model the application calls has to be enabled on the [Models page](/docs/ai-gateway/using-the-router), not just the one used for the first request. Access to those models comes from [**Credits**](/docs/ai-studio/organization/credits) or [**BYOK**](/docs/ai-gateway/providers-overview). Export the key before starting. The steps below read it, and so does the coding agent when it checks model names against the catalog. ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} export ORQ_API_KEY="your-api-key" ``` ## Migrate with a coding agent Paste the prompt below into a coding agent such as **Claude Code**, **Cursor**, or **Codex**. The agent finds the LLM calls in the repository and rewrites them. The prompt does the following: 1. Points every LLM client at `https://api.orq.ai/v3/router`. 2. Replaces the old API key variable with `ORQ_API_KEY`, in code and in environment files. 3. Rewrites model names to the `provider/model` format. 4. Converts routing settings from the previous gateway instead of deleting them. 5. Checks every model name against the catalog, and leaves a call site untouched rather than substituting a model or half-migrating it. 6. Leaves request and response handling untouched. Start from a clean git working tree, so every change is visible in `git diff` and revertible with `git checkout .`. Review the diff before running the application. Migrating by hand instead? Skip to [Migrate from the OpenAI SDK](#migrate-from-the-openai-sdk). ```text Migration prompt theme={"theme":{"light":"github-light","dark":"github-dark"}} Migrate this project to the orq.ai AI Gateway. Make the smallest change that works. Do not restructure the application. Step 1. Find every place the project calls an LLM. Check source files, configuration files, environment files, and container definitions. Step 2. Identify the current setup, then apply the matching rule: - OpenAI SDK with no custom base URL: add the base URL below and change the API key variable. - OpenRouter (base URL contains openrouter.ai): replace the base URL and the API key variable, then convert the OpenRouter settings using Step 3. - LiteLLM proxy (base URL points at a self-hosted proxy): replace the base URL and the API key variable. Replace each model alias with the full provider/model name, read from litellm_params.model in the LiteLLM config file. - LiteLLM Python library used directly: keep the library and set api_base to the base URL below, and prefix each model with openai/, giving openai//. LiteLLM strips that openai/ prefix before forwarding, so the gateway receives /. Do not replace the library. Step 3. Convert OpenRouter settings. Do not delete them silently, because they carry routing behavior: - models array with route: "fallback" becomes fallbacks: [{"model": "..."}]. The first entry of the models array repeats the top-level model field, so leave it out and list only the entries after it. Including it makes the request fall back to the model that just failed. - provider preferences: remove every sub-option from the request and report each one by name. There is no drop-in request-body equivalent. Choosing between models per request maps to the load_balancer field, and organization-wide routing maps to Routing Rules in the orq.ai interface. Do not write either one, because both need a human decision. - plugins: remove the field and report every entry by name. orq.ai uses the same field name with different accepted id values, and an unrecognized id makes the whole request fail with a 400, so keeping the array breaks the call site. Do not write a replacement, because the equivalent needs a human decision. - debug: remove. - Headers HTTP-Referer, X-Title, X-OpenRouter-Title, X-OpenRouter-Categories: remove and report each one. Do not write a replacement. The application name maps to the name field, which a human can add later. Step 4. Apply these values: - Base URL: https://api.orq.ai/v3/router - Auth: Authorization: Bearer $ORQ_API_KEY - Environment variable: ORQ_API_KEY. Do not add fallback variables. Rename the variable in code, in committed templates such as .env.example, and in container definitions. In templates, leave the value empty. Remove an old provider or gateway variable only when nothing in this repository still reads it, and list every variable removed. Never read, print, or copy a secret value. If a file holds real secrets, report its path instead of editing it. - Model format: provider/model, for example openai/gpt-5.6-sol or anthropic/claude-sonnet-5. Step 5. Model names. Never substitute a different model version, because cost and behavior differ. A newer version is not an equivalent, and a similar name carrying a different identifier is not the same model. Check every name against the catalog: curl https://api.orq.ai/v3/router/models -H "Authorization: Bearer $ORQ_API_KEY" Match names exactly. A punctuation difference or an added date suffix makes it a different model. At a LiteLLM Python library call site, look up only the part after the openai/ prefix, because that is the name the gateway receives. Take the provider segment from the code where it is stated: a client configured against one specific provider, or a config entry that names it. Where the code does not state it, a provider may be proposed, but only keep the result if that exact provider/model appears in the catalog, and report the inference. Never keep a provider that the catalog does not confirm. Flag a model instead of changing it when the name is absent from the catalog, when the catalog cannot be reached, or when no candidate provider/model can be confirmed against the catalog. Step 6. Leave every call site that has a flagged model completely unchanged, including its base URL, its API key variable, and any environment variable only that call site still uses. Migrating the rest of such a call site produces a request that fails at runtime. Report each one as blocked on a decision. This overrides Step 3: report the settings on a blocked call site instead of converting them. Where one shared client object serves both migrated and blocked calls, leave that client as it is and add a separate client for the migrated calls, rather than repointing the one the blocked call still depends on. Step 7. Remove configuration that existed only to run the previous gateway, such as a LiteLLM config file and its proxy service definition. Keep anything a call site blocked in Step 6 still needs. List every file removed. Step 8. Do not change message content, response handling, streaming logic, or tool-calling logic. The API format is identical. The only body changes permitted are the field conversions in Step 3. Step 9. Report three lists: - model names changed, as old name and new name - model names flagged for a human decision, and the call sites left unmigrated because of them - settings that moved to the orq.ai interface instead of code Show the full diff. Do not commit. ``` Model names are not guaranteed to match between gateways. Check every name the agent reports against [Supported Models](/docs/ai-gateway/supported-models) before running the application in production. A `404` means either the name is wrong or the model is not enabled on the [Models page](/docs/ai-gateway/using-the-router). `fallbacks` does not cover this case, because the model is resolved before routing runs. ## Migrate from the OpenAI SDK Change the base URL and the API key. Add the provider prefix to the model name. Both `/chat/completions` and `/responses` are available, so keep whichever the application already uses. **Before** ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}} from openai import OpenAI import os client = OpenAI( api_key=os.environ.get("OPENAI_API_KEY"), ) response = client.responses.create( model="gpt-5.6-sol", input="Hello!", ) ``` ```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}} import OpenAI from "openai"; const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY, }); const response = await client.responses.create({ model: "gpt-5.6-sol", input: "Hello!", }); ``` **After** ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}} from openai import OpenAI import os client = OpenAI( base_url="https://api.orq.ai/v3/router", api_key=os.environ.get("ORQ_API_KEY"), ) response = client.responses.create( model="openai/gpt-5.6-sol", input="Hello!", ) ``` ```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}} import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://api.orq.ai/v3/router", apiKey: process.env.ORQ_API_KEY, }); const response = await client.responses.create({ model: "openai/gpt-5.6-sol", input: "Hello!", }); ``` Models from every other provider now work through the same client. See [Supported Models](/docs/ai-gateway/supported-models). ## Migrate from OpenRouter **OpenRouter** and the **AI Gateway** both use the `provider/model` naming format, so most model names stay the same. The catalogs differ, so confirm every name against `GET /models` or the **Models** page before switching. **Before** ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}} from openai import OpenAI import os client = OpenAI( base_url="https://openrouter.ai/api/v1", api_key=os.environ.get("OPENROUTER_API_KEY"), ) response = client.chat.completions.create( model="openai/gpt-4o-mini", messages=[{"role": "user", "content": "Hello!"}], ) ``` ```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}} import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://openrouter.ai/api/v1", apiKey: process.env.OPENROUTER_API_KEY, }); const response = await client.chat.completions.create({ model: "openai/gpt-4o-mini", messages: [{ role: "user", content: "Hello!" }], }); ``` **After** ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}} from openai import OpenAI import os client = OpenAI( base_url="https://api.orq.ai/v3/router", api_key=os.environ.get("ORQ_API_KEY"), ) response = client.chat.completions.create( model="openai/gpt-4o-mini", messages=[{"role": "user", "content": "Hello!"}], ) ``` ```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}} import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://api.orq.ai/v3/router", apiKey: process.env.ORQ_API_KEY, }); const response = await client.chat.completions.create({ model: "openai/gpt-4o-mini", messages: [{ role: "user", content: "Hello!" }], }); ``` ### Move OpenRouter settings across Most old **OpenRouter** fields are ignored rather than rejected, so a partial migration does not fail. The routing they configured no longer applies. Move each one deliberately. `plugins` is the exception. Both gateways use that field name, and the **AI Gateway** reads it. A leftover **OpenRouter** `plugins` array fails the whole call with a generic `Invalid request body` that does not name the field. | OpenRouter setting | **Orq.ai** equivalent | | ---------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `models` array with `route: "fallback"` | [`fallbacks`](/docs/ai-gateway/features/retries#fallbacks), listing the entries after the first. The first repeats `model` | | `provider.order`, `provider.only`, `provider.ignore` | [`load_balancer`](/docs/ai-gateway/features/load-balancing) to choose between models per request, or [Routing Rules](/docs/ai-gateway/configuration/routing-rules) to restrict routing for the whole organization | | `provider.allow_fallbacks` | [`fallbacks`](/docs/ai-gateway/features/retries#fallbacks), which is explicit rather than automatic | | `provider.data_collection`, `provider.zdr` | Zero data retention is a property of the model. Filter for it on the **Models** page. See [Sovereign AI](/docs/enterprise/sovereign-ai) | | `provider.sort` | [`load_balancer`](/docs/ai-gateway/features/load-balancing) with latency-based selection | | `plugins` array | [`plugins`](/docs/ai-gateway/features/plugins/overview): same field name, different accepted `id` values. Remove the **OpenRouter** array rather than leaving it, because an unrecognized `id` returns `400` | | `HTTP-Referer`, `X-Title`, `X-OpenRouter-Title`, `X-OpenRouter-Categories` headers | [`name`](/docs/ai-gateway/app-tracking) for the application name, [`tags`](/docs/ai-gateway/orq-attributes#custom-metadata-and-attribution) for filterable labels | The remaining `provider` sub-options have no per-request equivalent: `require_parameters`, `quantizations`, `enforce_distillable_text`, `preferred_min_throughput`, `preferred_max_latency`, and `max_price`. Applications that depend on any of these need their routing decided ahead of the request, by selecting models explicitly or through [Routing Rules](/docs/ai-gateway/configuration/routing-rules). ## Migrate from LiteLLM **LiteLLM Proxy** is a server run inside the organization's own infrastructure. Applications call it instead of calling providers directly, and it forwards each request to the real provider. It exposes an OpenAI-compatible endpoint, on port 4000 by default. There are two ways to move off it. Pick one before making any change. | Option | What happens | Tradeoff | | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Retire the proxy** (covered below) | The application calls the **AI Gateway** directly and the **LiteLLM** server is shut down | Every model name in the application has to be rewritten | | **Keep the proxy behind Orq.ai** | The **LiteLLM** instance is connected as a provider and its models are imported. Each name keeps its nickname, prefixed as `@litellm/`, so no real model has to be looked up | The **LiteLLM** server stays in the stack, and every model name still needs the prefix added. See [LiteLLM custom model provider](/docs/ai-studio/integrations/providers/litellm) | The rest of this section covers the first option. ### Find the real model name **LiteLLM Proxy** does not use provider model names. It uses nicknames defined in its `config.yaml`. Each entry pairs the nickname the application calls with the real model behind it: ```yaml config.yaml theme={"theme":{"light":"github-light","dark":"github-dark"}} model_list: - model_name: fast-model # the nickname the application sends litellm_params: model: openai/gpt-5.6-sol # the real model it forwards to ``` The application above sends `fast-model`. The **AI Gateway** has no such name, so replace it with the real model, `openai/gpt-5.6-sol`. Open `config.yaml` and read the `litellm_params.model` value for every nickname the application uses. A nickname can look exactly like a real model name. `model_name: gpt-4o` can point at `litellm_params.model: azure/gpt-4o-eu`, which is a different model on a different provider. Read `litellm_params.model` for every entry, including the ones that already look correct. ### Change the client **Before**, pointing at the proxy and calling it by its nickname: ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}} from openai import OpenAI import os client = OpenAI( base_url="http://localhost:4000", api_key=os.environ.get("LITELLM_API_KEY"), ) response = client.chat.completions.create( model="fast-model", messages=[{"role": "user", "content": "Hello!"}], ) ``` ```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}} import OpenAI from "openai"; const client = new OpenAI({ baseURL: "http://localhost:4000", apiKey: process.env.LITELLM_API_KEY, }); const response = await client.chat.completions.create({ model: "fast-model", messages: [{ role: "user", content: "Hello!" }], }); ``` **After**, pointing at the **AI Gateway** and calling the real model: ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}} from openai import OpenAI import os client = OpenAI( base_url="https://api.orq.ai/v3/router", api_key=os.environ.get("ORQ_API_KEY"), ) response = client.chat.completions.create( model="openai/gpt-5.6-sol", messages=[{"role": "user", "content": "Hello!"}], ) ``` ```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}} import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://api.orq.ai/v3/router", apiKey: process.env.ORQ_API_KEY, }); const response = await client.chat.completions.create({ model: "openai/gpt-5.6-sol", messages: [{ role: "user", content: "Hello!" }], }); ``` Routing behavior set in `config.yaml`, such as fallbacks and retries, moves to the request body. See [What Orq.ai adds](#what-orq-ai-adds). To keep **LiteLLM** as the caller and send its traces to **Orq.ai** instead, see [LiteLLM observability](/docs/ai-gateway/integrations/frameworks/litellm). ## Verify the migration ```bash cURL theme={"theme":{"light":"github-light","dark":"github-dark"}} curl -X POST https://api.orq.ai/v3/router/responses \ -H "Authorization: Bearer $ORQ_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "openai/gpt-5.6-sol", "input": "Confirm the migration works." }' ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}} from openai import OpenAI import os client = OpenAI( base_url="https://api.orq.ai/v3/router", api_key=os.environ.get("ORQ_API_KEY"), ) response = client.responses.create( model="openai/gpt-5.6-sol", input="Confirm the migration works.", ) print(response.output_text) ``` ```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}} import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://api.orq.ai/v3/router", apiKey: process.env.ORQ_API_KEY, }); const response = await client.responses.create({ model: "openai/gpt-5.6-sol", input: "Confirm the migration works.", }); console.log(response.output_text); ``` A `200` response confirms that the base URL, the API key, and the model name are correct. For an application that uses chat completions, replace `/responses` with `/chat/completions` and send `messages` instead of `input`. Open [**Traces**](/docs/ai-gateway/traces) in the **AI Gateway** and open the newest request. Confirm all four: | Check | Expected | If it does not match | | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------- | | Total cost | Greater than zero | The model has no pricing attached, or the provider connection is missing. Check the model on the [Models page](/docs/ai-gateway/using-the-router) | | Model that answered | The same model that was requested, shown without the `provider/` prefix. Requesting `openai/gpt-5.6-sol` reports `gpt-5.6-sol` | A different model means a fallback or a [Routing Rule](/docs/ai-gateway/configuration/routing-rules) redirected the call | | Input and output tokens | Both greater than zero | The request or the response was empty | | Status | Completed, no error | Inspect the provider error on the trace | If no request appears at all, the application is still calling the old endpoint. Search the repository for the previous base URL and the previous key variable. Repeat for every service that calls an LLM. Filter [**Traces**](/docs/ai-gateway/traces) by model to confirm that every model the application uses now appears, and to find traffic that has not moved yet. ## What Orq.ai adds Once traffic is flowing, these optional fields go on the same request body. | Field | What it does | Reference | | --------------- | ---------------------------------------------------------------------------- | --------------------------------------------------------------------------- | | `fallbacks` | Tries another model when the first one fails | [Fallbacks](/docs/ai-gateway/features/retries#fallbacks) | | `retry` | Repeats a failed request, with a configurable count and error codes | [Retries](/docs/ai-gateway/features/retries) | | `timeout` | Stops waiting for a provider after a set number of milliseconds | [Timeouts](/docs/ai-gateway/features/timeouts) | | `load_balancer` | Splits traffic across several models by weight or latency | [Load Balancing](/docs/ai-gateway/features/load-balancing) | | `cache` | Returns a stored answer for a repeated identical request | [Cache](/docs/ai-gateway/features/cache) | | `guardrails` | Checks the input or the output, and blocks the request when a check fails | [Guardrails](/docs/ai-gateway/configuration/guardrails) | | `plugins` | Removes personal data before the provider sees it, or repairs malformed JSON | [Plugins](/docs/ai-gateway/features/plugins/overview) | | `security` | Masks request and response data in stored traces | [Security](/docs/ai-gateway/features/security) | | `name` | Names the calling application on the trace | [App Tracking](/docs/ai-gateway/app-tracking) | | `tags` | Labels a request so it can be filtered later | [Metadata](/docs/ai-gateway/orq-attributes#custom-metadata-and-attribution) | | `thread` | Groups the turns of one conversation | [Threads](/docs/ai-gateway/thread-management) | | `identity` | Assigns a request, and its cost, to one end user | [Identities](/docs/ai-studio/observability/identities) | Set in the **Orq.ai** interface rather than the request body: [Budgets](/docs/ai-gateway/budgets) for spend limits and alerts, [Private Models](/docs/ai-gateway/private-models) for self-hosted or fine-tuned models, and [Sovereign AI](/docs/enterprise/sovereign-ai) for data residency and zero-retention providers. Adding a fallback and a cache to an existing call: ```bash cURL theme={"theme":{"light":"github-light","dark":"github-dark"}} curl -X POST https://api.orq.ai/v3/router/chat/completions \ -H "Authorization: Bearer $ORQ_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "openai/gpt-5.6-sol", "messages": [{ "role": "user", "content": "Hello!" }], "fallbacks": [{ "model": "anthropic/claude-sonnet-5" }], "cache": { "type": "exact_match", "ttl": 3600 } }' ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}} from openai import OpenAI import os client = OpenAI( base_url="https://api.orq.ai/v3/router", api_key=os.environ.get("ORQ_API_KEY"), ) response = client.chat.completions.create( model="openai/gpt-5.6-sol", messages=[{"role": "user", "content": "Hello!"}], extra_body={ "fallbacks": [{"model": "anthropic/claude-sonnet-5"}], "cache": {"type": "exact_match", "ttl": 3600}, }, ) ``` ```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}} import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://api.orq.ai/v3/router", apiKey: process.env.ORQ_API_KEY, }); const orqFields = { fallbacks: [{ model: "anthropic/claude-sonnet-5" }], cache: { type: "exact_match", ttl: 3600 }, }; const response = await client.chat.completions.create({ model: "openai/gpt-5.6-sol", messages: [{ role: "user", content: "Hello!" }], ...orqFields, }); ``` ## Next steps Keep requests flowing when a provider fails. Browse the model catalog and provider capabilities. Review the full list of compatible endpoints. Inspect cost, latency, and token usage per request.