Skip to main content
Use private models alongside public models through the same AI Gateway endpoint. Private models are useful when you have fine-tuned models, dedicated deployments, or self-hosted inference that should remain under your infrastructure and provider contracts.

​
Supported private model sources

OpenAI Compatible

Add models from any endpoint that implements the OpenAI API, including self-hosted and private deployments.

Azure AI Foundry

Import OpenAI-compatible and publisher model deployments from an Azure AI Foundry project.

AWS Bedrock

Add models using Amazon Bedrock inference profiles.

LiteLLM

Import self-hosted or custom models exposed through your LiteLLM instance.

​
Add a private model

1

Open Models

In the AI Gateway sidebar, open Models and select Model.
2

Connect the private provider

Select OpenAI Compatible, Azure, AWS Bedrock, or Import from LiteLLM, then follow the prompts for that source.
3

Import and enable models

Import or add the models to expose, then enable each model that should be available through the AI Gateway.
4

Route requests

Reference an imported model as <workspace>@<provider>/<model>.
Enabled private models are available for requests through routing rules.

Manage all models

Enable models, filter by capability and region, and control model access by project.

​
OpenAI Compatible

Choose this source for any endpoint that implements the OpenAI API, including self-hosted inference servers and private gateways.
OpenAI Compatible configuration form showing Connection fields for Type, Base URL, Model ID and API Key, General fields for model name, region, description, token limits and per-token prices, and Capabilities toggles for function calling, vision, structured output and reasoning

OpenAI Compatible configuration form

For the configuration fields, see OpenAI-compatible models.