Skip to main content
Use private models alongside public models through the same AI Gateway endpoint. Private models are useful when you have fine-tuned models, dedicated deployments, or self-hosted inference that should remain under your infrastructure and provider contracts.

​
Supported private model sources

Azure AI Foundry

Import OpenAI-compatible and publisher model deployments from an Azure AI Foundry project.

Google Vertex AI

Connect models deployed in your Google Cloud project using a Vertex AI service account.

LiteLLM

Import self-hosted or custom models exposed through your LiteLLM instance.

​
Add a private model

1

Open Models

In the AI Gateway sidebar, open Models and select Model.
2

Connect the private provider

Select Azure AI Foundry, Vertex AI, or LiteLLM and enter the requested endpoint and credentials.
3

Import and enable models

Import the available deployments, then enable each model that should be available through the AI Gateway.
4

Route requests

Reference an imported model as <workspace>@<provider>/<model>.
Enabled private models are available for requests through routing rules.

Manage all models

Enable models, filter by capability and region, and control model access by project.