- Analyzing uploaded images or PDFs without a separate preprocessing pipeline.
- Generating images from text prompts through the same API and key.
- Transcribing audio recordings or voice inputs for downstream processing.
- Extracting structured data from scanned documents, receipts, or screenshots.
The AI Gateway supports all input and output modalities through a single OpenAI-compatible API, including the Responses API. All endpoints share the same base URL, authentication, and AI Gateway features: fallbacks, caching, load balancing, and retries.
All endpoints use the same base URL and authentication:
Image input
Analyze images alongside text. Pass image URLs or base64-encoded files in
chat/completions messages.PDF input
Send PDF documents for extraction and analysis. Supported natively by compatible models.
Image generation
Generate, edit, and vary images using DALL-E 2, DALL-E 3, and GPT Image 1.
Audio
Convert text to speech, transcribe audio files, and translate audio to English.
Image input
Analyze images alongside text usingPOST /v3/router/responses or POST /v3/router/chat/completions. Pass images as public URLs or base64-encoded data in the message content array. For the file lifecycle, see the Files API.
Supported formats
Detail levels
Set
detail in the image_url object:
Patterns
Multiple images:Limitations
PDF input
Send PDF documents directly in messages for analysis and content extraction usingPOST /v3/router/responses or POST /v3/router/chat/completions.
PDF input support varies by model. See the Supported Models page and check your provider’s documentation for PDF capability. For the file lifecycle and for turning uploaded documents into knowledge base datasources, see the Files API.
input_file and file content part fields are specified once, in Sending files to models.
Use cases
Limitations
Image generation
Generate images from a text prompt usingPOST /v3/router/images/generations.
For the full and up-to-date list of supported image models, see Image models on the Supported Models page.
Parameters
Set
response_format to url to receive a hosted image link, or b64_json to receive the image inline as a base64-encoded string.
Streaming image generation (
stream: true) has these constraints:nmust be1response_formatis not supportedsizemust be1024x1024,1024x1536,1536x1024, orauto
Image editing
Modify an existing image using a prompt and an optional mask withPOST /v3/router/images/edits.
Image variations
Generate variations of an existing image withPOST /v3/router/images/variations. See Image models for which models support variations.
Fallbacks and reliability
Image endpoints support the samefallbacks and retry parameters as chat completions:
TypeScript
Audio
The AI Gateway exposes three OpenAI-compatible audio endpoints. All support fallbacks, load balancing, and retries.Text to speech
Convert text to audio usingPOST /v3/router/audio/speech.
For the full and up-to-date list of TTS models, see Text-to-Speech models on the Supported Models page.
Voices:
Transcription
Transcribe an audio file to text usingPOST /v3/router/audio/transcriptions.
For the full and up-to-date list of transcription models, see Speech-to-Text models on the Supported Models page.
Translation
The OpenAI translation endpoint only supports
openai/whisper-1. gpt-4o-transcribe and gpt-4o-mini-transcribe do not support translation.POST /v3/router/audio/translations. The output is always in English regardless of the source language.
response_format and temperature parameters as transcription.