Responses API

Use Cases

Use built-in server-side tools

Use web search and other built-in tools without client-side tool orchestration.

Retain server-side conversation state

Server-side state is retained between turns via previous_response_id, without resending the full message history.

Send multimodal input

Pass input_image and input_file items in a single request.

Build agent-style workflows

Combine function calling, reasoning, and tool continuation for agent-style loops.

The Responses API is the OpenAI-style /responses endpoint on the AI Gateway, exposed at POST /v3/router/responses. It implements the OpenResponses specification: a request carries a model and an input, and the response returns structured output items together with token and cost usage. The endpoint supports built-in tools such as web search, server-side conversation state, streaming, and multimodal input. Choose it when those capabilities matter; for a classic messages-based flow, use Chat Completions.

Invoke a configured agent by setting model to agent/<key>; the agent’s tools, knowledge bases, and memory apply automatically. See Run Agents.

Responses vs Chat Completions

Both endpoints share the same base URL (https://api.orq.ai/v3/router), authentication, and AI Gateway features: fallbacks, retries, caching, guardrails, and budgets. The table below lists the differences.

Consideration	Responses API (`POST /v3/router/responses`)	Chat Completions (`POST /v3/router/chat/completions`)
Built-in tools	Supported. Web search (`web_search`, `web_search_preview`) is Responses-only	Not supported
Conversation state	Server-side: `previous_response_id` continues from a stored response (`store` defaults to `true`)	Client-side: resend the full message history each turn
Multimodal input	`input` items: `input_text`, `input_image`, `input_file`	`messages` content: `text`, `image_url`, `file`
Reasoning controls	`reasoning` object for OpenAI models	`reasoning_effort` (OpenAI) and `thinking` (Anthropic, Google Gemini)
Streaming events	`response.output_text.delta` and other response events	`choices[0].delta.content`
Output shape	Item-based `output` array with the `output_text` helper	`choices[0].message`

Function calling works on both endpoints through the tools array; see Tool Calling.

Quick Start

Use the OpenAI SDK against the AI Gateway base URL and call client.responses.create.

curl -X POST https://api.orq.ai/v3/router/responses \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.6-sol",
    "input": "What is the capital of France?"
  }'

const response = await client.responses.create({
  model: "openai/gpt-5.6-sol",
  input: "What is the capital of France?",
});

console.log(response.output_text);

response = client.responses.create(
    model="openai/gpt-5.6-sol",
    input="What is the capital of France?",
)

print(response.output_text)

Statefulness

Responses are persisted server-side by default (store defaults to true) and can be retrieved by ID (GET /v3/router/responses/{response_id}, see Retrieve Response). Continue a conversation by passing previous_response_id on the next request; the gateway uses the stored conversation instead of requiring the full history again. previous_response_id requires store: true on the original response.

Set store: false to skip persisting a response. The response cannot be retrieved later, and previous_response_id will not work on follow-up requests.

# First turn. The response ID is returned as "id" in the response body.
curl -X POST https://api.orq.ai/v3/router/responses \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.6-sol",
    "input": "My name is Ada and I am writing a book about AI."
  }'

# Continue the conversation from the stored response.
curl -X POST https://api.orq.ai/v3/router/responses \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.6-sol",
    "previous_response_id": "resp_01KP6DFXWPKZ12AS254R4W6C08",
    "input": "What is the title of my book?"
  }'

# Retrieve the stored response by ID.
curl https://api.orq.ai/v3/router/responses/resp_01KP6DFXWPKZ12AS254R4W6C08 \
  -H "Authorization: Bearer $ORQ_API_KEY"

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.ORQ_API_KEY,
  baseURL: "https://api.orq.ai/v3/router",
});

const first = await client.responses.create({
  model: "openai/gpt-5.6-sol",
  input: "My name is Ada and I am writing a book about AI.",
});

const followUp = await client.responses.create({
  model: "openai/gpt-5.6-sol",
  previous_response_id: first.id,
  input: "What is the title of my book?",
});

const retrieved = await client.responses.retrieve(first.id);

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ.get("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router",
)

first = client.responses.create(
    model="openai/gpt-5.6-sol",
    input="My name is Ada and I am writing a book about AI.",
)

follow_up = client.responses.create(
    model="openai/gpt-5.6-sol",
    previous_response_id=first.id,
    input="What is the title of my book?",
)

retrieved = client.responses.retrieve(first.id)

Streaming

Set "stream": true on the request body. The server responds with a Server-Sent Events stream in the OpenAI Responses format: incremental text arrives as response.output_text.delta events, and the stream ends with a completed event carrying final usage. See Streaming for event handling, retry, and error patterns.

Built-in tools

Built-in tools run server-side: pass them in the tools array and the gateway executes them during generation. Web search is available only through the Responses API, as web_search and web_search_preview tool types. See Web search in Responses API for tool fields, provider mapping, and the include option. Function tools work on both endpoints; see Tool Calling.

Multimodal input

Send images and PDFs alongside text in the input array using input_image and input_file items. See Multimodal inputs and generation for supported formats and full examples.

AI Gateway features

All AI Gateway features apply to the Responses API. Most are configured per request through request-body fields; budgets are configured in the console and apply by scope.

Feature	How it applies	Request field	Docs
Fallbacks	Try a fallback model when the primary fails	`fallbacks`	Retries
Retries	Automatic retries on configured status codes	`retry`	Retries
Response caching	Exact-match cache with TTL control	`cache`, `cache_control`	Cache
Guardrails	LLM and Python evaluators on requests and responses	`guardrails`	Guardrails
Budgets	Spending limits by workspace, project, identity, API key, provider, or model	—	Budgets
Load balancing	Distribute requests across models	`load_balancer`	Load Balancing
Plugins	PII redaction transforms on request and response text	`plugins`	PII Redaction
Trace masking	Mask request and response data in stored traces	`security`	Security
Threads	Group related requests in observability	`thread`	Thread Management

Overview

APIs

Routing

Security & Privacy

Model Capabilities

Analytics & Monitoring

Integrations

Responses vs Chat Completions

Quick Start

Statefulness

Streaming

Built-in tools

Multimodal input

AI Gateway features

See also

​Responses vs Chat Completions

​Quick Start

​Statefulness

​Streaming

​Built-in tools

​Multimodal input

​AI Gateway features

​See also

Responses vs Chat Completions

Quick Start

Statefulness

Streaming

Built-in tools

Multimodal input

AI Gateway features

See also