Responses API

Use Cases

Use built-in server-side tools

Use web search and other built-in tools without client-side tool orchestration.

Retain server-side conversation state

Server-side state is retained between turns via previous_response_id, without resending the full message history.

Send multimodal input

Pass input_image and input_file items in a single request.

Build agent-style workflows

Combine function calling, reasoning, and tool continuation for agent-style loops.

The Responses API is the OpenAI-style /responses endpoint on the AI Gateway, exposed at POST /v3/router/responses. It implements the OpenResponses specification: a request carries a model and an input, and the response returns structured output items together with token and cost usage. The endpoint supports built-in tools such as web search, server-side conversation state, streaming, and multimodal input. Choose it when those capabilities matter; for a classic messages-based flow, use Chat Completions.

Invoke a configured agent by setting model to agent/<key>; the agent’s tools, knowledge bases, and memory apply automatically. See Run Agents.

Responses vs Chat Completions

Both endpoints share the same base URL (https://my.orq.ai/v3/router), authentication, and AI Gateway features: fallbacks, retries, caching, guardrails, and budgets. The table below lists the differences.

Consideration	Responses API (`POST /v3/router/responses`)	Chat Completions (`POST /v3/router/chat/completions`)
Built-in tools	Supported. Web search (`web_search`, `web_search_preview`) is Responses-only	Not supported
Conversation state	Server-side: `previous_response_id` continues from a stored response (`store` defaults to `true`)	Client-side: resend the full message history each turn
Multimodal input	`input` items: `input_text`, `input_image`, `input_file`	`messages` content: `text`, `image_url`, `file`
Reasoning controls	`reasoning` object for OpenAI models	`reasoning_effort` (OpenAI) and `thinking` (Anthropic, Google Gemini)
Streaming events	`response.output_text.delta` and other response events	`choices[0].delta.content`
Output shape	Item-based `output` array with the `output_text` helper	`choices[0].message`

Function calling works on both endpoints through the tools array; see Tool Calling.

Quick Start

Use the OpenAI SDK against the AI Gateway base URL and call client.responses.create.

curl -X POST https://my.orq.ai/v3/router/responses \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.6-sol",
    "input": "What is the capital of France?"
  }'

const response = await client.responses.create({
  model: "openai/gpt-5.6-sol",
  input: "What is the capital of France?",
});

console.log(response.output_text);

response = client.responses.create(
    model="openai/gpt-5.6-sol",
    input="What is the capital of France?",
)

print(response.output_text)

Statefulness

Responses are persisted server-side by default (store defaults to true) and can be retrieved by ID (GET /v3/router/responses/{response_id}, see Retrieve Response). Continue a conversation by passing previous_response_id on the next request; the gateway uses the stored conversation instead of requiring the full history again. previous_response_id requires store: true on the original response.

Set store: false to skip persisting a response. The response cannot be retrieved later, and previous_response_id will not work on follow-up requests.

# First turn. The response ID is returned as "id" in the response body.
curl -X POST https://my.orq.ai/v3/router/responses \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.6-sol",
    "input": "My name is Ada and I am writing a book about AI."
  }'

# Continue the conversation from the stored response.
curl -X POST https://my.orq.ai/v3/router/responses \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.6-sol",
    "previous_response_id": "resp_01KP6DFXWPKZ12AS254R4W6C08",
    "input": "What is the title of my book?"
  }'

# Retrieve the stored response by ID.
curl https://my.orq.ai/v3/router/responses/resp_01KP6DFXWPKZ12AS254R4W6C08 \
  -H "Authorization: Bearer $ORQ_API_KEY"

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.ORQ_API_KEY,
  baseURL: "https://my.orq.ai/v3/router",
});

const first = await client.responses.create({
  model: "openai/gpt-5.6-sol",
  input: "My name is Ada and I am writing a book about AI.",
});

const followUp = await client.responses.create({
  model: "openai/gpt-5.6-sol",
  previous_response_id: first.id,
  input: "What is the title of my book?",
});

const retrieved = await client.responses.retrieve(first.id);

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ.get("ORQ_API_KEY"),
    base_url="https://my.orq.ai/v3/router",
)

first = client.responses.create(
    model="openai/gpt-5.6-sol",
    input="My name is Ada and I am writing a book about AI.",
)

follow_up = client.responses.create(
    model="openai/gpt-5.6-sol",
    previous_response_id=first.id,
    input="What is the title of my book?",
)

retrieved = client.responses.retrieve(first.id)

Streaming

Set "stream": true on the request body. The server responds with a Server-Sent Events stream in the OpenAI Responses format: incremental text arrives as response.output_text.delta events, and the stream ends with a completed event carrying final usage. See Streaming for event handling, retry, and error patterns.

Telemetry

Successful responses carry the OpenTelemetry ids of the turn. The response body includes a top-level telemetry object with the trace and span ids of the request:

{
  "id": "resp_01KP6GDNJY5B0TT0R35KS23PYV",
  "status": "completed",
  "telemetry": {
    "trace_id": "9f2c1b3a4d5e6f708192a3b4c5d6e7f8",
    "span_id": "1a2b3c4d5e6f7081"
  }
}

trace_id is the 32-character hex id of the trace and span_id the 16-character hex id of the response span. The same values are echoed as the x-orq-trace-id and x-orq-trace-span-id response headers, which every router endpoint sets.

The telemetry object is present only on successful calls. Failed calls do not include it in the response body: read the ids from the x-orq-trace-id and x-orq-trace-span-id response headers instead, which are set on error responses as well.

When streaming, the ids arrive on the final response.completed event inside response.telemetry; the response.created event does not carry them. Read them from the completed event, or from the response headers. Use the ids to correlate the call with its Trace and wherever a span must be referenced, such as annotating a span or logging feedback.

Built-in tools

Built-in tools run server-side: pass them in the tools array and the gateway executes them during generation. Web search is available only through the Responses API, as web_search and web_search_preview tool types. See Web search in Responses API for tool fields, provider mapping, and the include option. Function tools work on both endpoints; see Tool Calling.

Multimodal input

Send images and PDFs alongside text in the input array using input_image and input_file items. See Image, PDF, and audio: multimodal inputs and generation for supported formats and full examples.

AI Gateway features

All AI Gateway features apply to the Responses API. Most are configured per request through request-body fields; budgets are configured in the console and apply by scope.

Feature	How it applies	Request field	Docs
Fallbacks	Try a fallback model when the primary fails	`fallbacks`	Retries
Retries	Automatic retries on configured status codes	`retry`	Retries
Response caching	Exact-match cache with TTL control	`cache`, `cache_control`	Cache
Guardrails	LLM and Python evaluators on requests and responses	`guardrails`	Guardrails
Budgets	Spending limits by workspace, project, identity, API key, provider, or model	—	Budgets
Load balancing	Distribute requests across models	`load_balancer`	Load Balancing
Plugins	PII redaction transforms on request and response text	`plugins`	PII Redaction
Trace masking	Mask request and response data in stored traces	`security`	Security
Threads	Group related requests in observability	`thread`	Thread Management

Overview

APIs

MCP Portal

Routing

Security & Privacy

Model Capabilities

Server Tools

Common Scenarios

Responses vs Chat Completions

Quick Start

Statefulness

Streaming

Telemetry

Built-in tools

Multimodal input

AI Gateway features

See also

​Responses vs Chat Completions

​Quick Start

​Statefulness

​Streaming

​Telemetry

​Built-in tools

​Multimodal input

​AI Gateway features

​See also

Responses vs Chat Completions

Quick Start

Statefulness

Streaming

Telemetry

Built-in tools

Multimodal input

AI Gateway features

See also