Prompt caching for reduced token costs

Use Cases

Reusing long system prompts across many requests to cut input token costs.
Referencing large documents or codebases without re-sending them every call.
Multi-turn conversations with a large, stable context that doesn’t change between turns.
RAG pipelines where the same retrieved context is shared across many user queries.

Overview

Prompt Caching is a provider-level feature that caches prompts so that repeated requests are charged at a reduced rate. This is most effective when your requests share a large, stable prefix:

a long system prompt.
a reference document.
a tool definition list. Unlike Response Caching, which serves a stored response for identical requests, Prompt Caching still calls the model on every request, at a reduced cost. Both can be used together.

How caching is enabled and what gets cached varies by provider. See the provider sections below.

Anthropic

Prompt caching on Anthropic models requires explicit opt-in via cache_control markers on individual message parts. A breakpoint marks the end of a cacheable prefix: the provider caches everything up to and including the marked block, and later requests that share that prefix read the cached portion instead of reprocessing it. On the Anthropic-compatible endpoint, breakpoints pass through exactly as in the native Anthropic API; see the Anthropic Messages API for endpoint details and a multi-turn example.

Supported models

All current Claude models, including Claude Fable 5 and the Opus, Sonnet, and Haiku families.

Enabling caching

Add a cache_control object to any message part you want to mark as cacheable:

{
  "cache_control": { "type": "ephemeral" }
}

"ephemeral" is the only supported type. You can place it on:

System message text parts.
User message text parts.
User message images, documents, and files (including PDFs).
Tool definitions.
Tool result content.

Minimum token thresholds

Caching only activates once the marked content exceeds a minimum token count. Requests below the threshold are processed normally at full cost.

Model	Minimum tokens
Claude Opus 5, Fable 5	512
Claude Opus 4.8, Sonnet 5, Sonnet 4.6, Sonnet 4.5, Opus 4.1, Opus 4, Sonnet 4, Sonnet 3.7	1,024
Claude Opus 4.7, Haiku 3.5, Haiku 3	2,048
Claude Opus 4.6, Opus 4.5, Haiku 4.5	4,096

Cache TTL

The ttl parameter controls how long cached content persists before expiring.

Value	Duration
`"5m"` (default)	5 minutes from last use
`"1h"`	1 hour

{
  "cache_control": {
    "type": "ephemeral",
    "ttl": "1h"
  }
}

Cache reads are billed at a fraction of the regular input token price. Cache writes carry a premium: 1.25x the input price for the 5-minute TTL and 2x for the 1-hour TTL. Caching pays off from the second request onward on a shared prefix.

Example

curl -X POST https://my.orq.ai/v3/router/responses \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-5",
    "input": [
      {
        "role": "system",
        "content": [
          {
            "type": "input_text",
            "text": "You are a senior legal assistant. The following is our complete contract template library...",
            "cache_control": { "type": "ephemeral" }
          }
        ]
      },
      {
        "role": "user",
        "content": "Summarize clause 7 of the NDA template."
      }
    ]
  }'

curl -X POST https://my.orq.ai/v3/router/chat/completions \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-5",
    "messages": [
      {
        "role": "system",
        "content": [
          {
            "type": "text",
            "text": "You are a senior legal assistant. The following is our complete contract template library...",
            "cache_control": { "type": "ephemeral" }
          }
        ]
      },
      {
        "role": "user",
        "content": "Summarize clause 7 of the NDA template."
      }
    ]
  }'

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.ORQ_API_KEY,
  baseURL: "https://my.orq.ai/v3/router",
});

const response = await client.responses.create({
  model: "anthropic/claude-sonnet-5",
  input: [
    {
      role: "system",
      content: [
        {
          type: "input_text",
          text: "You are a senior legal assistant. The following is our complete contract template library...",
          cache_control: { type: "ephemeral" },
        },
      ],
    },
    { role: "user", content: "Summarize clause 7 of the NDA template." },
  ],
});

console.log(response.output_text);

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ.get("ORQ_API_KEY"),
    base_url="https://my.orq.ai/v3/router",
)

response = client.responses.create(
    model="anthropic/claude-sonnet-5",
    input=[
        {
            "role": "system",
            "content": [
                {
                    "type": "input_text",
                    "text": "You are a senior legal assistant. The following is our complete contract template library...",
                    "cache_control": {"type": "ephemeral"},
                }
            ],
        },
        {"role": "user", "content": "Summarize clause 7 of the NDA template."},
    ],
)

print(response.output_text)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.ORQ_API_KEY,
  baseURL: "https://my.orq.ai/v3/router",
});

const response = await client.chat.completions.create({
  model: "anthropic/claude-sonnet-5",
  messages: [
    {
      role: "system",
      content: [
        {
          type: "text",
          text: "You are a senior legal assistant. The following is our complete contract template library...",
          cache_control: { type: "ephemeral" },
        },
      ],
    },
    { role: "user", content: "Summarize clause 7 of the NDA template." },
  ],
});

console.log(response.choices[0].message.content);

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ.get("ORQ_API_KEY"),
    base_url="https://my.orq.ai/v3/router",
)

response = client.chat.completions.create(
    model="anthropic/claude-sonnet-5",
    messages=[
        {
            "role": "system",
            "content": [
                {
                    "type": "text",
                    "text": "You are a senior legal assistant. The following is our complete contract template library...",
                    "cache_control": {"type": "ephemeral"},
                }
            ],
        },
        {"role": "user", "content": "Summarize clause 7 of the NDA template."},
    ],
)

print(response.choices[0].message.content)

OpenAI

Prompt caching on OpenAI models is fully automatic. No cache_control or any request changes are required. The AI Gateway forwards requests normally; OpenAI caches the prompt prefix on its side and applies the discount transparently. Caching activates on prompts longer than 1,024 tokens, in 128-token increments from that threshold. The API caches the longest matching prefix from prior requests on the same machine. Cache retention duration is model-dependent and determined by OpenAI. Refer to OpenAI’s prompt caching documentation for the current retention policy per model. Cache hits are reflected in the response usage object the same way as Anthropic. See Usage in the response below.

OpenAI

Set up your OpenAI API key to use GPT models with automatic prompt caching.

Google Gemini

Google Gemini supports two caching modes through the AI Gateway. Implicit caching is enabled by default on Gemini 2.5 and newer models. No request changes are needed. The AI Gateway forwards requests normally and Google applies the cache discount automatically when a matching prefix exists. Implicit caching activates at a model-dependent minimum: 2,048 tokens on Gemini 2.5 models, higher on newer generations. Verify current thresholds and rates in Google’s caching documentation. Explicit caching is managed by the AI Gateway on supported Gemini models (Gemini 2.5 and newer). Add cache_control markers to system or user message parts (text and files), using the same syntax as on Anthropic models. The gateway creates a Google cache object for the marked content, reuses it on subsequent requests that mark the same content, and lets it expire after the TTL (5m by default, 1h supported). There is no need to create or reference Google cache objects directly.

Google AI

Set up your Google AI API key to use Gemini models with implicit prompt caching.

Usage in the response

On the Responses API (/v3/router/responses), cache activity is reported under usage.input_tokens_details:

{
  "usage": {
    "input_tokens": 1200,
    "output_tokens": 180,
    "total_tokens": 1380,
    "input_tokens_details": {
      "cached_tokens": 1024,
      "cache_creation_tokens": 0
    }
  }
}

cached_tokens counts tokens read from the cache; cache_creation_tokens counts tokens written to the cache. A first request on a new prefix writes the cache (cache_creation_tokens above zero) and subsequent requests read it (cached_tokens above zero). On the Chat Completions API (/v3/router/chat/completions), cache reads appear under prompt_tokens_details.cached_tokens:

{
  "usage": {
    "prompt_tokens": 1200,
    "completion_tokens": 180,
    "total_tokens": 1380,
    "prompt_tokens_details": {
      "cached_tokens": 1024
    }
  }
}

Overview

APIs

Routing

Security & Privacy

Model Capabilities

Analytics & Monitoring

Prompt caching for reduced token costs

Overview

Anthropic

Supported models

Enabling caching

Minimum token thresholds

Cache TTL

Example

OpenAI

OpenAI

Google Gemini

Google AI

Usage in the response

​Overview

​Anthropic

​Supported models

​Enabling caching

​Minimum token thresholds

​Cache TTL

​Example

​OpenAI

OpenAI

​Google Gemini

Google AI

​Usage in the response

Overview

Anthropic

Supported models

Enabling caching

Minimum token thresholds

Cache TTL

Example

OpenAI

Google Gemini

Usage in the response