> ## Documentation Index > Fetch the complete documentation index at: https://docs.orq.ai/llms.txt > Use this file to discover all available pages before exploring further. # Prompt caching for reduced token costs > Reduce input token costs by caching repeated prompt prefixes at the provider level. Save on long system prompts and reference documents with Anthropic, OpenAI, and Google Gemini. **Use Cases** * Reusing long system prompts across many requests to cut input token costs. * Referencing large documents or codebases without re-sending them every call. * Multi-turn conversations with a large, stable context that doesn't change between turns. * RAG pipelines where the same retrieved context is shared across many user queries. *** ## Overview Prompt Caching is a provider-level feature that caches prompts so that **repeated requests** are charged at a reduced rate. This is most effective when your requests share a **large, stable prefix**: * a long system prompt. * a reference document. * a tool definition list. Unlike [Response Caching](/ai-gateway/features/cache), which serves a stored response for identical requests, Prompt Caching still calls the model on every request, at a reduced cost. Both can be used together. How caching is enabled and what gets cached varies by provider. See the provider sections below. ## Anthropic Prompt caching on Anthropic models requires explicit opt-in via `cache_control` markers on individual message parts. A breakpoint marks the end of a cacheable prefix: the provider caches everything up to and including the marked block, and later requests that share that prefix read the cached portion instead of reprocessing it. On the Anthropic-compatible endpoint, breakpoints pass through exactly as in the native Anthropic API; see the [Anthropic Messages API](/ai-gateway/features/anthropic-messages-api) for endpoint details and a multi-turn example. ### Supported models All current Claude models, including Claude Fable 5 and the Opus, Sonnet, and Haiku families. ### Enabling caching Add a `cache_control` object to any message part you want to mark as cacheable: ```json theme={"theme":{"light":"github-light","dark":"github-dark"}} { "cache_control": { "type": "ephemeral" } } ``` `"ephemeral"` is the only supported type. You can place it on: * System message text parts. * User message text parts. * User message images, documents, and files (including PDFs). * Tool definitions. * Tool result content. ### Minimum token thresholds Caching only activates once the marked content exceeds a minimum token count. Requests below the threshold are processed normally at full cost. | Model | Minimum tokens | | ----------------------------------------------------------------------------------------- | -------------- | | Claude Opus 5, Fable 5 | 512 | | Claude Opus 4.8, Sonnet 5, Sonnet 4.6, Sonnet 4.5, Opus 4.1, Opus 4, Sonnet 4, Sonnet 3.7 | 1,024 | | Claude Opus 4.7, Haiku 3.5, Haiku 3 | 2,048 | | Claude Opus 4.6, Opus 4.5, Haiku 4.5 | 4,096 | ### Cache TTL The `ttl` parameter controls how long cached content persists before expiring. | Value | Duration | | ---------------- | ----------------------- | | `"5m"` (default) | 5 minutes from last use | | `"1h"` | 1 hour | ```json theme={"theme":{"light":"github-light","dark":"github-dark"}} { "cache_control": { "type": "ephemeral", "ttl": "1h" } } ``` Cache reads are billed at a fraction of the regular input token price. Cache writes carry a premium: 1.25x the input price for the 5-minute TTL and 2x for the 1-hour TTL. Caching pays off from the second request onward on a shared prefix. ### Multi-turn conversations **Default: mark the system prompt.** Put `cache_control` on the system text part and send the full input on every request, including any conversation history the client keeps. The system prompt is written once and read on every later turn, no matter what the user says. Every other example on this page uses this form. **When continuing with `previous_response_id`, use the top-level parameter instead.** The gateway rebuilds the history from storage, and block-level markers from earlier turns are not carried over. Send only the new user message and set `cache_control` at the top level of the request. Each turn then reads the whole previous conversation and writes only the new message. ```json theme={"theme":{"light":"github-light","dark":"github-dark"}} { "model": "anthropic/claude-sonnet-5", "previous_response_id": "resp_...", "cache_control": { "type": "ephemeral" }, "input": [ { "role": "user", "content": "And what about clause 9?" } ] } ``` Do not resend the system prompt on a `previous_response_id` turn. The stored history already contains it, and a second copy fails validation on Anthropic models. Avoid the top-level parameter on independent requests. It marks the last block, which is the user message, so requests with different questions never share a prefix and each one pays the cache write premium without a read. ### Example ```bash cURL theme={"theme":{"light":"github-light","dark":"github-dark"}} curl -X POST https://my.orq.ai/v3/router/responses \ -H "Authorization: Bearer $ORQ_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "anthropic/claude-sonnet-5", "input": [ { "role": "system", "content": [ { "type": "input_text", "text": "You are a senior legal assistant. The following is our complete contract template library...", "cache_control": { "type": "ephemeral" } } ] }, { "role": "user", "content": "Summarize clause 7 of the NDA template." } ] }' ``` ```bash cURL (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}} curl -X POST https://my.orq.ai/v3/router/chat/completions \ -H "Authorization: Bearer $ORQ_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "anthropic/claude-sonnet-5", "messages": [ { "role": "system", "content": [ { "type": "text", "text": "You are a senior legal assistant. The following is our complete contract template library...", "cache_control": { "type": "ephemeral" } } ] }, { "role": "user", "content": "Summarize clause 7 of the NDA template." } ] }' ``` ```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}} import OpenAI from "openai"; const client = new OpenAI({ apiKey: process.env.ORQ_API_KEY, baseURL: "https://my.orq.ai/v3/router", }); const response = await client.responses.create({ model: "anthropic/claude-sonnet-5", input: [ { role: "system", content: [ { type: "input_text", text: "You are a senior legal assistant. The following is our complete contract template library...", cache_control: { type: "ephemeral" }, }, ], }, { role: "user", content: "Summarize clause 7 of the NDA template." }, ], }); console.log(response.output_text); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}} from openai import OpenAI import os client = OpenAI( api_key=os.environ.get("ORQ_API_KEY"), base_url="https://my.orq.ai/v3/router", ) response = client.responses.create( model="anthropic/claude-sonnet-5", input=[ { "role": "system", "content": [ { "type": "input_text", "text": "You are a senior legal assistant. The following is our complete contract template library...", "cache_control": {"type": "ephemeral"}, } ], }, {"role": "user", "content": "Summarize clause 7 of the NDA template."}, ], ) print(response.output_text) ``` ```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}} import OpenAI from "openai"; const client = new OpenAI({ apiKey: process.env.ORQ_API_KEY, baseURL: "https://my.orq.ai/v3/router", }); const response = await client.chat.completions.create({ model: "anthropic/claude-sonnet-5", messages: [ { role: "system", content: [ { type: "text", text: "You are a senior legal assistant. The following is our complete contract template library...", cache_control: { type: "ephemeral" }, }, ], }, { role: "user", content: "Summarize clause 7 of the NDA template." }, ], }); console.log(response.choices[0].message.content); ``` ```python Python (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}} from openai import OpenAI import os client = OpenAI( api_key=os.environ.get("ORQ_API_KEY"), base_url="https://my.orq.ai/v3/router", ) response = client.chat.completions.create( model="anthropic/claude-sonnet-5", messages=[ { "role": "system", "content": [ { "type": "text", "text": "You are a senior legal assistant. The following is our complete contract template library...", "cache_control": {"type": "ephemeral"}, } ], }, {"role": "user", "content": "Summarize clause 7 of the NDA template."}, ], ) print(response.choices[0].message.content) ``` ## OpenAI Prompt caching on OpenAI models is **fully automatic**. No `cache_control` or any request changes are required. The **AI Gateway** forwards requests normally; OpenAI caches the prompt prefix on its side and applies the discount transparently. Caching activates on prompts longer than 1,024 tokens, in 128-token increments from that threshold. The API caches the longest matching prefix from prior requests on the same machine. Cache retention duration is model-dependent and determined by OpenAI. Refer to [OpenAI's prompt caching documentation](https://platform.openai.com/docs/guides/prompt-caching) for the current retention policy per model. Cache hits are reflected in the response `usage` object. See [Usage in the response](#usage-in-the-response) below. Set up your OpenAI API key to use GPT models with automatic prompt caching. ## Google Gemini Google Gemini supports two caching modes through the **AI Gateway**. **Implicit caching** is enabled by default on Gemini 2.5 and newer models. No request changes are needed. The **AI Gateway** forwards requests normally and Google applies the cache discount automatically when a matching prefix exists. Implicit caching activates at a model-dependent minimum: 2,048 tokens on Gemini 2.5 models, higher on newer generations. Verify current thresholds and rates in [Google's caching documentation](https://ai.google.dev/gemini-api/docs/caching). **Explicit caching** is managed by the **AI Gateway** on supported Gemini models (Gemini 2.5 and newer). Add `cache_control` markers to system or user message parts (text and files), using the same syntax as on Anthropic models. The gateway creates a Google cache object for the marked content, reuses it on subsequent requests that mark the same content, and lets it expire after the TTL (`5m` by default, `1h` supported). There is no need to create or reference Google cache objects directly. Explicit caching works on Vertex AI deployments with service-account auth. The gateway skips it on Google AI (Gemini API) when the request has a system prompt or tools, and entirely on Vertex AI Express (API-key) deployments, where only implicit caching applies. Set up your Google AI API key to use Gemini models with implicit prompt caching. ## Usage in the response Every endpoint reports cache reads and cache writes in the `usage` object. Field names follow the endpoint's own response shape, so the same request looks slightly different on each API. ### Responses API On `/v3/router/responses`, cache activity is reported under `usage.input_tokens_details`. The same fields appear in the final `response.completed` event when streaming. | Field | Meaning | | -------------------------- | ---------------------------------------------------------------------------- | | `cached_tokens` | Tokens read from the cache. | | `cache_creation_tokens` | Tokens written to the cache, across all TTLs. | | `cache_write_tokens` | Same value as `cache_creation_tokens`, kept for compatibility. | | `cache_creation_5m_tokens` | Tokens written with the 5-minute TTL. Present only on a write with that TTL. | | `cache_creation_1h_tokens` | Tokens written with the 1-hour TTL. Present only on a write with that TTL. | A first request on a new prefix writes the cache: ```json theme={"theme":{"light":"github-light","dark":"github-dark"}} { "usage": { "input_tokens": 2673, "output_tokens": 30, "total_tokens": 2703, "input_tokens_details": { "cached_tokens": 0, "cache_creation_tokens": 2669, "cache_write_tokens": 2669, "cache_creation_5m_tokens": 2669 } } } ``` A later request on the same prefix reads it. The per-TTL fields are omitted when nothing was written: ```json theme={"theme":{"light":"github-light","dark":"github-dark"}} { "usage": { "input_tokens": 2673, "output_tokens": 30, "total_tokens": 2703, "input_tokens_details": { "cached_tokens": 2669, "cache_creation_tokens": 0, "cache_write_tokens": 0 } } } ``` `input_tokens` includes cached tokens, matching the OpenAI convention. ### Chat Completions API On `/v3/router/chat/completions`, cache activity is reported under `usage.prompt_tokens_details`. The per-TTL split is not available on this endpoint. ```json theme={"theme":{"light":"github-light","dark":"github-dark"}} { "usage": { "prompt_tokens": 2673, "completion_tokens": 30, "total_tokens": 2703, "prompt_tokens_details": { "cached_tokens": 2669, "cache_creation_tokens": 0, "audio_tokens": 0 } } } ``` ### Anthropic Messages API On `/v3/anthropic/v1/messages`, usage uses the native Anthropic field names. `input_tokens` counts only uncached tokens. Cache fields are present only when non-zero: a write omits `cache_read_input_tokens`, and a pure read omits `cache_creation_input_tokens` and the nested `cache_creation` object. ```json theme={"theme":{"light":"github-light","dark":"github-dark"}} { "usage": { "input_tokens": 4, "output_tokens": 30, "cache_creation_input_tokens": 2669, "cache_creation": { "ephemeral_5m_input_tokens": 2669, "ephemeral_1h_input_tokens": 0 } } } ``` ## Cache usage in Traces Cache reads and writes are recorded on every LLM span, so they can be inspected per request in [Traces](/ai-studio/observability/traces) and aggregated in Analytics. | Where | What is shown | | ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Span attributes | `gen_ai.usage.prompt_tokens_details.cached_tokens`, `gen_ai.usage.prompt_tokens_details.cache_creation_tokens`, and the per-TTL `gen_ai.usage.prompt_tokens_details.cache_creation_5m_tokens` / `gen_ai.usage.prompt_tokens_details.cache_creation_1h_tokens`. | | Economics panel and trace list | Cached input tokens, plus `orq.billing.cache_read_cost` and `orq.billing.cache_write_cost` in USD. The write cost already includes the 1.25x or 2x TTL premium. | | Analytics | Cached tokens and cache read/write cost roll up into the workspace token and cost breakdowns. | The per-TTL token split is stored as a span attribute only. It is not surfaced as a separate column in the trace list or in Analytics. See [Span attributes](/ai-studio/observability/span-attributes#token-usage) for the token usage and billing attributes and [Token and cost tracking](/ai-studio/observability/token-cost-tracking) for how cache costs are computed.