Prompt caching for reduced token costs

• Reusing long system prompts across many requests to cut input token costs.
• Referencing large documents or codebases without re-sending them every call.
• Multi-turn conversations with a large, stable context that doesn’t change between turns.
• RAG pipelines where the same retrieved context is shared across many user queries.

​

Overview

• a long system prompt.
• a reference document.
• a tool definition list. Unlike Response Caching, which serves a stored response for identical requests, Prompt Caching still calls the model on every request, at a reduced cost. Both can be used together.

​

Anthropic

​

Supported models

​

Enabling caching

{
  "cache_control": { "type": "ephemeral" }
}

• System message text parts.
• User message text parts.
• User message images, documents, and files (including PDFs).
• Tool result content.

​

Minimum token thresholds

​

Cache TTL

{
  "cache_control": {
    "type": "ephemeral",
    "ttl": "1h"
  }
}

​

Example

curl -X POST https://api.orq.ai/v3/router/responses \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-4-6",
    "input": [
      {
        "role": "system",
        "content": [
          {
            "type": "input_text",
            "text": "You are a senior legal assistant. The following is our complete contract template library...",
            "cache_control": { "type": "ephemeral" }
          }
        ]
      },
      {
        "role": "user",
        "content": "Summarize clause 7 of the NDA template."
      }
    ]
  }'

​

OpenAI

​

Google Gemini

Google AI

Set up your Google AI API key to use Gemini models with implicit prompt caching.

​

Usage in the response

{
  "usage": {
    "prompt_tokens": 1200,
    "completion_tokens": 180,
    "total_tokens": 1380,
    "prompt_tokens_details": {
      "cached_tokens": 1024
    }
  }
}