> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orq.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Session Affinity

> Keep every turn of a conversation on the model the Smart Router chose for its first turn, so the provider prompt cache survives and later turns skip scoring on the request path.

A [**Smart Router**](/docs/ai-gateway/smart-router) picks a model per request. In a multi-turn conversation that means every turn is scored again and can land on a different model, which resets the provider prompt cache. **Session affinity** pins a conversation to the model chosen on its first turn. Later turns reuse that model with no scoring call on the request path, and keep the cache warm.

Session affinity is off by default and configured per router. Only requests that carry a thread id or, on the Responses endpoint, a conversation id take part. Everything else is routed per request as before.

## Use Cases

* Chat assistants and copilots where consecutive turns share a long system prompt or tool set and benefit from the provider prompt cache.
* Agent loops that call the router several times per user turn and should not change models halfway through.
* Support conversations where the tone and level of detail should stay consistent from the first reply to the last.

## Enable Session Affinity

### In the UI

1. Open the **Smart Router** in **AI Gateway** and edit it, or create a new one.
2. Under **Session affinity**, turn on **Pin conversations to one model**.
3. Set **Pin for** to the idle window: 1, 5, 10, 30, or 60 minutes. The default is 10 minutes.
4. Click **Save changes**, or **Create** for a new router.

### Through the API

Send `session_affinity` when creating or updating a **Smart Router**. `ttl_seconds` is the idle window in seconds, between 60 and 3600, and defaults to 600 when omitted. On update the object is replaced as a whole; omitting `ttl_seconds` resets the window to the default of 600.

<CodeGroup>
  ```bash cURL theme={"theme":{"light":"github-light","dark":"github-dark"}}
  curl -X PATCH https://api.orq.ai/v2/smart-routers/$SMART_ROUTER_ID \
    -H "Authorization: Bearer $ORQ_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "session_affinity": { "enabled": true, "ttl_seconds": 600 }
    }'
  ```

  ```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
  const response = await fetch(
    `https://api.orq.ai/v2/smart-routers/${process.env.SMART_ROUTER_ID}`,
    {
      method: "PATCH",
      headers: {
        Authorization: `Bearer ${process.env.ORQ_API_KEY}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        session_affinity: { enabled: true, ttl_seconds: 600 },
      }),
    },
  );

  console.log(await response.json());
  ```

  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  import os
  import requests

  response = requests.patch(
      f"https://api.orq.ai/v2/smart-routers/{os.environ['SMART_ROUTER_ID']}",
      headers={"Authorization": f"Bearer {os.environ['ORQ_API_KEY']}"},
      json={"session_affinity": {"enabled": True, "ttl_seconds": 600}},
  )

  print(response.json())
  ```
</CodeGroup>

The same object is accepted on `POST /v2/smart-routers`. See the [Smart Routers API reference](/reference/smart-routers/create-a-smart-router).

## Seed the Pin

The pin is keyed on an id the caller sends. Nothing is derived from the message content.

| Endpoint | Seed | How to send it |
| - | - | - |
| `/v3/router/chat/completions` | Thread id | `thread.id` in the body, or the `X-ORQ-THREAD-ID` header |
| `/v3/anthropic/v1/messages` | Thread id | `thread.id` in the body, or the `X-ORQ-THREAD-ID` header |
| `/v3/router/responses` | Thread id, else conversation id | `thread.id` or `X-ORQ-THREAD-ID`; `conversation.id` when no thread id is sent |
| `/v2/deployments/invoke` | Thread id | `thread.id` in the body, or the `X-ORQ-THREAD-ID` header |
| **Agents** | Thread id, else conversation id | Same as the Responses endpoint |

When both are present the body wins over the header. A thread id and a conversation id with the same value produce two independent pins. See [Request Metadata](/docs/ai-gateway/request-metadata) for the thread field and headers.

## Quick Start

Call the **Smart Router** twice with the same thread id. The first turn is scored and returns `X-Orq-Session-Affinity: miss`; the second turn reuses the pinned model and returns `hit`.

<CodeGroup>
  ```bash cURL theme={"theme":{"light":"github-light","dark":"github-dark"}}
  for turn in "What does the refund policy say about digital goods?" "And for physical goods?"; do
    curl -s -D - -o /dev/null -X POST https://my.orq.ai/v3/router/chat/completions \
      -H "Authorization: Bearer $ORQ_API_KEY" \
      -H "Content-Type: application/json" \
      -d "{
        \"model\": \"workspace_name@orq/my-smart-router\",
        \"messages\": [{ \"role\": \"user\", \"content\": \"$turn\" }],
        \"thread\": { \"id\": \"conversation-abc123\" }
      }" | grep -i x-orq-session-affinity
  done
  ```

  ```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
  import OpenAI from "openai";

  const client = new OpenAI({
    apiKey: process.env.ORQ_API_KEY,
    baseURL: "https://my.orq.ai/v3/router",
  });

  const thread = { id: "conversation-abc123" };

  for (const content of [
    "What does the refund policy say about digital goods?",
    "And for physical goods?",
  ]) {
    const { data, response } = await client.chat.completions
      .create({
        model: "workspace_name@orq/my-smart-router",
        messages: [{ role: "user", content }],
        // @ts-expect-error Orq.ai extension
        thread,
      })
      .withResponse();

    console.log(response.headers.get("x-orq-session-affinity"), data.model);
  }
  ```

  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  from openai import OpenAI
  import os

  client = OpenAI(
      api_key=os.environ.get("ORQ_API_KEY"),
      base_url="https://my.orq.ai/v3/router",
  )

  for content in [
      "What does the refund policy say about digital goods?",
      "And for physical goods?",
  ]:
      raw = client.chat.completions.with_raw_response.create(
          model="workspace_name@orq/my-smart-router",
          messages=[{"role": "user", "content": content}],
          extra_body={"thread": {"id": "conversation-abc123"}},
      )
      print(raw.headers.get("x-orq-session-affinity"), raw.parse().model)
  ```
</CodeGroup>

The `model` field in the response names the concrete model on both turns.

## How a Pin Is Decided

The first turn of a thread is scored and routed as usual. The pin is written only after that turn's first provider call succeeds, so a failed or retried first turn never pins a model. A turn the scorer could not classify with confidence is served but not pinned; the next turn tries again.

On a hit, the request goes straight to the pinned model with no scoring call on the request path, and the idle window is refreshed. **Easy** and **Medium** hits still trigger one background scoring call for [escalation](#escalation); **Hard** hits make none.

## Escalation

A thread that starts easy can turn hard. On every hit with an **Easy** or **Medium** pin, the router re-scores the turn in the background. When the new score ranks in a higher band, the pin moves up to that band's model and stays there. A pin never moves down: a downgrade would reset the provider cache for a small saving. **Hard** pins are never re-scored.

The escalation is a compare-and-set write that keeps the pin's remaining lifetime, so two turns racing on one thread cannot move the decision backwards.

## Pin Lifetime

The idle window is sliding. Every hit refreshes it, so an active conversation keeps its model for as long as it stays active, and a thread that stays quiet longer than the window is routed fresh on its next turn. There is no absolute ceiling.

A pin is dropped, and the request routed per request instead, when:

* The pinned model fails on the first provider call of a turn.
* The pinned model is no longer in the router's pool, or no longer callable in the workspace.
* The stored pin cannot be parsed.

When the pin store cannot be read, the pin is ignored for that turn and the turn is routed fresh. A dropped or ignored pin never fails the request.

## Where It Applies

| Surface | Session affinity | `X-Orq-Session-Affinity` header |
| - | - | - |
| `/v3/router/chat/completions` | Yes | `hit` or `miss` |
| `/v3/anthropic/v1/messages` | Yes | `hit` or `miss` |
| `/v3/router/responses` | Yes | `hit` or `miss` |
| `/v2/deployments/invoke` (JSON and streaming) | Yes, when the variant's model is a **Smart Router** | `hit` or `miss` |
| **Agents** whose primary model is a **Smart Router** | Yes | Not set |
| Background responses (`background: true`) | Reads an existing pin only | `hit` or `miss` |

On the Responses and Messages endpoints, a turn that runs a tool loop settles the pin on its first provider call only. Sub-agents pin under their own router and their own key. Scheduled agent runs carry no thread or conversation id and are routed per request.

## Observability

Every routed request records the decision on its span:

| Attribute | Meaning |
| - | - |
| `orq.session_affinity.enabled` | Session affinity is on for this router |
| `orq.session_affinity.seed` | `thread` or `conversation` |
| `orq.session_affinity.hit` | The turn reused a pin |
| `orq.session_affinity.ttl_seconds` | The router's idle window |
| `orq.session_affinity.escalated` | The pin moved up on this turn |
| `orq.session_affinity.escalated_from`, `orq.session_affinity.escalated_to` | The bands involved |

The escalation attributes are recorded on a separate `auto-router.escalation` span in the same trace, because the re-score runs after the response is sent.

The metrics `router.session_affinity.hits`, `router.session_affinity.misses`, `router.session_affinity.dissolves`, and `router.session_affinity.escalations` count the same events per router. See [Traces](/docs/ai-studio/observability/traces).

<Note>
  A pin holds a model name, a band, and a timestamp. No message content or user data is stored with it.
</Note>

## Limitations

| Limitation | Impact | Workaround |
| - | - | - |
| Needs a thread id or conversation id | Requests without either are routed per request | Send `thread.id`, the `X-ORQ-THREAD-ID` header, or `conversation.id` on the Responses endpoint |
| Pins are per router | A [load balancer](/docs/ai-gateway/features/load-balancing) in front of several **Smart Routers** reshuffles requests every turn | Put session affinity on one **Smart Router** and call it directly |
| Pins are per API key and identity | The same thread called with a different API key or identity gets its own pin | Keep one API key and identity per conversation |
| Background responses | `background: true` requests reuse a pin but never write or drop one | Start the thread with a foreground request |
| **Agents** return no header | The decision is visible on the span, not on the response | Filter traces on `orq.session_affinity.hit` |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.