Thread management for grouped requests

Use Cases

Viewing all turns of a multi-turn conversation as a single trace in observability.
Debugging agent workflows by inspecting the full request sequence end-to-end.
Organizing support conversations to analyze resolution patterns across sessions.
Separating concurrent conversations from the same user to avoid mixing traces.

Group related requests into conversation Threads so they appear together in observability. Threads are a labeling mechanism and do not store or inject message history.

Threads are tracked within the orq.ai Observability stack, to learn more, see Threads.

Quick Start

curl -X POST https://api.orq.ai/v3/router/responses \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o",
    "input": "Start a new conversation about AI",
    "orq": {
      "thread": {
        "id": "conversation-abc123",
        "tags": ["ai-discussion", "user-123"]
      }
    }
  }'

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.ORQ_API_KEY,
  baseURL: "https://api.orq.ai/v3/router",
});

const response = await client.responses.create({
  model: "openai/gpt-4o",
  input: "Start a new conversation about AI",
  orq: {
    thread: {
      id: `conversation-${crypto.randomUUID()}`,
      tags: ["ai-discussion", "user-123"],
    },
  },
});

console.log(response.output_text);

from openai import OpenAI
import os
import uuid

client = OpenAI(
    api_key=os.environ.get("ORQ_API_KEY"),
    base_url="https://api.orq.ai/v3/router",
)

response = client.responses.create(
    model="openai/gpt-4o",
    input="Start a new conversation about AI",
    extra_body={
        "orq": {
            "thread": {
                "id": f"conversation-{uuid.uuid4()}",
                "tags": ["ai-discussion", "user-123"],
            }
        }
    },
)

print(response.output_text)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.ORQ_API_KEY,
  baseURL: "https://api.orq.ai/v3/router",
});

const response = await client.chat.completions.create({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Start a new conversation about AI" }],
  orq: {
    thread: {
      id: `conversation-${crypto.randomUUID()}`,
      tags: ["ai-discussion", "user-123"],
    },
  },
});

Configuration

Parameter	Type	Required	Description
`id`	string	Yes	Unique thread identifier for grouping requests
`tags`	string[]	No	Metadata tags for filtering and categorization

Best Practices

Consistent Naming: Use predictable thread ID patterns
Meaningful Tags: Choose tags that aid in filtering and analysis
Session Management: Tie thread IDs to user sessions
Unique IDs: Use UUIDs or composite keys to avoid cross-session overlap

// Good: Descriptive and unique
const threadId = `support-${userId}-${sessionId}-${timestamp}`;

// Good: UUID-based for uniqueness
const threadId = `conversation-${crypto.randomUUID()}`;

// Avoid: Too generic
const threadId = "conversation1";

Use Cases

Scenario	Thread ID Strategy	Tags Example
User Sessions	`user-{userId}-{sessionId}`	`["user-session", "support"]`
Support Tickets	`ticket-{ticketId}`	`["support", "priority-high"]`
A/B Testing	`test-{variant}-{userId}`	`["experiment", "variant-a"]`
Multi-Step Workflows	`workflow-{processId}`	`["onboarding", "step-2"]`

Implementation Examples

Session-Based Threading

Assign the same thread.id to all requests in a session to group them together in observability. Your application manages the messages array for each request.

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.ORQ_API_KEY,
  baseURL: "https://api.orq.ai/v3/router",
});

// Initialize conversation
const userId = "user-123";
const sessionId = crypto.randomUUID();
const threadId = `user-${userId}-${sessionId}`;

const response = await client.chat.completions.create({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "How do I reset my password?" }],
  orq: {
    thread: {
      id: threadId,
      tags: ["support", "password-reset", `user-${userId}`],
    },
  },
});

// Continue conversation
const followUp = await client.chat.completions.create({
  model: "openai/gpt-4o",
  messages: [
    { role: "user", content: "How do I reset my password?" },
    { role: "assistant", content: response.choices[0].message.content },
    { role: "user", content: "I didn't receive the reset email" },
  ],
  orq: {
    thread: {
      id: threadId, // Same thread ID
      tags: ["support", "email-issue", `user-${userId}`],
    },
  },
});

Multi-Language Support

curl -X POST https://api.orq.ai/v3/router/responses \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o",
    "input": "Explain machine learning basics",
    "orq": {
      "thread": {
        "id": "tech-discussion-001",
        "tags": ["education", "ml-basics", "beginner"]
      }
    }
  }'

curl -X POST https://api.orq.ai/v3/router/chat/completions \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o",
    "messages": [{"role": "user", "content": "Explain machine learning basics"}],
    "orq": {
      "thread": {
        "id": "tech-discussion-001",
        "tags": ["education", "ml-basics", "beginner"]
      }
    }
  }'

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.ORQ_API_KEY,
  baseURL: "https://api.orq.ai/v3/router",
});

const threadId = `conversation-${crypto.randomUUID()}`;

const response = await client.responses.create({
  model: "openai/gpt-4o",
  input: "Explain machine learning basics",
  orq: {
    thread: {
      id: threadId,
      tags: ["education", "ml-basics", "beginner"],
    },
  },
});

console.log(response.output_text);

import os
from openai import OpenAI
import uuid

client = OpenAI(
    api_key=os.environ["ORQ_API_KEY"],
    base_url="https://api.orq.ai/v3/router",
)

thread_id = f"conversation-{uuid.uuid4()}"

response = client.responses.create(
    model="openai/gpt-4o",
    input="Explain machine learning basics",
    extra_body={
        "orq": {
            "thread": {
                "id": thread_id,
                "tags": ["education", "ml-basics", "beginner"],
            }
        }
    },
)

print(response.output_text)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.ORQ_API_KEY,
  baseURL: "https://api.orq.ai/v3/router",
});

const threadId = `conversation-${crypto.randomUUID()}`;

const response = await client.chat.completions.create({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Explain machine learning basics" }],
  orq: {
    thread: {
      id: threadId,
      tags: ["education", "ml-basics", "beginner"],
    },
  },
});

import os
from openai import OpenAI
import uuid

client = OpenAI(
    api_key=os.environ["ORQ_API_KEY"],
    base_url="https://api.orq.ai/v3/router",
)

thread_id = f"conversation-{uuid.uuid4()}"

response = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Explain machine learning basics"}],
    extra_body={
        "orq": {
            "thread": {
                "id": thread_id,
                "tags": ["education", "ml-basics", "beginner"],
            }
        }
    },
)

Advanced Patterns

Dynamic Thread Management

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.ORQ_API_KEY,
  baseURL: "https://api.orq.ai/v3/router",
});

interface ThreadConfig {
  userId: string;
  sessionType: "support" | "sales" | "general";
  priority?: "low" | "medium" | "high";
}

function createThreadId(config: ThreadConfig): string {
  const timestamp = Date.now();
  return `${config.sessionType}-${config.userId}-${timestamp}`;
}

function generateTags(config: ThreadConfig): string[] {
  const baseTags = [config.sessionType, `user-${config.userId}`];

  if (config.priority) {
    baseTags.push(`priority-${config.priority}`);
  }

  return baseTags;
}

const threadConfig: ThreadConfig = {
  userId: "user123",
  sessionType: "support",
  priority: "high",
};

const response = await client.chat.completions.create({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Urgent: System not responding" }],
  orq: {
    thread: {
      id: createThreadId(threadConfig),
      tags: generateTags(threadConfig),
    },
  },
});

Batch Thread Processing

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.ORQ_API_KEY,
  baseURL: "https://api.orq.ai/v3/router",
});

const conversations = [
  { userId: "user1", message: "Question about billing" },
  { userId: "user2", message: "Technical support needed" },
  { userId: "user3", message: "Feature request" },
];

const responses = await Promise.all(
  conversations.map(async (conv, index) => {
    return client.chat.completions.create({
      model: "openai/gpt-4o",
      messages: [{ role: "user", content: conv.message }],
      orq: {
        thread: {
          id: `batch-${Date.now()}-${index}`,
          tags: ["batch-processing", `user-${conv.userId}`],
        },
      },
    });
  }),
);

Conversation Flow Tracking

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.ORQ_API_KEY,
  baseURL: "https://api.orq.ai/v3/router",
});

// Track conversation progression
const conversationSteps = [
  "greeting",
  "problem-identification",
  "solution-proposal",
  "confirmation",
];

async function processConversationStep(step: string, threadId: string, message: string) {
  return await client.chat.completions.create({
    model: "openai/gpt-4o",
    messages: [{ role: "user", content: message }],
    orq: {
      thread: {
        id: threadId,
        tags: ["workflow", `step-${step}`, "customer-service"],
      },
    },
  });
}

Troubleshooting

Issue	Cause	Solution
Requests not grouping together	Different thread IDs used across requests	Use a consistent thread ID for all requests in the same session or workflow
Thread overlap	Non-unique thread IDs shared across sessions	Use UUID or timestamp-based IDs

Get Started

AI Gateway

AI Observability

AI Engineering

AI Governance

AI Chat

Organization

Thread management for grouped requests

Quick Start

Configuration

Best Practices

Use Cases

Implementation Examples

Session-Based Threading

Multi-Language Support

Advanced Patterns

Dynamic Thread Management

Batch Thread Processing

Conversation Flow Tracking

Troubleshooting

​Quick Start

​Configuration

​Best Practices

​Use Cases

​Implementation Examples

​Session-Based Threading

​Multi-Language Support

​Advanced Patterns

​Dynamic Thread Management

​Batch Thread Processing

​Conversation Flow Tracking

​Troubleshooting

Quick Start

Configuration

Best Practices

Use Cases

Implementation Examples

Session-Based Threading

Multi-Language Support

Advanced Patterns

Dynamic Thread Management

Batch Thread Processing

Conversation Flow Tracking

Troubleshooting