Thread Management

📖
This page describes features extending the AI Proxy, which provides a unified API for accessing multiple AI providers. To learn more, see AI Proxy.

Group related requests into conversation Threads for multi-turn tracking and analysis.

📘
Threads are tracked within the orq.ai Observabilitty stack, to learn more, see Threads.

Quick Start

const response = await openai.chat.completions.create({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Start a new conversation about AI" }],
  orq: {
    thread: {
      id: "conversation-001",
      tags: ["ai-discussion", "user-123"],
    },
  },
});

Configuration

Parameter	Type	Required	Description
`id`	string	Yes	Unique thread identifier for grouping requests
`tags`	string[]	No	Metadata tags for filtering and categorization

Use Cases

Scenario	Thread ID Strategy	Tags Example
User Sessions	`user-{userId}-{sessionId}`	`["user-session", "support"]`
Support Tickets	`ticket-{ticketId}`	`["support", "priority-high"]`
A/B Testing	`test-{variant}-{userId}`	`["experiment", "variant-a"]`
Multi-Step Workflows	`workflow-{processId}`	`["onboarding", "step-2"]`

Implementation Examples

Session-Based Threading

// Initialize conversation
const sessionId = generateSessionId();
const threadId = `user-${userId}-${sessionId}`;

const response = await openai.chat.completions.create({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "How do I reset my password?" }],
  orq: {
    thread: {
      id: threadId,
      tags: ["support", "password-reset", `user-${userId}`],
    },
  },
});

// Continue conversation
const followUp = await openai.chat.completions.create({
  model: "openai/gpt-4o",
  messages: [
    { role: "user", content: "How do I reset my password?" },
    { role: "assistant", content: response.choices[0].message.content },
    { role: "user", content: "I didn't receive the reset email" },
  ],
  orq: {
    thread: {
      id: threadId, // Same thread ID
      tags: ["support", "email-issue", `user-${userId}`],
    },
  },
});

Multi-Language Support

curl -X POST https://api.orq.ai/v2/proxy/chat/completions \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o",
    "messages": [{"role": "user", "content": "Start technical discussion"}],
    "orq": {
      "thread": {
        "id": "tech-discussion-001",
        "tags": ["technical", "architecture"]
      }
    }
  }'

import openai
import uuid

client = openai.OpenAI(
    api_key=os.environ["ORQ_API_KEY"],
    base_url="https://api.orq.ai/v2/proxy"
)

thread_id = f"conversation-{uuid.uuid4()}"

response = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Explain machine learning basics"}],
    extra_body={
        "orq": {
            "thread": {
                "id": thread_id,
                "tags": ["education", "ml-basics", "beginner"]
            }
        }
    }
)

Advanced Patterns

Dynamic Thread Management

interface ThreadConfig {
  userId: string;
  sessionType: "support" | "sales" | "general";
  priority?: "low" | "medium" | "high";
}

function createThreadId(config: ThreadConfig): string {
  const timestamp = Date.now();
  return `${config.sessionType}-${config.userId}-${timestamp}`;
}

function generateTags(config: ThreadConfig): string[] {
  const baseTags = [config.sessionType, `user-${config.userId}`];

  if (config.priority) {
    baseTags.push(`priority-${config.priority}`);
  }

  return baseTags;
}

const threadConfig: ThreadConfig = {
  userId: "user123",
  sessionType: "support",
  priority: "high",
};

const response = await openai.chat.completions.create({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Urgent: System not responding" }],
  orq: {
    thread: {
      id: createThreadId(threadConfig),
      tags: generateTags(threadConfig),
    },
  },
});

Batch Thread Processing

const conversations = [
  { userId: "user1", message: "Question about billing" },
  { userId: "user2", message: "Technical support needed" },
  { userId: "user3", message: "Feature request" },
];

const responses = await Promise.all(
  conversations.map(async (conv, index) => {
    return openai.chat.completions.create({
      model: "openai/gpt-4o",
      messages: [{ role: "user", content: conv.message }],
      orq: {
        thread: {
          id: `batch-${Date.now()}-${index}`,
          tags: ["batch-processing", `user-${conv.userId}`],
        },
      },
    });
  }),
);

Thread Analysis

Conversation Flow Tracking

// Track conversation progression
const conversationSteps = [
  "greeting",
  "problem-identification",
  "solution-proposal",
  "confirmation",
];

async function processConversationStep(step, threadId, message) {
  return await openai.chat.completions.create({
    model: "openai/gpt-4o",
    messages: [{ role: "user", content: message }],
    orq: {
      thread: {
        id: threadId,
        tags: ["workflow", `step-${step}`, "customer-service"],
      },
    },
  });
}

Troubleshooting

Conversations not grouping

Cause: Different thread IDs used
Solution: Use consistent thread ID for related requests

Missing conversation history

Cause: Thread ID not persistent
Solution: Store thread ID in session/database

Performance degradation

Cause: Too many tags per thread
Solution: Limit to 5-10 relevant tags

Thread overlap

Cause: Non-unique thread IDs
Solution: Use UUID or timestamp-based IDs

Common Debugging Steps

// Validate thread configuration
function validateThreadConfig(thread) {
  if (!thread.id) {
    throw new Error("Thread ID is required");
  }

  if (thread.id.length > 100) {
    throw new Error("Thread ID too long (max 100 chars)");
  }

  if (thread.tags && thread.tags.length > 10) {
    console.warn("Too many tags, consider reducing for performance");
  }

  return true;
}

// Usage
const threadConfig = {
  id: "conversation-123",
  tags: ["support", "billing"],
};

validateThreadConfig(threadConfig);

Performance Optimization

Thread ID Best Practices

// ✅ Good: Descriptive and unique
const threadId = `support-${userId}-${sessionId}-${timestamp}`;

// ✅ Good: UUID-based for uniqueness
const threadId = `conversation-${crypto.randomUUID()}`;

// ❌ Avoid: Too generic
const threadId = "conversation1";

// ❌ Avoid: Too long or complex
const threadId = `very-long-descriptive-thread-identifier-with-too-much-detail-${timestamp}`;

Tag Optimization

// ✅ Efficient tagging
const tags = ["support", "billing", "priority-high"];

// ❌ Too many tags (impacts performance)
const tags = [
  "support",
  "billing",
  "high",
  "priority",
  "urgent",
  "customer",
  "issue",
  "payment",
  "subscription",
  "account",
];

Limitations

Limitation	Description	Workaround
Thread ID Length	Maximum 100 characters	Use shorter, meaningful identifiers
Tag Count	Recommended maximum 10 tags	Group related concepts into single tags
Thread Persistence	Threads expire after 30 days of inactivity	Archive important conversations externally
Concurrent Access	No built-in thread locking	Implement application-level synchronization

Best Practices

Consistent Naming: Use predictable thread ID patterns
Meaningful Tags: Choose tags that aid in filtering and analysis
Session Management: Tie thread IDs to user sessions
Cleanup Strategy: Archive or delete old threads regularly
Error Handling: Validate thread configuration before requests