AI Proxy - Superpower your LLM requests

Today we are bring all the power Deployments to our AI Proxy. Now teams will be able to run their AI workloads in a reliable and battle tested AI Proxy.

Features support Via the Proxy:

Fallbacks
Retry
Contact Tracking
Thread Management
Cache
Knowledge Bases

curl --location 'https://api.orq.ai/v2/router/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $ORQ_API_KEY' \
--data-raw '{
  "model": "openai/gpt-4o",
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful customer support agent for {{company_name}}. Use available knowledge to assist {{customer_tier}} customers."
    },
    {
      "role": "user",
      "content": "I need help with API integration for my {{use_case}} project"
    }
  ],
  "orq": {
    "retry": {
      "count": 3,
      "on_codes": [429, 500, 502, 503, 504]
    },
    "fallbacks": [
      {
        "model": "anthropic/claude-3-5-sonnet-20241022"
      },
      {
        "model": "openai/gpt-4o-mini"
      }
    ],
    "cache": {
      "type": "exact_match",
      "ttl": 1800
    },
    "knowledge_bases": [
      {
        "knowledge_id": "api-documentation",
        "top_k": 5,
        "threshold": 0.75
      },
      {
        "knowledge_id": "integration-examples",
        "top_k": 3,
        "threshold": 0.8
      }
    ],
    "contact": {
      "id": "enterprise_customer_001",
      "display_name": "Enterprise User",
      "email": "[email protected]"
    },
    "thread": {
      "id": "support_session_001",
      "tags": ["api-integration", "enterprise", "technical-support"]
    },
    "inputs": {
      "company_name": "Orq AI",
      "customer_tier": "Enterprise",
      "use_case": "e-commerce platform"
    }
  }
}'

Start building today with the AI Proxy https://docs.orq.ai/docs/proxy#/