Skip to main content

​
AI Gateway

​
Overview

Microsoft AutoGen is a framework for building multi-agent conversational AI systems with collaborative problem-solving through automated agent interactions. Connecting AutoGen to Orq.ai’s AI Gateway provides access to 300+ models for multi-agent workflows with a single configuration change.

​
Key Benefits

Orq.ai’s AI Gateway enhances AutoGen applications with:

Complete Observability

Track every agent conversation, tool use, and multi-agent interaction

Built-in Reliability

Automatic fallbacks, retries, and load balancing for production resilience

Cost Optimization

Real-time cost tracking and spend management across all AI operations

Multi-Provider Access

Access 300+ LLMs and 20+ providers through a single, unified integration

​
Prerequisites

Before integrating AutoGen with Orq.ai, ensure the following are in place:
  • An Orq.ai account and API Key
  • Python 3.10 or higher
To set up an API key, see API keys & Endpoints.

​
Installation

pip install "autogen-agentchat" "autogen-ext[openai]"

​
Configuration

Configure AutoGen to use Orq.ai’s AI Gateway via OpenAIChatCompletionClient with a custom base_url:
Python
import os
from autogen_ext.models.openai import OpenAIChatCompletionClient

model_client = OpenAIChatCompletionClient(
    model="gpt-4o",
    base_url="https://api.orq.ai/v3/router",
    api_key=os.getenv("ORQ_API_KEY"),
    model_info={
        "vision": False,
        "function_calling": True,
        "json_output": True,
        "family": "unknown",
        "structured_output": True,
    },
)
base_url: https://api.orq.ai/v3/router
The model_info dict is required when using a custom base_url so AutoGen knows the model’s capabilities.

​
Basic Agent Example

Python
import asyncio
import os
from autogen_agentchat.agents import AssistantAgent
from autogen_ext.models.openai import OpenAIChatCompletionClient

async def main():
    model_client = OpenAIChatCompletionClient(
        model="gpt-4o",
        base_url="https://api.orq.ai/v3/router",
        api_key=os.getenv("ORQ_API_KEY"),
        model_info={
            "vision": False,
            "function_calling": True,
            "json_output": True,
            "family": "unknown",
            "structured_output": True,
        },
    )

    agent = AssistantAgent(
        name="assistant",
        model_client=model_client,
        system_message="You are a helpful assistant.",
    )

    result = await agent.run(task="What is quantum computing?")
    print(result.messages[-1].content)
    await model_client.close()

asyncio.run(main())

​
Multi-Agent Team

Orchestrate multiple specialized agents with RoundRobinGroupChat:
Python
import asyncio
import os
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.conditions import MaxMessageTermination
from autogen_ext.models.openai import OpenAIChatCompletionClient

async def main():
    model_client = OpenAIChatCompletionClient(
        model="gpt-4o",
        base_url="https://api.orq.ai/v3/router",
        api_key=os.getenv("ORQ_API_KEY"),
        model_info={
            "vision": False,
            "function_calling": True,
            "json_output": True,
            "family": "unknown",
            "structured_output": True,
        },
    )

    researcher = AssistantAgent(
        name="researcher",
        model_client=model_client,
        system_message="You research topics and provide key facts.",
    )

    writer = AssistantAgent(
        name="writer",
        model_client=model_client,
        system_message="You write clear summaries based on research.",
    )

    team = RoundRobinGroupChat(
        participants=[researcher, writer],
        termination_condition=MaxMessageTermination(max_messages=2),
    )

    result = await team.run(task="Research and summarize what LLMs are.")
    print(result.messages[-1].content)
    await model_client.close()

asyncio.run(main())

​
Model Selection

Orq.ai supports any model from 20+ providers:
Python
import os
from autogen_ext.models.openai import OpenAIChatCompletionClient

model_info = {
    "vision": False,
    "function_calling": True,
    "json_output": True,
    "family": "unknown",
    "structured_output": True,
}

# Use Claude
claude_client = OpenAIChatCompletionClient(
    model="anthropic/claude-sonnet-4-6",
    base_url="https://api.orq.ai/v3/router",
    api_key=os.getenv("ORQ_API_KEY"),
    model_info=model_info,
)

# Use Gemini
gemini_client = OpenAIChatCompletionClient(
    model="google-ai/gemini-2.5-flash",
    base_url="https://api.orq.ai/v3/router",
    api_key=os.getenv("ORQ_API_KEY"),
    model_info=model_info,
)

# Use Groq
groq_client = OpenAIChatCompletionClient(
    model="groq/llama-3.3-70b-versatile",
    base_url="https://api.orq.ai/v3/router",
    api_key=os.getenv("ORQ_API_KEY"),
    model_info=model_info,
)