Skip to main content

Observability

Instrument your code with OpenTelemetry to capture traces, logs, and metrics for every LLM call, agent step, and tool use.

Observability

Overview

BeeAI is IBM’s open-source agent framework for building production-ready multi-agent systems. It uses the openinference-instrumentation-beeai library to export traces via OpenTelemetry.

Prerequisites

  • An Orq.ai account and API Key
  • Python 3.10+
  • An OpenAI API key (OPENAI_API_KEY) — the examples use OpenAI models

Installation

Configuring Orq.ai Observability

Basic Example

Evaluations & Experiments

Once your agents are running, use Evaluatorq to score outputs across a dataset and Experiments to compare configurations side-by-side.

Run Evaluations with Evaluatorq

Run parallel evaluations across your agents and compare results.

Run Experiments via the API

Compare agent configurations and view results in the AI Studio.