SmolAgents
Integrate Orq.ai with SmolAgents (Hugging Face) using OpenTelemetry and MLflow
Getting Started
SmolAgents is Hugging Face's lightweight agent framework for building AI agents with tool calling capabilities. Tracing SmolAgents with Orq.ai provides comprehensive insights into agent reasoning, tool usage, code execution, and performance to optimize your agent workflows.
Prerequisites
Before you begin, ensure you have:
- An Orq.ai account and API key
- Python 3.8+ installed
- SmolAgents installed in your project
- API keys for your LLM providers (OpenAI, Anthropic, etc.)
Install Dependencies
# Core SmolAgents
pip install smolagents
# OpenTelemetry packages
pip install opentelemetry-sdk opentelemetry-exporter-otlp
# MLflow for advanced tracking
pip install mlflow
# LLM providers (choose what you need)
pip install openai anthropic transformers
Configure Orq.ai
Set up your environment variables to connect to Orq.ai's OpenTelemetry collector:
Unix/Linux/macOS:
export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.orq.ai/v2/otel"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer <ORQ_API_KEY>"
export OTEL_RESOURCE_ATTRIBUTES="service.name=smolagents-app,service.version=1.0.0"
export OPENAI_API_KEY="<YOUR_OPENAI_API_KEY>"
Windows (PowerShell):
$env:OTEL_EXPORTER_OTLP_ENDPOINT = "https://api.orq.ai/v2/otel"
$env:OTEL_EXPORTER_OTLP_HEADERS = "Authorization=Bearer <ORQ_API_KEY>"
$env:OTEL_RESOURCE_ATTRIBUTES = "service.name=smolagents-app,service.version=1.0.0"
$env:OPENAI_API_KEY = "<YOUR_OPENAI_API_KEY>"
Using .env file:
OTEL_EXPORTER_OTLP_ENDPOINT=https://api.orq.ai/v2/otel
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer <ORQ_API_KEY>
OTEL_RESOURCE_ATTRIBUTES=service.name=smolagents-app,service.version=1.0.0
OPENAI_API_KEY=<YOUR_OPENAI_API_KEY>
Integrations
SmolAgents supports multiple integration approaches for comprehensive observability:
MLflow Integration
Set up MLflow tracking:
# smolagents_mlflow.py
import mlflow
import mlflow.openai
from smolagents import CodeAgent, ToolCallingAgent
from smolagents.tools import PythonInterpreterTool, DuckDuckGoSearchTool
# Configure MLflow
mlflow.set_tracking_uri("https://api.orq.ai/v2/mlflow")
mlflow.set_experiment("smolagents-experiment")
# Enable autologging
mlflow.openai.autolog()
def create_agent_with_mlflow():
with mlflow.start_run():
# Log parameters
mlflow.log_param("agent_type", "CodeAgent")
mlflow.log_param("model", "gpt-4")
# Create and configure agent
agent = CodeAgent(
tools=[PythonInterpreterTool(), DuckDuckGoSearchTool()],
model="gpt-4",
max_steps=10
)
# Log agent configuration
mlflow.log_param("max_steps", agent.max_steps)
mlflow.log_param("tool_count", len(agent.tools))
return agent
def run_agent_task(agent, task):
with mlflow.start_run(nested=True, run_name="agent_task"):
mlflow.log_param("task", task)
start_time = time.time()
result = agent.run(task)
duration = time.time() - start_time
# Log metrics
mlflow.log_metric("execution_time", duration)
mlflow.log_metric("steps_taken", len(result.steps) if hasattr(result, 'steps') else 0)
# Log the result
mlflow.log_text(result, "agent_output.txt")
return result
Manual OpenTelemetry Setup
Create comprehensive tracing configuration:
# tracing.py
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.sdk.resources import Resource
from opentelemetry.semconv.resource import ResourceAttributes
from opentelemetry.propagate import set_global_textmap
from opentelemetry.propagate.textmap import CompositePropagator
from opentelemetry.propagate.b3 import B3MultiFormat
from opentelemetry.propagate.baggage import BaggagePropagator
import os
# Configure resource
resource = Resource.create({
ResourceAttributes.SERVICE_NAME: "smolagents-app",
ResourceAttributes.SERVICE_VERSION: "1.0.0",
ResourceAttributes.DEPLOYMENT_ENVIRONMENT: os.getenv("ENVIRONMENT", "development"),
})
# Configure exporter
exporter = OTLPSpanExporter(
endpoint=os.getenv("OTEL_EXPORTER_OTLP_ENDPOINT") + "/v1/traces",
headers={"Authorization": os.getenv("OTEL_EXPORTER_OTLP_HEADERS").split("=")[1]}
)
# Set up tracing
provider = TracerProvider(resource=resource)
processor = BatchSpanProcessor(exporter)
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)
# Configure propagation
set_global_textmap(CompositePropagator([
B3MultiFormat(),
BaggagePropagator()
]))
# Get tracer
tracer = trace.get_tracer("smolagents", "1.0.0")
Examples
Basic Code Agent with Tracing
import time
from smolagents import CodeAgent
from smolagents.tools import PythonInterpreterTool
from tracing import tracer
from opentelemetry import trace, baggage
from opentelemetry.semconv.trace import SpanAttributes
async def basic_code_agent_example():
# Set baggage for context propagation
ctx = baggage.set_baggage("user.id", "user123")
with tracer.start_as_current_span(
"smolagents.code_agent.run",
context=ctx,
attributes={
"agent.type": "CodeAgent",
"agent.model": "gpt-4",
"agent.max_steps": 5,
}
) as span:
try:
# Create agent
agent = CodeAgent(
tools=[PythonInterpreterTool()],
model="gpt-4",
max_steps=5
)
task = "Calculate the fibonacci sequence up to 20 and plot it"
span.set_attribute("agent.task", task)
start_time = time.time()
result = agent.run(task)
duration = time.time() - start_time
# Add metrics
span.set_attributes({
"agent.execution_time": duration,
"agent.result.length": len(str(result)),
"agent.tools.count": len(agent.tools),
})
print("Agent Result:", result)
return result
except Exception as e:
span.record_exception(e)
span.set_status(trace.StatusCode.ERROR, str(e))
raise
# Run example
import asyncio
asyncio.run(basic_code_agent_example())
Tool Calling Agent with Custom Tools
from smolagents import ToolCallingAgent, Tool
from smolagents.tools import DuckDuckGoSearchTool
from tracing import tracer
import requests
import json
class WeatherTool(Tool):
name = "get_weather"
description = "Get current weather for a location"
def forward(self, location: str) -> str:
with tracer.start_as_current_span(
"tool.weather.execute",
attributes={
"tool.name": self.name,
"tool.location": location,
}
) as span:
try:
# Mock weather API call
weather_data = {
"location": location,
"temperature": "22°C",
"condition": "sunny",
"humidity": "65%"
}
result = f"Weather in {location}: {weather_data['temperature']}, {weather_data['condition']}"
span.set_attributes({
"tool.result.temperature": weather_data["temperature"],
"tool.result.condition": weather_data["condition"],
"tool.result.humidity": weather_data["humidity"],
})
return result
except Exception as e:
span.record_exception(e)
span.set_status(trace.StatusCode.ERROR, str(e))
raise
class CalculatorTool(Tool):
name = "calculator"
description = "Perform mathematical calculations"
def forward(self, expression: str) -> str:
with tracer.start_as_current_span(
"tool.calculator.execute",
attributes={
"tool.name": self.name,
"tool.expression": expression,
}
) as span:
try:
# Safe evaluation for basic math
allowed_chars = set('0123456789+-*/.() ')
if not all(c in allowed_chars for c in expression):
raise ValueError("Invalid characters in expression")
result = eval(expression) # Note: In production, use a safer math parser
span.set_attributes({
"tool.result.value": result,
"tool.expression.safe": True,
})
return f"Result: {result}"
except Exception as e:
span.record_exception(e)
span.set_status(trace.StatusCode.ERROR, str(e))
raise
async def tool_calling_agent_example():
with tracer.start_as_current_span(
"smolagents.tool_calling_agent.run",
attributes={
"agent.type": "ToolCallingAgent",
"agent.model": "gpt-4",
}
) as span:
# Create agent with custom tools
agent = ToolCallingAgent(
tools=[
WeatherTool(),
CalculatorTool(),
DuckDuckGoSearchTool()
],
model="gpt-4"
)
span.set_attribute("agent.tools.count", len(agent.tools))
span.set_attribute("agent.tools.names", [tool.name for tool in agent.tools])
task = "What's the weather in Paris? Then calculate 15% tip on a $50 bill."
span.set_attribute("agent.task", task)
start_time = time.time()
result = agent.run(task)
duration = time.time() - start_time
span.set_attributes({
"agent.execution_time": duration,
"agent.result.length": len(str(result)),
})
print("Agent Result:", result)
return result
# Run example
import asyncio
asyncio.run(tool_calling_agent_example())
Multi-Step Agent Workflow with Detailed Tracing
from smolagents import CodeAgent
from smolagents.tools import PythonInterpreterTool, DuckDuckGoSearchTool
from tracing import tracer
import time
class TracedAgent:
def __init__(self, agent_type="CodeAgent", model="gpt-4"):
self.agent_type = agent_type
self.model = model
self.tools = [PythonInterpreterTool(), DuckDuckGoSearchTool()]
def create_agent(self):
with tracer.start_as_current_span(
"smolagents.agent.create",
attributes={
"agent.type": self.agent_type,
"agent.model": self.model,
"agent.tools.count": len(self.tools),
}
) as span:
agent = CodeAgent(
tools=self.tools,
model=self.model,
max_steps=10
)
span.set_attributes({
"agent.max_steps": agent.max_steps,
"agent.tools.names": [tool.__class__.__name__ for tool in self.tools],
})
return agent
def execute_task(self, agent, task, step_name):
with tracer.start_as_current_span(
f"smolagents.task.{step_name}",
attributes={
"task.name": step_name,
"task.description": task,
"agent.type": self.agent_type,
}
) as span:
try:
start_time = time.time()
result = agent.run(task)
duration = time.time() - start_time
span.set_attributes({
"task.execution_time": duration,
"task.result.length": len(str(result)),
"task.status": "completed",
})
return result
except Exception as e:
span.record_exception(e)
span.set_status(trace.StatusCode.ERROR, str(e))
span.set_attribute("task.status", "failed")
raise
async def multi_step_workflow_example():
with tracer.start_as_current_span(
"smolagents.workflow.data_analysis",
attributes={
"workflow.name": "data_analysis",
"workflow.steps.count": 3,
}
) as workflow_span:
traced_agent = TracedAgent()
agent = traced_agent.create_agent()
tasks = [
("research", "Search for recent data on climate change trends"),
("analysis", "Create a Python script to analyze temperature data from the last 10 years"),
("visualization", "Generate a graph showing the temperature trends and save it as an image")
]
workflow_span.set_attribute("workflow.tasks", [task[0] for task in tasks])
results = []
for step_name, task in tasks:
result = traced_agent.execute_task(agent, task, step_name)
results.append((step_name, result))
# Add delay between steps for demonstration
time.sleep(1)
workflow_span.set_attributes({
"workflow.status": "completed",
"workflow.results.count": len(results),
})
return results
# Run workflow example
import asyncio
results = asyncio.run(multi_step_workflow_example())
for step_name, result in results:
print(f"Step '{step_name}' completed: {str(result)[:100]}...")
Error Handling and Recovery
from smolagents import CodeAgent
from smolagents.tools import PythonInterpreterTool
from tracing import tracer
from opentelemetry.semconv.trace import SpanAttributes
async def error_handling_example():
with tracer.start_as_current_span(
"smolagents.error_handling.demo",
attributes={
"demo.type": "error_recovery",
}
) as span:
agent = CodeAgent(
tools=[PythonInterpreterTool()],
model="gpt-4",
max_steps=3
)
# Test scenarios with potential failures
test_cases = [
("valid_task", "Calculate 2 + 2"),
("invalid_syntax", "Run this broken Python: print('hello' + 123 +)"),
("resource_intensive", "Create an infinite loop and see what happens"),
]
results = []
for test_name, task in test_cases:
with tracer.start_as_current_span(
f"smolagents.test_case.{test_name}",
attributes={
"test.name": test_name,
"test.task": task,
}
) as test_span:
try:
start_time = time.time()
result = agent.run(task)
duration = time.time() - start_time
test_span.set_attributes({
"test.status": "success",
"test.execution_time": duration,
"test.result.length": len(str(result)),
})
results.append((test_name, "success", result))
except Exception as e:
test_span.record_exception(e)
test_span.set_status(trace.StatusCode.ERROR, str(e))
test_span.set_attributes({
"test.status": "failed",
"test.error.type": type(e).__name__,
"test.error.message": str(e),
})
results.append((test_name, "failed", str(e)))
# Summary
successful_tests = sum(1 for _, status, _ in results if status == "success")
failed_tests = len(results) - successful_tests
span.set_attributes({
"demo.total_tests": len(results),
"demo.successful_tests": successful_tests,
"demo.failed_tests": failed_tests,
"demo.success_rate": successful_tests / len(results) if results else 0,
})
return results
# Run error handling example
import asyncio
results = asyncio.run(error_handling_example())
for test_name, status, result in results:
print(f"Test '{test_name}': {status} - {str(result)[:100]}...")
Next Steps
✅ Verify traces: Check your Orq.ai dashboard to see incoming traces ✅ Add custom attributes: Enhance traces with business-specific metadata ✅ Set up alerts: Configure monitoring for agent performance degradation ✅ Explore metrics: Use trace data for agent optimization ✅ Monitor tool usage: Track which tools are used most frequently
Related Documentation
Support
Updated about 24 hours ago