Release 3.12

🔍 Enhanced Observability & Debugging

Human-in-the-Loop Reviews

• Collect structured feedback on AI outputs with customizable Human Review Sets per trace type
• Directly add spans to datasets for continuous improvement
• Track contact IDs and thread context across chat completions

Faster Root Cause Analysis

• View retrieval configurations directly in span properties
• See evaluator names on spans for quick performance assessment
• Expanded OpenTelemetry support for more frameworks

Cost Optimization

• Optional response caching to reduce latency and API costs
• Fixed cost aggregation for image operations and Azure OpenAI
• More accurate token and billing tracking

🧪 Streamlined Experimentation

Improved Experiment Management

• Search across experiment entries
• Protection against accidental re-runs
• Persistent column settings and better cancellation handling
• Enhanced UI with clearer active states and progress indicators

🚀 AI Proxy Enhancements

Advanced Request Handling

• Automatic retries and fallback models for improved reliability
• Thread and contact tracking for conversation continuity
• Specify prompt versions directly in LLM calls
• Improved SSE streaming performance

💰 Budget Controls

Workspace-Level Cost Management

• Set and monitor budgets at workspace and contact levels
• New Budgets API for programmatic cost control
• Automated alerts and spending limits

🎯 Platform Improvements

Model Management

• New image generation models and providers
• Intelligent model filtering based on capabilities
• Improved cost extraction and model selection UI

Developer Experience

• Better API parameter documentation for Knowledge Bases
• Unsaved changes protection across Teams and Contacts
• Improved error handling and retry logic throughout