Retries & Error Handling

Overview

Who is this for? Developers building production AI applications who need robust error handling and automatic recovery from provider failures, rate limits, and network issues.

What you'll achieve: Implement resilient AI applications with intelligent retry strategies, provider fallbacks, and comprehensive error handling that automatically recovers from transient failures.

The AI Proxy includes sophisticated retry mechanisms with provider-specific error handling, intelligent backoff strategies, and automatic failover to ensure maximum reliability.

Retry Configuration

Basic Retry Setup

<CODE_PLACEHOLDER>

Retry Configuration Parameters

Parameter	Description	Default	Range
`max_attempts`	Maximum number of retry attempts	`3`	`1-10`
`base_delay_ms`	Initial delay between retries	`1000`	`100-60000`
`max_delay_ms`	Maximum delay between retries	`30000`	`1000-300000`
`exponential_base`	Multiplier for exponential backoff	`2`	`1.1-10`
`jitter`	Add randomization to delay	`true`	`true/false`
`retry_on`	Specific error types to retry	`["rate_limit", "timeout", "server_error"]`	Array

Provider-Specific Error Handling

Rate Limit Handling

The AI Proxy automatically detects and handles provider-specific rate limits:

OpenAI & Compatible Providers

<CODE_PLACEHOLDER>

Anthropic Claude

<CODE_PLACEHOLDER>

Google AI (Gemini)

<CODE_PLACEHOLDER>

Automatic Rate Limit Detection

<CODE_PLACEHOLDER>

Advanced Retry Strategies

Exponential Backoff with Jitter

<CODE_PLACEHOLDER>

Provider Fallback Chain

<CODE_PLACEHOLDER>

Custom Retry Conditions

<CODE_PLACEHOLDER>

Error Types and Handling

Retriable Errors (Default Behavior)

Error Type	Description	Provider Examples
`rate_limit`	Too many requests	All providers
`timeout`	Request timeout	Network/provider issues
`server_error`	5xx HTTP status	Provider downtime
`model_overloaded`	Model capacity exceeded	High demand periods
`insufficient_quota`	Account limits reached	Usage-based providers

Non-Retriable Errors (Fail Fast)

Error Type	Description	Action
`invalid_request`	Malformed request	Fix request format
`authentication_error`	Invalid API key	Check credentials
`content_policy_violation`	Content blocked	Modify content
`model_not_found`	Invalid model name	Use valid model
`context_length_exceeded`	Input too long	Reduce input size

Implementation Examples

Node.js with Custom Error Handling

<CODE_PLACEHOLDER>

Python with Async Retries

<CODE_PLACEHOLDER>

React Hook with Error Recovery

<CODE_PLACEHOLDER>

Monitoring and Debugging

Retry Metrics

<CODE_PLACEHOLDER>

Response Headers:
<CODE_PLACEHOLDER>

Error Response Format

<CODE_PLACEHOLDER>

Best Practices

Production Configuration

<CODE_PLACEHOLDER>

Circuit Breaker Pattern

<CODE_PLACEHOLDER>

Graceful Degradation

<CODE_PLACEHOLDER>

Troubleshooting

Common Retry Issues

Infinite Retry Loops
<CODE_PLACEHOLDER>

Insufficient Rate Limit Handling
<CODE_PLACEHOLDER>

Memory Leaks in Long Retry Chains
<CODE_PLACEHOLDER>

Next Steps

Fallbacks: Configure provider fallback strategies
Load Balancing: Distribute requests across providers
Monitoring: Track retry metrics and provider health