Skip to main content
Orq.ai captures token usage from every LLM request and computes cost automatically. No configuration is needed for most providers.

​
How token counts are captured

Token counts arrive from provider responses via OpenTelemetry attributes:
  • gen_ai.usage.input_tokens (input tokens)
  • gen_ai.usage.output_tokens (output tokens)
  • gen_ai.usage.total_tokens (sum, computed by Orq.ai)
Detailed breakdowns are captured when the provider reports them. These follow the provider’s response shape (e.g. OpenAI, Anthropic) rather than a single standard:
  • input_tokens_details.cached_tokens (tokens read from the prompt cache)
  • input_tokens_details.cache_creation_tokens (tokens written to the prompt cache), with an Anthropic-only split into cache_creation_5m_tokens and cache_creation_1h_tokens by cache TTL
  • output_tokens_details.reasoning_tokens (reasoning/thinking tokens)
  • Audio input and output tokens
When a provider does not return usage data, the span is stored but cost attributes may be zero or absent.

​
How cost is calculated

Cost is computed per span server-side. The AI Gateway multiplies token counts by the model’s per-token rate:
Rates are stored per model in the model catalog and expressed as USD per 1M tokens.

​
Cost categories

​
Pricing tiers

Some providers charge different rates based on context length. Orq.ai applies the correct rate automatically: The active tier is recorded as orq.billing.pricing_tier on each span.

​
Billable flag

orq.billing.billable is false for cached responses and automated evaluations run by the platform. These do not count toward workspace usage billing.

​
Cost rollup

Cost rolls up through the data model:

​
Where to see cost breakdowns

  • Span Attributes: full reference of orq.billing.* attributes
  • Billing and Usage: platform billing (seats, storage, plan capacity), separate from LLM cost tracking
  • Budgets: set cost and token limits per scope