Skip to main content

​
What are Traces

Traces let you dive into the workflow of each model generation and understand the inner workings of an LLM call on orq.ai. Traces correspond to events within the generations. Events within a Deployment can be various: Traces let you dive into the events, understanding the inputs, outputs and execution of each step.

​
Creating Traces

Traces are automatically generated when using Orq.ai Deployments. You can also create custom traces for your application code using framework instrumentation or the Orq.ai SDK.

​
Framework Instrumentation

For popular AI frameworks and libraries, use automatic instrumentation with OpenTelemetry:

Framework Integrations

Explore automatic instrumentation for OpenAI, LangChain, LlamaIndex, CrewAI, Autogen, and 15+ other frameworks with OpenTelemetry integration.

​
Custom Tracing using the @traced decorator

For custom functions or application workflows, use the @traced decorator from the Python SDK: 
import os
from orq_ai_sdk import Orq
from orq_ai_sdk.traced import traced

# Initialize client
orq = Orq(api_key=os.getenv("ORQ_API_KEY"))

@traced(name="process_document", type="agent")
def process_document(doc_id: str):
    # Your custom processing logic
    result = {"status": "processed", "doc_id": doc_id}
    return result

@traced(name="fetch_context", type="retrieval")
def fetch_relevant_context(query: str):
    # Custom retrieval logic (database, search, etc.)
    return {"context": "relevant information", "sources": ["doc1", "doc2"]}

@traced(name="format_response", type="function")
def format_response(raw_output: str, metadata: dict):
    # Post-processing logic around LLM output
    return {"formatted": raw_output.strip(), "metadata": metadata}

# Execute traced functions
doc = process_document("doc-123")
context = fetch_relevant_context("user query")

# Deployment invocations are already auto-traced
response = orq.deployments.invoke(
    key="summarizer",
    context={"environments": ["production"]},
    inputs={"text": doc["status"], "context": context}
)

formatted = format_response(response.choices[0].message.content, {"doc_id": "doc-123"})
The @traced decorator supports span types: agent, embedding, function, llm, retrieval, and tool. You can also add custom attributes with the attributes parameter.

​
How to Lookup Traces

To find traces, head to the orq.ai Studio and to the Traces section.
By looking up events, you can explore traces for each action taken during a generation:
  • The left column shows the list of Traces available.
  • By selecting a trace, the middle column opens, showing the hierarchy of events shown in order of execution.
  • By selecting a single trace, the right column opens, showing details for the specific step.

​
Trace Views

Each trace can be inspected in three views to suit different debugging and analysis needs.
The Trace view shows the full execution tree for a single run. Each step is displayed hierarchically, including LLM calls, tool invocations, knowledge retrievals, and memory interactions. Use this view to inspect inputs, outputs, token usage, and latency at every step.
You can manually evaluate responses using Human Reviews directly on individual spans. To enable this, a Human Review set must be configured for the span type you are reviewing. Learn more in Human Review.

​
Filtering Traces

The Trace panel offers Filters to search for traces.
When working with Agents, you can access traces directly from the Agent page with automatic filtering for that specific agent.

​
Configuring Column

Columns can be enabled and toggled to display more available data in the traces list.

​
Creating Custom Views

Save frequently used filter combinations as reusable views:
  1. Set your desired filters
  2. Click All Rows (top right)
  3. Select Create New View
  4. Give your view a title
  5. Optionally check Set view private (default is shared with project members)
  6. Your filtered view is now saved and accessible from the All Rows dropdown

​
Threads

You can visualize conversation past as Threads, to learn more see Threads.