> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orq.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Databricks integration

> Reach 300+ models from inside Databricks by registering the AI Gateway as an external model, and capture traces from agents on Model Serving with OpenTelemetry.

**Databricks** Model Serving hosts both external models and agents authored against the MLflow `ResponsesAgent` interface. Register the **AI Gateway** as an external model to reach 300+ models from inside **Databricks**, and instrument agents with OpenTelemetry to capture their LLM calls in **Orq.ai**.

<CardGroup cols={2}>
  <Card title="AI Gateway" icon="arrow-right-arrow-left" href="#ai-gateway">
    Serve any **Orq.ai** model through a **Databricks** serving endpoint, with cost tracking and fallbacks.
  </Card>

  <Card title="Observability" icon="chart-line" href="#observability">
    Export traces from agents running on Model Serving, with prompts, tokens, and cost.
  </Card>
</CardGroup>

## Prerequisites

* An **Orq.ai** account and [API key](/docs/ai-studio/organization/api-keys)
* A **Databricks** workspace with Model Serving enabled
* MLflow 3.11 or later, plus `databricks-sdk` and `databricks-agents` installed locally

## Store the API key

Both sections below read the key from a **Databricks** secret. Keep it there rather than in an endpoint configuration, where anyone with view access on the endpoint can read it. Skip `create_scope` if the scope already exists:

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  from databricks.sdk import WorkspaceClient

  workspace = WorkspaceClient()
  workspace.secrets.create_scope(scope="orq")
  workspace.secrets.put_secret(
      scope="orq", key="api_key", string_value="<ORQ_API_KEY>"
  )
  ```
</CodeGroup>

## AI Gateway

**Databricks** external models accept any OpenAI-compatible endpoint through the `custom` provider, so the **AI Gateway** can back a **Databricks** serving endpoint. Set `external_model.name` to the model to serve, and point `custom_provider_url` at the **AI Gateway**:

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  import mlflow.deployments

  client = mlflow.deployments.get_deploy_client("databricks")

  client.create_endpoint(
      config={
          "name": "orq-gateway",
          "config": {
              "served_entities": [
                  {
                      "name": "orq-gateway",
                      "external_model": {
                          "name": "openai/gpt-5.6-sol",
                          "provider": "custom",
                          "task": "llm/v1/chat",
                          "custom_provider_config": {
                              "custom_provider_url": "https://api.orq.ai/v3/router/chat/completions",
                              "bearer_token_auth": {
                                  "token": "{{secrets/orq/api_key}}"
                              },
                          },
                      },
                  }
              ]
          },
      },
  )
  ```
</CodeGroup>

Query it like any other **Databricks** serving endpoint. Calls appear in **Traces** with model, token usage, and cost:

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  response = client.predict(
      endpoint="orq-gateway",
      inputs={"messages": [{"role": "user", "content": "Hello"}]},
  )
  print(response["choices"][0]["message"]["content"])
  ```
</CodeGroup>

<Info>
  Any enabled model works. Set `external_model.name` to its **Orq.ai** slug and see [supported models](/docs/ai-gateway/supported-models). One endpoint serves one model, so create an endpoint per model.
</Info>

## Observability

Adding OpenTelemetry to an agent already running on Model Serving sends its LLM calls to **Orq.ai**, with full request and response content, token usage, and cost. There are four changes: add an exporter at module level, flush at the end of `predict`, declare three extra packages, and pass the endpoint and key at deploy time.

<Info>
  This covers agents deployed from code with `agents.deploy()`. Agents created in the no-code **Agent Bricks** builder run on an endpoint that **Databricks** manages, which exposes neither environment variables nor model dependencies, so they cannot be instrumented this way.
</Info>

### Add the exporter to the agent

Add this at module level in the agent file, above the agent class. It opens the connection to **Orq.ai** and attaches the instrumentor that records LLM calls. `OpenAIInstrumentor` records calls made through the OpenAI SDK, which is what `WorkspaceClient().serving_endpoints.get_open_ai_client()` returns. Agents built on another client library need the matching OpenInference instrumentor in its place, otherwise the agent deploys cleanly and produces no spans:

<CodeGroup>
  ```python agent.py theme={"theme":{"light":"github-light","dark":"github-dark"}}
  import os

  from openinference.instrumentation.openai import OpenAIInstrumentor
  from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
  from opentelemetry.sdk.trace import TracerProvider
  from opentelemetry.sdk.trace.export import BatchSpanProcessor

  tracer_provider = TracerProvider()
  tracer_provider.add_span_processor(
      BatchSpanProcessor(
          OTLPSpanExporter(
              endpoint=os.environ["ORQ_OTEL_ENDPOINT"],
              headers={"Authorization": f"Bearer {os.environ['ORQ_API_KEY']}"},
          )
      )
  )
  OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)
  ```
</CodeGroup>

Then flush at the end of `predict`, before returning. Model Serving keeps the container alive between requests, so the flush is what sends spans promptly rather than leaving them queued:

<CodeGroup>
  ```python agent.py theme={"theme":{"light":"github-light","dark":"github-dark"}}
      def predict(self, request: ResponsesAgentRequest) -> ResponsesAgentResponse:
          response = ...  # existing agent logic, unchanged
          tracer_provider.force_flush(15000)
          return response
  ```
</CodeGroup>

### Declare the dependencies

The serving environment installs only the packages declared on the model, so add these to the existing `pip_requirements` on `log_model`. Without them the agent fails to import at request time:

<CodeGroup>
  ```python deploy.py theme={"theme":{"light":"github-light","dark":"github-dark"}}
          pip_requirements=[
              # existing requirements
              "opentelemetry-sdk",
              "opentelemetry-exporter-otlp-proto-http",
              "openinference-instrumentation-openai",
          ],
  ```
</CodeGroup>

### Pass the endpoint and key

Add these to `environment_vars` on `agents.deploy()`. `OTEL_SERVICE_NAME` is optional and sets the name used to filter traces later:

<CodeGroup>
  ```python deploy.py theme={"theme":{"light":"github-light","dark":"github-dark"}}
      environment_vars={
          "ORQ_OTEL_ENDPOINT": "https://api.orq.ai/v2/otel/v1/traces",
          "ORQ_API_KEY": "{{secrets/orq/api_key}}",
          "OTEL_SERVICE_NAME": "databricks-support-agent",
      },
  ```
</CodeGroup>

<Note>
  Configuring OTLP through the standard `OTEL_EXPORTER_OTLP_*` variables alone does not export traces from Model Serving. The managed runtime sets up its own MLflow tracing, which takes precedence, and spans reach the **Databricks** experiment rather than **Orq.ai**. Add the exporter to the agent as shown above instead.
</Note>

<Warning>
  A secret reference is substituted only when it is the entire value of a variable. Writing `Authorization=Bearer {{secrets/orq/api_key}}` sends the braces literally and **Orq.ai** rejects the spans with a 401. Pass the key in its own variable, as above, and build the header in the agent.
</Warning>

### View traces

Traces appear in [AI Studio](https://my.orq.ai) under the **Traces** tab. Each LLM call arrives as a span carrying the request and response messages, prompt and completion tokens, the resolved model and provider, and computed cost. Filter on the value set in `OTEL_SERVICE_NAME` to isolate one agent.

### Complete example

This example calls a **Databricks** native endpoint. To route the agent through the **AI Gateway** instead, set `LLM_ENDPOINT` to the endpoint created above.

<Accordion title="Full agent module and deployment script">
  <CodeGroup>
    ```python agent.py theme={"theme":{"light":"github-light","dark":"github-dark"}}
    import os

    from databricks.sdk import WorkspaceClient
    from mlflow.models import set_model
    from mlflow.pyfunc import ResponsesAgent
    from mlflow.types.responses import (
        ResponsesAgentRequest,
        ResponsesAgentResponse,
        create_text_output_item,
    )
    from openinference.instrumentation.openai import OpenAIInstrumentor
    from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
    from opentelemetry.sdk.trace import TracerProvider
    from opentelemetry.sdk.trace.export import BatchSpanProcessor

    LLM_ENDPOINT = "databricks-gemini-3-5-flash"

    tracer_provider = TracerProvider()
    tracer_provider.add_span_processor(
        BatchSpanProcessor(
            OTLPSpanExporter(
                endpoint=os.environ["ORQ_OTEL_ENDPOINT"],
                headers={"Authorization": f"Bearer {os.environ['ORQ_API_KEY']}"},
            )
        )
    )
    OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)


    class SupportAgent(ResponsesAgent):
        def predict(self, request: ResponsesAgentRequest) -> ResponsesAgentResponse:
            client = WorkspaceClient().serving_endpoints.get_open_ai_client()
            completion = client.chat.completions.create(
                model=LLM_ENDPOINT,
                messages=[{"role": "user", "content": self._question(request)}],
                max_tokens=300,
            )
            tracer_provider.force_flush(15000)
            return ResponsesAgentResponse(
                output=[
                    create_text_output_item(
                        text=completion.choices[0].message.content, id="answer"
                    )
                ]
            )

        def _question(self, request: ResponsesAgentRequest) -> str:
            last_message = request.input[-1]
            as_dict = (
                last_message
                if isinstance(last_message, dict)
                else last_message.model_dump()
            )
            return as_dict.get("content", "")


    set_model(SupportAgent())
    ```

    ```python deploy.py theme={"theme":{"light":"github-light","dark":"github-dark"}}
    import mlflow
    from databricks import agents
    from mlflow.models.resources import DatabricksServingEndpoint

    mlflow.set_registry_uri("databricks-uc")

    with mlflow.start_run():
        logged_agent = mlflow.pyfunc.log_model(
            name="agent",
            python_model="agent.py",
            resources=[
                DatabricksServingEndpoint(endpoint_name="databricks-gemini-3-5-flash")
            ],
            pip_requirements=[
                "mlflow>=3.11",
                "databricks-sdk",
                "openai",
                "httpx",
                "opentelemetry-sdk",
                "opentelemetry-exporter-otlp-proto-http",
                "openinference-instrumentation-openai",
            ],
        )

    registered_agent = mlflow.register_model(
        logged_agent.model_uri, "catalog.schema.support_agent"
    )

    agents.deploy(
        "catalog.schema.support_agent",
        registered_agent.version,
        environment_vars={
            "ORQ_OTEL_ENDPOINT": "https://api.orq.ai/v2/otel/v1/traces",
            "ORQ_API_KEY": "{{secrets/orq/api_key}}",
            "OTEL_SERVICE_NAME": "databricks-support-agent",
        },
    )
    ```
  </CodeGroup>
</Accordion>

## Evaluations & Experiments

Once agents are running, use **Evaluatorq** to score outputs across a dataset and **Experiments** to compare configurations side by side.

<CardGroup cols={2}>
  <Card title="Run Evaluations with Evaluatorq" icon="flask" href="/docs/ai-studio/optimize/evaluators#evaluatorq">
    Run parallel evaluations across agents and compare results.
  </Card>

  <Card title="Run Experiments via the API" icon="flask-vial" href="/docs/ai-studio/optimize/experiments">
    Compare agent configurations and view results in the AI Studio.
  </Card>
</CardGroup>