Attaching files to a Deployment

There are 3 options to attach files to a model Deployment.

Attaching PDFs directly to the model in a Deployment.
Uploading a file and including that file to the Deployment.
Attaching a Knowledge Base to a Deployment.

Option 1: Sending PDFs directly to the model

⚠️
This feature is only supported with OpenAI and Google Gemini Models

For compatible models, files can be embedded directly within the Invoke payload by adding a Message type file.

The message should hold data for the file as a standard data URI scheme: data:content/type;base64followed by the base64 encoded file data.

See below how to use the new message type:

curl --request POST \
     --url https://api.orq.ai/v2/deployments/invoke \
     --header 'accept: application/json' \
     --header 'authorization: Bearer <your_orq_key>' \
     --header 'content-type: application/json' \
     --data '
{
  "key": "key",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "prompt"
        },
        {
          "type": "file",
          "file": {
            "file_data": "data:application/pdf;base64,<base64-encoded-data>"
          }
        }
      ]
    }
  ]
}
'

generation = client.deployments.invoke(
    key="deployment_key",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "prompt"
                },
                {
                    "type": "file",
                    "file": {
                        "file_data": "data:application/pdf;base64,<base64-encoded-data>",
                        "filename": "filename"
                    }
                }
            ]
        }
    ],
    metadata={
        "user_id": "123",
        "session_id": "456",
    }
)

const generation = await client.deployments.invoke({
  key: 'deployment_key',
  messages: [
    {
      role: 'user',
      content: [
        {
          type: 'text',
          text: 'prompt'
        },
        {
          type: 'file',
          file: {
            file_data: 'data:application/pdf;base64,<base64-encoded-data>',
            filename: 'filename.pdf'
          }
        }
      ]
    }
  ],
  metadata: {
    user_id: '123',
    session_id: '456'
  }
});

Option 2: Attaching files to a Deployment

Attaching files to a Deployment is a 2 step process, first you have to upload the file, which returns an id. Afterwards you attach this id when invoking the Deployment.

Step 1: Upload a file

To attach files during generation, they need to be uploaded before the generation happens.

To upload a file use the following API call:

You can find latest SDK documentation in the Python SDK and Node.js SDK

curl --location 'https://my.orq.ai/v2/files' \
--header 'Authorization: Bearer xxxxxx' \
--form 'purpose="retrieval"' \
--form 'file=@"/Users/cormick/Downloads/filename.pdf"'

import { Orq } from "@orq-ai/node";
import { openAsBlob } from "node:fs";

const orq = new Orq({
  apiKey: process.env["ORQ_API_KEY"] ?? "",
});

async function run() {
  const result = await orq.files.create({
    file: await openAsBlob("example.file"),
  });

  // Handle the result
  console.log(result);
}

run();

from orq_ai_sdk import Orq
import os


with Orq(
    api_key=os.getenv("ORQ_API_KEY", ""),
) as orq:

    res = orq.files.create(file={
        "file_name": "example.file",
        "content": open("example.file", "rb"),
    })

    assert res is not None

    # Handle response
    print(res)

Here is an example response, store the _id for future usage.

{
    "_id": "file_01JA5D27ZVW2N702Z0D3B1G8EK",
    "object_name": "files-api/workspaces/e747f6ac-19b0-47cd-8e79-0e1bf72b2a3e/retrieval/file_01JA5D27ZVW2N702Z0D3B1G8EK.vnd.openxmlformats-officedocument.spreadsheetml.sheet",
    "purpose": "retrieval",
    "file_name": "file_01JA5D27ZVW2N702Z0D3B1G8EK.vnd.openxmlformats-officedocument.spreadsheetml.sheet",
    "bytes": 5295,
    "created": "2024-10-14T11:36:54.189Z"
}

Step 2: Attach a file during invocation

When invoking a Deployment, attach your file id in the file_ids/ fileIds array as follow:

curl --location 'https://my.orq.ai/v2/deployments/invoke' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer xxxxx' \
--data '{
    "key": "deployment_key",
    "messages": [
        {
            "role": "user",
            "content": ""
        }
    ],
    "file_ids": [
        "file_01JA5D27ZVW2N702Z0D3B1G8EK"
    ]
}'

generation = client.deployments.invoke(
  key="deployment-key",
  messages=[
    {
      "role": "user",
      "content": ""
    }
  ],
  file_ids=[
    "file_01JA5D27ZVW2N702Z0D3B1G8EK"
  ]
)

print(generation.choices[0].message.content)

const deployment = await client.deployments.invoke({
   key: "deployment-key",
   messages: [ {
      role: "user", 
      content: ""
   } ],
   fileIds: ["file_01JA5D27ZVW2N702Z0D3B1G8EK"]
});

Option 3: Attaching a Knowledge Base to a Deployment.

📘
Read here how to set up a Knowledge Base, or how to Use a Knowledge Base in a Prompt

When to use Knowledge Base vs Attaching files

The need for full context understanding

Knowledge Bases and RAG (Retrieval Augmented Generation) retrieve relevant chunks, which works for focused queries but falls short for tasks like summarization that require full-document understanding.

Attaching files gives the LLM access to the entire document, ensuring it has the complete context.

For example, when summarizing reports, legal cases, or research papers, the LLM needs to process the full document to capture key details and connections that partial text retrieval can’t provide. Full context access leads to better comprehension and more accurate outputs, particularly for tasks requiring a holistic view, such as summarization and detailed analysis.

Dynamic document context

Unlike a static knowledge base, attached files can provide ad-hoc, context-specific documents for one-time or immediate use without the need for integration into a broader knowledge repository.

When a user is dealing with unique documents—such as one-off reports, meeting notes, or specific contracts—they can attach these files directly to a deployment. The LLM can instantly use these documents to provide answers or insights. This feature is especially useful for situations where time-sensitive or project-specific documents need to be used on the fly, giving flexibility to quickly incorporate new, temporary knowledge without modifying or updating the knowledge base.

Private or sensitive data

Due to privacy concerns, confidential or sensitive files (e.g., contracts and medical records) may not be suitable for a general knowledge base. Attaching files directly allows secure, temporary interaction with this data.