HomepageAdmin PortalSign up
Resources
Start typing to search…

Attaching files in a Deployment

Attach a file to include its content as context for your model.

You can attach files to an LLM call within a Deployment. This is useful when you want to extract data from a PDF or interact with its content.

The content of the files is incorporated into the initial system or user message. We recommend adjusting your prompt as you would when using variables in a prompt.

The added content will be highlighted in green.

The added content will be highlighted in green.

📘

Supported filetypes are pdf, txt, docx, csv, xls - 10MB max

🚧

Only the raw data is extracted, no images or objects. Also, encrypted files are not supported.


Attaching Files during Generation

⚠️

This feature is only supported with OpenAI and Google Gemini Models

For compatible models, files can be embedded directly within the Invoke payload by adding a Message type file.

The message should hold data for the file as a standard data URI scheme: data:content/type;base64followed by the base64 encoded file data.

See below how to use the new message type:

curl --request POST \
     --url https://api.orq.ai/v2/deployments/invoke \
     --header 'accept: application/json' \
     --header 'authorization: Bearer <your_orq_key>' \
     --header 'content-type: application/json' \
     --data '
{
  "key": "key",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "prompt"
        },
        {
          "type": "file",
          "file": {
            "file_data": "data:application/pdf;base64,<base64-encoded-data>"
          }
        }
      ]
    }
  ]
}
'
generation = client.deployments.invoke(
    key="deployment_key",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "prompt"
                },
                {
                    "type": "file",
                    "file": {
                        "file_data": "data:application/pdf;base64,<base64-encoded-data>",
                        "filename": "filename"
                    }
                }
            ]
        }
    ],
    metadata={
        "user_id": "123",
        "session_id": "456",
    }
)
const generation = await client.deployments.invoke({
  key: 'deployment_key',
  messages: [
    {
      role: 'user',
      content: [
        {
          type: 'text',
          text: 'prompt'
        },
        {
          type: 'file',
          file: {
            file_data: 'data:application/pdf;base64,<base64-encoded-data>',
            filename: 'filename.pdf'
          }
        }
      ]
    }
  ],
  metadata: {
    user_id: '123',
    session_id: '456'
  }
});

Attaching Files ahead of generation

Attaching files to a Deployment is a 2 step process, first you have to upload the file, which returns an id. Afterwards you attach this id when invoking the Deployment.

Step 1: Upload a file

To attach files during generation, they need to be uploaded before the generation happens.

To upload a file use the following API call:

You can find latest SDK documentation in the Python SDK and Node.js SDK

curl --location 'https://my.orq.ai/v2/files' \
--header 'Authorization: Bearer xxxxxx' \
--form 'purpose="retrieval"' \
--form 'file=@"/Users/cormick/Downloads/filename.pdf"'
import { Orq } from "@orq-ai/node";
import { openAsBlob } from "node:fs";

const orq = new Orq({
  apiKey: process.env["ORQ_API_KEY"] ?? "",
});

async function run() {
  const result = await orq.files.create({
    file: await openAsBlob("example.file"),
  });

  // Handle the result
  console.log(result);
}

run();
from orq_ai_sdk import Orq
import os


with Orq(
    api_key=os.getenv("ORQ_API_KEY", ""),
) as orq:

    res = orq.files.create(file={
        "file_name": "example.file",
        "content": open("example.file", "rb"),
    })

    assert res is not None

    # Handle response
    print(res)

Here is an example response, store the _id for future usage.

{
    "_id": "file_01JA5D27ZVW2N702Z0D3B1G8EK",
    "object_name": "files-api/workspaces/e747f6ac-19b0-47cd-8e79-0e1bf72b2a3e/retrieval/file_01JA5D27ZVW2N702Z0D3B1G8EK.vnd.openxmlformats-officedocument.spreadsheetml.sheet",
    "purpose": "retrieval",
    "file_name": "file_01JA5D27ZVW2N702Z0D3B1G8EK.vnd.openxmlformats-officedocument.spreadsheetml.sheet",
    "bytes": 5295,
    "created": "2024-10-14T11:36:54.189Z"
}

Step 2: Attach a file during invocation

When invoking a Deployment, attach your file id in the file_ids/ fileIds array as follow:

curl --location 'https://my.orq.ai/v2/deployments/invoke' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer xxxxx' \
--data '{
    "key": "deployment_key",
    "messages": [
        {
            "role": "user",
            "content": ""
        }
    ],
    "file_ids": [
        "file_01JA5D27ZVW2N702Z0D3B1G8EK"
    ]
}'
generation = client.deployments.invoke(
  key="deployment-key",
  messages=[
    {
      "role": "user",
      "content": ""
    }
  ],
  file_ids=[
    "file_01JA5D27ZVW2N702Z0D3B1G8EK"
  ]
)

print(generation.choices[0].message.content)
const deployment = await client.deployments.invoke({
   key: "deployment-key",
   messages: [ {
      role: "user", 
      content: ""
   } ],
   fileIds: ["file_01JA5D27ZVW2N702Z0D3B1G8EK"]
});

When to use Knowledge Base vs Attaching files

The need for full context understanding

Knowledge Bases and RAG (Retrieval Augmented Generation) retrieve relevant chunks, which works for focused queries but falls short for tasks like summarization that require full-document understanding.

Attaching files gives the LLM access to the entire document, ensuring it has the complete context.

For example, when summarizing reports, legal cases, or research papers, the LLM needs to process the full document to capture key details and connections that partial text retrieval can’t provide. Full context access leads to better comprehension and more accurate outputs, particularly for tasks requiring a holistic view, such as summarization and detailed analysis.


Dynamic document context

Unlike a static knowledge base, attached files can provide ad-hoc, context-specific documents for one-time or immediate use without the need for integration into a broader knowledge repository.

When a user is dealing with unique documents—such as one-off reports, meeting notes, or specific contracts—they can attach these files directly to a deployment. The LLM can instantly use these documents to provide answers or insights. This feature is especially useful for situations where time-sensitive or project-specific documents need to be used on the fly, giving flexibility to quickly incorporate new, temporary knowledge without modifying or updating the knowledge base.


Private or sensitive data

Due to privacy concerns, confidential or sensitive files (e.g., contracts and medical records) may not be suitable for a general knowledge base. Attaching files directly allows secure, temporary interaction with this data.

Updated 8 days ago