Skip to main content
POST
/
v2
/
deployments
/
stream
Stream
curl --request POST \
  --url https://api.orq.ai/v2/deployments/stream \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "key": "<string>",
  "inputs": {},
  "context": {},
  "prefix_messages": [
    {
      "role": "developer",
      "content": "<string>",
      "name": "<string>"
    }
  ],
  "messages": [
    {
      "role": "developer",
      "content": "<string>",
      "name": "<string>"
    }
  ],
  "file_ids": [
    "<string>"
  ],
  "metadata": {},
  "extra_params": {},
  "documents": [
    {
      "text": "<string>",
      "metadata": {
        "file_name": "<string>",
        "file_type": "<string>",
        "page_number": 123
      }
    }
  ],
  "invoke_options": {
    "include_retrievals": false,
    "mock_response": "<string>"
  },
  "thread": {
    "id": "<string>",
    "tags": [
      "<string>"
    ]
  },
  "knowledge_filter": {}
}'
{
  "id": "<string>",
  "created": "2023-11-07T05:31:56Z",
  "object": "chat",
  "model": "<string>",
  "provider": "cohere",
  "is_final": true,
  "integration_id": "<string>",
  "finalized": "2023-11-07T05:31:56Z",
  "system_fingerprint": "<string>",
  "retrievals": [
    {
      "document": "<string>",
      "metadata": {
        "file_name": "<string>",
        "page_number": 123,
        "file_type": "<string>",
        "rerank_score": 123,
        "search_score": 123
      }
    }
  ],
  "provider_response": "<any>",
  "choices": [
    {
      "index": 123,
      "message": {
        "type": "tool_calls",
        "role": "system",
        "content": "<string>",
        "tool_calls": [
          {
            "id": "<string>",
            "index": 123,
            "type": "function",
            "function": {
              "name": "<string>",
              "arguments": "<string>"
            }
          }
        ],
        "reasoning": "<string>",
        "reasoning_signature": "<string>",
        "redacted_reasoning": "<string>"
      },
      "finish_reason": "<string>"
    }
  ]
}

Authorizations

​
Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json
​
key
string
required

The deployment key to invoke

​
inputs
object

Key-value pairs variables to replace in your prompts. If a variable is not provided that is defined in the prompt, the default variables are used.

​
context
object

Key-value pairs that match your data model and fields declared in your deployment routing configuration

​
prefix_messages
(Developer message · object | System message · object | User message · object | Assistant message · object | Tool message · object)[]

A list of messages to include after the System message, but before the User and Assistant pairs configured in your deployment.

  • Developer message
  • System message
  • User message
  • Assistant message
  • Tool message
​
messages
(Developer message · object | System message · object | User message · object | Assistant message · object | Tool message · object)[]

A list of messages to send to the deployment.

  • Developer message
  • System message
  • User message
  • Assistant message
  • Tool message
​
file_ids
string[]

A list of file IDs that are associated with the deployment request.

​
metadata
object

Key-value pairs that you want to attach to the log generated by this request.

​
extra_params
object

Utilized for passing additional parameters to the model provider. Exercise caution when using this feature, as the included parameters will overwrite any parameters specified in the deployment prompt configuration.

​
documents
object[]

A list of relevant documents that evaluators and guardrails can cite to evaluate the user input or the model response based on your deployment settings.

​
invoke_options
object
​
thread
object
​
knowledge_filter
object

A filter to apply to the knowledge base chunk metadata when using knowledge bases in the deployment.

  • Search operator
  • and
  • or

Response

200 - text/event-stream

Successful operation

​
id
string<ulid>
required

A unique identifier for the response. Can be used to add metrics to the transaction.

​
created
string<date-time>
required

A timestamp indicating when the object was created. Usually in a standardized format like ISO 8601

​
object
enum<string>
required

Indicates the type of model used to generate the response

Available options:
chat,
completion,
image
​
model
string
required

The model used to generate the response

​
provider
enum<string>
required

The provider used to generate the response

Available options:
cohere,
openai,
anthropic,
huggingface,
replicate,
google,
google-ai,
azure,
aws,
anyscale,
perplexity,
groq,
fal,
leonardoai,
nvidia,
jina,
togetherai,
elevenlabs,
litellm,
openailike,
cerebras,
bytedance,
mistral
​
is_final
boolean
required

Indicates if the response is the final response

​
choices
object[]
required

A list of choices generated by the model

​
integration_id
string

Indicates integration id used to generate the response

​
finalized
string<date-time>

A timestamp indicating when the object was finalized. Usually in a standardized format like ISO 8601

​
system_fingerprint
string | null

Provider backed system fingerprint.

​
retrievals
object[]

List of documents retrieved from the knowledge base. This property is only available when the include_retrievals flag is set to true in the invoke settings. When stream is set to true, the retrievals property will be returned in the last streamed chunk where the property is_final is set to true.

​
provider_response
any

Response returned by the model provider. This functionality is only supported when streaming is not used. If streaming is used, the provider_response property will be set to null.