Stream a deployment
Stream a deployment invocation with the Orq.ai REST API, receiving LLM completion output incrementally as server-sent events arrive.
Authorizations
Body
A list of messages to include after the System message, but before the User and Assistant pairs configured in your deployment.
Developer-provided instructions that the model should follow, regardless of messages sent by the user.
- System message
- Developer message
- User message
- Assistant message
- Tool message
A list of messages to send to the deployment.
Developer-provided instructions that the model should follow, regardless of messages sent by the user.
- System message
- Developer message
- User message
- Assistant message
- Tool message
A list of documents from your external knowledge base (e.g., chunks retrieved from your own vector database or RAG pipeline) that provide context for the model response. These documents can be used by evaluators and guardrails to assess the relevance and accuracy of the model output against the provided context.
Response
Successful operation
The provider used to generate the response
openai, groq, cohere, azure, aws, google, google-ai, huggingface, togetherai, perplexity, anthropic, leonardoai, fal, nvidia, jina, elevenlabs, litellm, cerebras, openailike, bytedance, mistral, deepseek, moonshotai, zai, minimax, xai, alibaba, tensorix, scaleway, hcompany, inception, particle, munito, inceptron, wafer, poolside, tencent, nebius, fireworks, baseten, reson8, meta, greenpt, liquid, typesafe, berget, cloudflare, slack, orq List of documents retrieved from the knowledge base. This property is only available when the include_retrievals flag is set to true in the invoke settings. When stream is set to true, the retrievals property will be returned in the last streamed chunk where the property is_final is set to true.