Retrieve the deployment configuration
Bearer authentication header of the form Bearer <token>, where <token> is your auth token.
The deployment key to invoke
Key-value pairs variables to replace in your prompts. If a variable is not provided that is defined in the prompt, the default variables are used.
Key-value pairs that match your data model and fields declared in your deployment routing configuration
A list of messages to include after the System message, but before the User and Assistant pairs configured in your deployment.
Developer-provided instructions that the model should follow, regardless of messages sent by the user.
A list of messages to send to the deployment.
Developer-provided instructions that the model should follow, regardless of messages sent by the user.
Information about the identity making the request. If the identity does not exist, it will be created automatically.
A list of file IDs that are associated with the deployment request.
Key-value pairs that you want to attach to the log generated by this request.
Utilized for passing additional parameters to the model provider. Exercise caution when using this feature, as the included parameters will overwrite any parameters specified in the deployment prompt configuration.
A list of documents from your external knowledge base (e.g., chunks retrieved from your own vector database or RAG pipeline) that provide context for the model response. These documents can be used by evaluators and guardrails to assess the relevance and accuracy of the model output against the provided context.
[
{
"text": "The refund policy allows customers to return items within 30 days of purchase for a full refund.",
"metadata": {
"file_name": "refund_policy.pdf",
"file_type": "application/pdf",
"page_number": 1
}
},
{
"text": "Premium members receive free shipping on all orders over $50.",
"metadata": {
"file_name": "membership_benefits.md",
"file_type": "text/markdown"
}
}
]
A filter to apply to the knowledge base chunk metadata when using knowledge bases in the deployment.
Configurations Retrieved.
The deployment configuration
A unique identifier for the response. Can be used to add metrics to the transaction.
The provider of the model
The model of the configuration
The current version of the deployment
Model Parameters: Not all parameters apply to every model
The type of the model. Current chat,completion and image are supported
chat, completion, embedding, image, tts, stt, rerank, ocr, moderation, vision A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for.