Quick start
Jumpstart your journey with orq.ai's LLM Ops platform. Our Quick Start Guide provides step-by-step instructions to navigate the platform, optimize your LLM operations, and elevate your AI capabilities. Harness the power of AI with orq.ai now.
Welcome to orq.ai
This page covers the fastest way to create your initial deployment and make your first LLM call via our platform. If you prefer to start with an offline module, please visit Playgrounds.
Prerequisite.
As a prerequisite, you should have an account within the orq.ai admin panel. If not, please contact [email protected], and we'll help you out.
Step 1: Enable models in the Model Garden
You can browse the Model Garden by going to the Model Garden tab.
Here, you can find all models available to use within orq.ai, including their capabilities and a short description. To make any model available, toggle the button next to its name, it will then be usable in your first Deployments. If you want to use your own API keys, head to the Integrations tab.
To learn more about all the models available, see Model Garden.
Step 2: Set up your first Deployment
To create your first deployment, head to the Deployment tab, then select Create Deployment.
You will then be taken to the Variant configuration screen. Here, you can select which parameters you want to use with your model and include the prompt(s) that will be sent to the language model.
Once ready, press the Deploy button at the top-right, this will save the latest configuration and make them available to reach through our API.
There are many more features available to configure your Deployment. Including setting up multiple models, fallbacks, function calling, etc. To learn more, see Setting up a Deployment.
Step 3: Install the orq.ai SDK
Within the Deployment page, you can open the code snippet panel to find instructions to install and run the SDK for the current deployment.
Then, proceed to install your chosen SDK. Execute the following in a terminal to download the latest version of the SDK on your machine.
npm install @orq-ai/node --save
pip install orq-ai-sdk
Step 4: Get your API Token
To setup your client you will need an API key, this can be found within the Settings > API Keys menu.
To learn more about our API Authentication, see Authentication.
Step 5: Use the SDK to invoke your Deployment
Once you have saved your API Token for use. You can refer back to the code snippet accessed before your Deployment to fetch the code to initialize and invoke a deployment:
Don't forget to replace
<API_KEY>
with the previously fetched API key.
import { createClient } from '@orq-ai/node';
const client = createClient({
apiKey: '<API_KEY>',
environment: 'production',
});
client.setUser({id: 2024})
const deployment = await client.deployments.invoke({
key: "quickstart-deployment",
context: {
environments: []
},
metadata: {
"custom-field-name": "custom-metadata-value"
}
});
import os
from orq_ai_sdk import OrqAI
client = OrqAI(
api_key=os.environ.get("ORQ_API_KEY", "<API_KEY>"),
environment="production"
)
client.set_user(id=2024)
generation = client.deployments.invoke(
key="quickstart-deployment",
context={
"environments": []
},
metadata={
"custom-field-name": "custom-metadata-value"
}
)
print(generation.choices[0].message.content)
Congratulations, you successfully ran your first deployment invocation with orq.ai, to learn more about deployments and all the possibilities they offer, see Setting up a Deployment.
Step 6: Track your projects and costs
orq.ai allows you to track costs and projects grouped in Contacts. Contacts are people within your organization manipulating generation in your Workspace.
You can use a Contact within any SDK before invoking a generation to log all usage within that contact.
import { createClient } from '@orq-ai/node';
const client = createClient({
apiKey: '<API_KEY>',
environment: 'production',
});
client.setUser({id: 2024})
client.setContact({
id: '<contact-id>',
display_name: 'John Doe',
email: '[email protected]',
avatar_url: 'https://acme.com/john-doe.jpg',
metadata: { role: 'admin' },
});
const deployment = await client.deployments.invoke({
key: "quickstart-deployment",
context: {
environments: []
},
metadata: {
"custom-field-name": "custom-metadata-value"
}
});
import os
from orq_ai_sdk import OrqAI
client = OrqAI(
api_key=os.environ.get("ORQ_API_KEY", "<API_KEY>"),
environment="production"
)
client.set_user(id=2024)
client.set_contact(
id="<contact-id>",
display_name="John Doe",
email="[email protected]",
avatar_url="https://acme.com/john-doe.jpg",
metadata={"role": "admin"},
)
generation = client.deployments.invoke(
key="quickstart-deployment",
context={
"environments": []
},
metadata={
"custom-field-name": "custom-metadata-value"
}
)
print(generation.choices[0].message.content)
To learn more about Contacts and the resulting Analytics, see Contact analytics
Updated about 15 hours ago