Quick start

Welcome to orq.ai

This page covers the fastest way to create your initial deployment and make your first LLM call via our platform. If you prefer to start with an offline module, please visit Playgrounds.

Prerequisite.

As a prerequisite, you should have an account within the orq.ai admin panel. If not, please contact [email protected], and we'll help you out.

Step 1: Enable models in the Model Garden

You can browse the Model Garden by going to the Model Garden tab.

Here, you can find all models available to use within orq.ai, including their capabilities and a short description. To make any model available, toggle the button next to its name, it will then be usable in your first Deployments. If you want to use your own API keys, head to the Integrations tab.

📘

To learn more about all the models available, see Model Garden.

Step 2: Set up your first Deployment

To create your first deployment, head to the Deployment tab, then select Create Deployment.

You will then be taken to the Variant configuration screen. Here, you can select which parameters you want to use with your model and include the prompt(s) that will be sent to the language model.

Once ready, press the Deploy button at the top-right, this will save the latest configuration and make them available to reach through our API.

📘

There are many more features available to configure your Deployment. Including setting up multiple models, fallbacks, function calling, etc. To learn more, see Setting up a Deployment.

Step 3: Install the orq.ai SDK

Within the Deployment page, you can open the code snippet panel to find instructions to install and run the SDK for the current deployment.

Then, proceed to install your chosen SDK. Execute the following in a terminal to download the latest version of the SDK on your machine.

npm install @orq-ai/node --save

pip install orq-ai-sdk

Step 4: Get your API Token

To setup your client you will need an API key, this can be found within the Settings > API Keys menu.

📘

To learn more about our API Authentication, see Authentication.

Step 5: Use the SDK to invoke your Deployment

Once you have saved your API Token for use. You can refer back to the code snippet accessed before your Deployment to fetch the code to initialize and invoke a deployment:

Don't forget to replace <API_KEY>with the previously fetched API key.

import { createClient } from '@orq-ai/node';

const client = createClient({
  apiKey: '<API_KEY>',
  environment: 'production',
});

client.setUser({id: 2024})

const deployment = await client.deployments.invoke({
   key: "quickstart-deployment",
   context: {
      environments: []
   },
   metadata: {
      "custom-field-name": "custom-metadata-value"
   }
});

import os

from orq_ai_sdk import OrqAI

client = OrqAI(
  api_key=os.environ.get("ORQ_API_KEY", "<API_KEY>"),
  environment="production"
)

client.set_user(id=2024)

generation = client.deployments.invoke(
  key="quickstart-deployment",
  context={
    "environments": []
  },
  metadata={
    "custom-field-name": "custom-metadata-value"
  }
)

print(generation.choices[0].message.content)

👍

Congratulations, you successfully ran your first deployment invocation with orq.ai, to learn more about deployments and all the possibilities they offer, see Setting up a Deployment.

Step 6: Track your projects and costs

orq.ai allows you to track costs and projects grouped in Contacts. Contacts are people within your organization manipulating generation in your Workspace.

You can use a Contact within any SDK before invoking a generation to log all usage within that contact.

import { createClient } from '@orq-ai/node';

const client = createClient({
  apiKey: '<API_KEY>',
  environment: 'production',
});

client.setUser({id: 2024})

client.setContact({
  id: '<contact-id>',
  display_name: 'John Doe',
  email: '[email protected]',
  avatar_url: 'https://acme.com/john-doe.jpg',
  metadata: { role: 'admin' },
});


const deployment = await client.deployments.invoke({
   key: "quickstart-deployment",
   context: {
      environments: []
   },
   metadata: {
      "custom-field-name": "custom-metadata-value"
   }
});

import os

from orq_ai_sdk import OrqAI

client = OrqAI(
  api_key=os.environ.get("ORQ_API_KEY", "<API_KEY>"),
  environment="production"
)

client.set_user(id=2024)

client.set_contact(
    id="<contact-id>",
    display_name="John Doe",
    email="[email protected]",
    avatar_url="https://acme.com/john-doe.jpg",
    metadata={"role": "admin"},
)

generation = client.deployments.invoke(
  key="quickstart-deployment",
  context={
    "environments": []
  },
  metadata={
    "custom-field-name": "custom-metadata-value"
  }
)

print(generation.choices[0].message.content)

📘

To learn more about Contacts and the resulting Analytics, see Contact analytics