HomepageAdmin PortalSign up
Resources

Cohere - Prompt Management

This article guides you through integrating your SaaS with orq.ai and Cohere using our Python SDK. By the end of the article, you'll know how to set up a Deployment in orq.ai, perform prompt engineering, request a Deployment variant using our SDK code generator, send a payload to Cohere, and add additional information to the request.

Suggest Edits

This guide shows you how to integrate your products with Cohere using orq.ai Python SDK.

Step 1: Install the SDK

# orquesta sdk
pip install orq-ai-sdk

# cohere
pip install cohere

// orquesta sdk
npm install @orq-ai/node --save

// cohere 
npm install -s cohere-ai

Step 2: Enable models in the Model Garden

Orq.ai allows you to pick and enable the models of your choice and work with them. Enabling a model(s) is very easy; all you have to do is navigate to the Model Garden and toggle on the model of your choice.

Step 3: Execute prompt

You can find your orq.ai API Key in your workspace: <https://my.orq.ai/><workspace-name>/settings/developers

import cohere

from orq_ai_sdk import OrqAI

api_key = "ORQ_API_KEY"

client = OrqAI(
  api_key=api_key,
  environment="production"
)

# Get deployment configuration
deployment_config = client.deployments.get_config(
  key="Deplyment-with-Cohere",
  context={ "environments": [ "production" ], "locale": [ "en" ] },
  metadata={"custom-field-name":"custom-metadata-value"}
)

config = deployment_config.to_dict()

# Convert the array of messages to a single string
prompt = "\n".join(message['content'] for message in config['messages'])

# Initialize the Cohere client
co = cohere.Client('COHERE_API_KEY') # Insert your Cohere API key
completion = co.generate(
    model=config['model'],
    prompt=prompt,
    temperature = config['parameters']['temperature'],
    max_tokens = config['parameters']['maxTokens'],
)

print(completion)

import { createClient } from '@orq-ai/node';
import { CohereClient } from "cohere-ai";

const client = createClient({
  apiKey: 'ORQ_API_KEY',
  environment: 'production',
});

// Get Orq Deployment Config
const deploymentConfig = await client.deployments.getConfig({
   key: "deployment-with-cohere",
   context: {
      environments: ['production'],
      locale: ['en']
   },
   metadata: {
      "custom-field-name": "custom-metadata-value"
   }
});

const prompt = deploymentConfig.messages
   .map((message) => message.content)
   .join('\n');

const cohere = new CohereClient({
    token: "COHERE_API_KEY",
});

// Query Cohere for Prediction
(async () => {
    const prediction = await cohere.generate({
      prompt: prompt,
      model: deploymentConfig.model,
      maxTokens: deploymentConfig.parameters.maxTokens,
      temperature: deploymentConfig.parameters.temperature,
    });

    console.log('Received prediction', prediction);
})();

Step 4: Report analytics back to orq.ai

Add metrics to each transaction using the add_metrics method. Below, you can find an example with all the supersets of metrics that can be reported.

deployment_config.add_metrics(
  chain_id="c4a75b53-62fa-401b-8e97-493f3d299316",
  conversation_id="ee7b0c8c-eeb2-43cf-83e9-a4a49f8f13ea",
  user_id="e3a202a6-461b-447c-abe2-018ba4d04cd0",
  feedback={"score": 100},
  metadata={
      "custom": "custom_metadata",
      "chain_id": "ad1231xsdaABw",
  },
  usage={
      "prompt_tokens": 100,
      "completion_tokens": 900,
      "total_tokens": 1000,
  },
  performance={
      "latency": 9000,
      "time_to_first_token": 250,
  }
)

deployment.addMetrics({
  chain_id: "c4a75b53-62fa-401b-8e97-493f3d299316",
  conversation_id: "ee7b0c8c-eeb2-43cf-83e9-a4a49f8f13ea",
  user_id: "e3a202a6-461b-447c-abe2-018ba4d04cd0",
  feedback: {
    score: 100
  },
  metadata: {
    custom: "custom_metadata",
    chain_id: "ad1231xsdaABw"
  },
  usage: {
    prompt_tokens: 100,
    completion_tokens: 900,
    total_tokens: 1000
  },
  performance: {
    latency: 9000,
    time_to_first_token: 250
  }
})

Updated about 1 month ago

What’s Next