Intent Classification with Orq.ai

This quick guide demonstrates how to build and evaluate an AI application for intent classification using the Orq platform.

Before you begin, make sure you have an Orq account. If you don't have one, sign up first. Additionally, we’ve prepared a Google Colab file that you can copy and run immediately, simplifying the setup process. Just replace the API key, and you’re ready to go! After completing this tutorial, visit the Orq documentation for more advanced topics

Why Intent Classification is Useful
Intent classification is a powerful tool for understanding and categorizing user inputs, making it essential for building smarter, more responsive applications. Here are a few use cases:

• Customer Support: Automatically categorize support tickets or chatbot queries to route them to the right team.
• E-commerce Personalization: Understand user intents like "buy," "browse," or "compare" to tailor shopping experiences.
• Task Automation: Trigger specific workflows based on user commands, such as scheduling meetings or setting reminders.

Step 1: Install Dependencies
Start by installing the required packages. These include the Orq SDK and additional libraries for handling datasets and managing environment variables.

!pip install orq-ai-sdk datasets huggingface_hub

Step 2: Initialize the Orq Client
The Orq client allows you to communicate with the Orq platform. Set it up using your API key, which can be stored as an environment variable (ORQ_API_KEY) or passed directly.

import os
from orq_ai_sdk import OrqAI

client = OrqAI(
    api_key=os.environ.get("ORQ_API_KEY", "your_orq_api_key"),
    environment="production"
)

Step 3: Load a Dataset
Use the Hugging Face datasets library to load the dataset for intent classification. Here, we use a public dataset that contains user queries labeled with intents.

from datasets import load_dataset

# Load dataset
dataset = load_dataset("Bhuvaneshwari/intent_classification")
df = dataset["train"].to_pandas()

Intent classification prompt

This deployment is designed to classify user inputs into specific intents to understand the purpose behind their requests. The model will identify the most appropriate intent from a predefined set, enabling precise and context-aware responses.

Intent recognition is particularly useful when setting up chatbots

This is the prompt in Orq.ai:

You are tasked with identifying the intent behind user inputs based on the following intents:
'PlayMusic', 'AddToPlaylist', 'RateBook', 'Greetings', 'SearchScreeningEvent', 'BookRestaurant', 'GetWeather', 'Book Meeting', 'SearchCreativeWork', 'Cancellation', 'Affirmation', 'excitment'.

Here are some examples: 

Input: "Hey there, how are you doing?"
Intent: Greetings

Input: "Play the album Abbey Road by The Beatles."
Intent: PlayMusic

Input: "Add this song to my workout playlist."
Intent: AddToPlaylist

Input: "This book deserves a solid five-star rating."
Intent: RateBook

ONLY OUTPUT THE LABEL WITHOUT ''

here is the input that needs intent classification: {{text}} 

Step 4: Invoke Orq Deployment
Integrate the Orq intent classification model by invoking a deployed model for predictions. Iterate through the dataset and store the results in a new column called "output".

# Placeholder for deployment key
deployment_key = "intent_classification"

# Add predictions to the DataFrame
outputs = []
for _, row in df.iterrows():
    generation = client.deployments.invoke(
        key=deployment_key,
        context={"environments": []},
        inputs={"text": row["text"]},
        metadata={"custom-field-name": "custom-metadata-value"}
    )
    outputs.append(generation.choices[0].message.content)

# Add the output column
df["output"] = outputs

Step 5: Evaluate Model Performance
Use metrics such as accuracy, precision, recall, and F1-score to assess the quality of your model's predictions.

from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score

# Calculate metrics
accuracy = accuracy_score(df["intent"], df["output"])
precision = precision_score(df["intent"], df["output"], average="macro")
recall = recall_score(df["intent"], df["output"], average="macro")
f1 = f1_score(df["intent"], df["output"], average="macro")

# Display results
print(f"Accuracy: {accuracy:.4f}")
print(f"Precision: {precision:.4f}")
print(f"Recall: {recall:.4f}")
print(f"F1 Score: {f1:.4f}")

Next Steps
Congratulations! You've successfully built and evaluated an intent classification application using Orq. To further enhance your application:

• Explore other datasets and use cases.
• Integrate the model into a chatbot or voice assistant.
• Automate deployment and testing with Orq’s advanced features.

For more resources, visit the Orq documentation.