> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orq.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Build an intent classification chatbot

> Build and evaluate an intent classification system with Orq.ai. Categorize user queries for chatbots, customer support, and task automation with Python.

This quick guide demonstrates how to build and evaluate an AI application for intent classification using the Orq platform.

Before you begin, make sure you have an Orq account. If you don't have one, [sign up](https://orq.ai/create-account) first. Additionally, we’ve prepared a [Google Colab](https://colab.research.google.com/drive/1jbb7PKHfvRmqpQhtFMBQqVpHig_nmzZM?usp=sharing) file that you can copy and run immediately, simplifying the setup process. Just replace the API key, and you’re ready to go! After completing this tutorial, visit the [Orq documentation](/docs/introduction) for more advanced topics

**Why Intent Classification is Useful** Intent classification is a powerful tool for understanding and categorizing user inputs, making it essential for building smarter, more responsive applications. Here are a few use cases:

* Customer Support: Automatically categorize support tickets or chatbot queries to route them to the right team.
* E-commerce Personalization: Understand user intents like "buy," "browse," or "compare" to tailor shopping experiences.
* Task Automation: Trigger specific workflows based on user commands, such as scheduling meetings or setting reminders.

**Step 1: Install Dependencies** Start by installing the required packages. These include the Orq SDK and additional libraries for handling datasets and managing environment variables.

<CodeGroup>
  ```bash Bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
  !pip install orq-ai-sdk datasets huggingface_hub
  ```
</CodeGroup>

**Step 2: Identity Tracking (Optional)**

Identities in Orq.ai help track user interactions and API usage across your application. They can represent users, teams, or projects and enable better analytics and budget management.

**Create an Identity through the AI Studio:**

1. Go to **Identity Analytics** in your workspace
2. Click **Create an Identity**
3. Add the user details (name, email, external ID)
4. Set optional metadata and budget limits

<Info>
  Learn more about creating identities, see [Creating an Identity](/docs/analytics/identity#creating-an-identity-using-the-api).
</Info>

**Step 3: Initialize the Orq Client** The Orq client allows you to communicate with the Orq platform. Set it up using your API key, which can be stored as an environment variable (ORQ\_API\_KEY) or passed directly.

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  import os
  from orq_ai_sdk import Orq

  client = Orq(
      api_key=os.environ.get("ORQ_API_KEY", "your_orq_api_key"),
      identity_id="your_contact_id"  # Optional: add identity ID from Studio for tracking
  )
  ```
</CodeGroup>

**Step 4: Load a Dataset** Use the Hugging Face datasets library to load the dataset for intent classification. Here, we use a public dataset that contains user queries labeled with intents.

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  from datasets import load_dataset

  # Load dataset
  dataset = load_dataset("Bhuvaneshwari/intent_classification")
  df = dataset["train"].to_pandas()
  ```
</CodeGroup>

#### **Intent classification prompt**

This deployment is designed to classify user inputs into specific intents to understand the purpose behind their requests. The model will identify the most appropriate intent from a predefined set, enabling precise and context-aware responses.

Intent recognition is particularly useful when setting up chatbots

This is the prompt in Orq.ai:

<CodeGroup>
  ```bash Bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
  You are tasked with identifying the intent behind user inputs based on the following intents:
  'PlayMusic', 'AddToPlaylist', 'RateBook', 'Greetings', 'SearchScreeningEvent', 'BookRestaurant', 'GetWeather', 'Book Meeting', 'SearchCreativeWork', 'Cancellation', 'Affirmation', 'excitment'.

  Here are some examples: 

  Input: "Hey there, how are you doing?"
  Intent: Greetings

  Input: "Play the album Abbey Road by The Beatles."
  Intent: PlayMusic

  Input: "Add this song to my workout playlist."
  Intent: AddToPlaylist

  Input: "This book deserves a solid five-star rating."
  Intent: RateBook

  ONLY OUTPUT THE LABEL WITHOUT ''

  here is the input that needs intent classification: {{text}} 
  ```
</CodeGroup>

**Step 5: Invoke Orq Deployment** Integrate the Orq intent classification model by invoking a deployed model for predictions. Iterate through the dataset and store the results in a new column called "output".

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  # Placeholder for deployment key
  deployment_key = "intent_classification"

  # Add predictions to the DataFrame
  outputs = []
  for _, row in df.iterrows():
      generation = client.deployments.invoke(
          key=deployment_key,
          context={"environments": []},
          inputs={"text": row["text"]},
          metadata={"custom-field-name": "custom-metadata-value"},
      )
      outputs.append(generation.choices[0].message.content)

  # Add the output column
  df["output"] = outputs
  ```
</CodeGroup>

**Step 6: Evaluate Model Performance** Use metrics such as accuracy, precision, recall, and F1-score to assess the quality of your model's predictions.

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
  from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score

  # Calculate metrics
  accuracy = accuracy_score(df["intent"], df["output"])
  precision = precision_score(df["intent"], df["output"], average="macro")
  recall = recall_score(df["intent"], df["output"], average="macro")
  f1 = f1_score(df["intent"], df["output"], average="macro")

  # Display results
  print(f"Accuracy: {accuracy:.4f}")
  print(f"Precision: {precision:.4f}")
  print(f"Recall: {recall:.4f}")
  print(f"F1 Score: {f1:.4f}")
  ```
</CodeGroup>

**Feedback Collection (Optional)**

Feedback in Orq.ai helps track response quality and identify areas for improvement. You can collect user ratings, defect classifications, and corrections to continuously enhance your application.

**Provide feedback through the AI Studio:**

1. Go to **Logs** in your workspace
2. Find the specific deployment invocation
3. Use the feedback interface to rate responses
4. Add defect classifications or corrections as needed

<Info>
  You can also [collect feedback programmatically via the API](/docs/tutorials/capturing-feedback-with-orq) if needed.
</Info>

**Next Steps** Congratulations! You've successfully built and evaluated an intent classification application using Orq. To further enhance your application:

* Explore other datasets and use cases.
* Integrate the model into a chatbot or voice assistant.
* Automate deployment and testing with Orq’s advanced features.

For more resources, visit the Orq documentation.