Function calling in Deployments

Deployments Tools let you use function calling abilities available on models that support it.

Function calling is a way for a model to output structured text automatically. The structured text follows a set of user-defined variables and is formatted in JSON. It can then be used to call any other function or system.

Function calling is a safe way to integrate a model with any external service, with common use-cases like:

• Answering user queries via external API calls.
• Converting natural language to a structured function call.
• Extracting structured data from any text.

Declaring a Tool

You can declare a new function within the Tools panel on the Variant by selecting the Add Tool button.

📘

The Tool panel will only appear with models compatible with Function Calling.

You will be presented with a new function for which you can define name, description, and JSON schema.

Here is an example of a JSON object for a function get_current_weather that declares the fields location (string)and unit (string):

{
  "type": "object",
  "required": [
    "location",
    "unit"
  ],
  "properties": {
    "unit": {
      "type": "string",
      "description": "The temperature unit, e.g. Celsius"
    },
    "location": {
      "type": "string",
      "description": "The city and state, e.g. San Francisco, CA"
    }
  }
}

Importing an existing Tool

By selecting the Import button, you can import an existing Tool from your Resources.

To learn how to add a Tool to your resources, see Tools.

Using a Tool

Using the previous example, we are able to use a prompt template that Gives the current weather for {{city}}, {{country}}, choose a unit fitting to the location and automatically converts the answer to the desired structured output.

Once your Variant is ready to be used, you can proceed with Integrating your Deployment. Reaching this variant will use function calling in its response, producing the following example payload:

{
    "id": "<id>",
    "created": "2024-05-06T09:14:16.139Z",
    "object": "chat",
    "model": "gpt-4",
    "provider": "openai",
    "is_final": true,
    "finalized": "2024-05-06T09:14:19.831Z",
    "system_fingerprint": null,
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": null,
                "tool_calls": [
                    {
                        "type": "function",
                        "id": "<call_id>",
                        "function": {
                            "name": "get_current_weather",
                            "arguments": "{\n  \"unit\": \"Celsius\",\n  \"location\": \"Amsterdam, Netherlands\"\n}"
                        }
                    }
                ]
            },
            "finish_reason": "tool_calls"
        }
    ]
}`