Tool calling and function execution

Use Cases

Connecting models to live data (databases, calendars, internal APIs) without prompt hacks.
Agents that take actions on behalf of users: create tickets, send emails, run queries.
Multi-step workflows where the model decides which tools to invoke and in what order.
Replacing brittle regex parsing with structured function calls for data extraction.

Quick Start

Enable AI models to call external functions with structured parameters.

curl -X POST https://api.orq.ai/v3/router/responses \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.4",
    "input": "What is the weather in NYC?",
    "tools": [{
      "type": "function",
      "name": "get_weather",
      "description": "Get current weather for a location",
      "parameters": {
        "type": "object",
        "properties": {
          "location": { "type": "string", "description": "City and state" },
          "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] }
        },
        "required": ["location"]
      }
    }],
    "tool_choice": "auto"
  }'

Configuration

Tool Definition

Responses API (/v3/router/responses): flat shape:

Parameter	Type	Required	Description
`type`	`"function"`	Yes	Tool type
`name`	string	Yes	Function identifier
`description`	string	Yes	What the function does
`parameters`	object	Yes	JSON Schema for parameters

Chat Completions (/v3/router/chat/completions): nested function wrapper:

Parameter	Type	Required	Description
`type`	`"function"`	Yes	Tool type
`function.name`	string	Yes	Function identifier
`function.description`	string	Yes	What the function does
`function.parameters`	object	Yes	JSON Schema for parameters

Tool Choice Options

Value	Behavior
`"auto"`	Model decides when to use tools
`"none"`	Disable tool usage
`"required"`	Force tool usage
`{type: "function", function: {name: "tool_name"}}`	Force specific tool

Tool Message Format

This format applies to the Chat Completions endpoint (/v3/router/chat/completions). On the Responses API, tool results use type: "function_call_output", call_id, and output instead.

When providing tool results back to the model, use the tool role:

Parameter	Type	Required	Description
`role`	`"tool"`	Yes	Message role for tool results
`tool_call_id`	string \| null	Yes	ID of the tool call being responded to
`content`	string	Yes	JSON-stringified result of the tool execution

The tool_call_id can be null in certain scenarios, such as when tool results are being provided without a corresponding tool call from the model, or when working with providers that don’t require tool call IDs.

Code examples

curl -X POST https://api.orq.ai/v3/router/responses \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.4",
    "input": "What is the weather like in San Francisco?",
    "tools": [
      {
        "type": "function",
        "name": "get_weather",
        "description": "Get the current weather for a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g. San Francisco, CA"
            },
            "unit": {
              "type": "string",
              "enum": ["celsius", "fahrenheit"],
              "description": "The temperature unit"
            }
          },
          "required": ["location"]
        }
      }
    ],
    "tool_choice": "auto"
  }'

Function Execution Patterns

Basic Tool Handler

A registry that maps tool names to handler functions, replacing per-call switch statements with a single dispatch path.

class ToolHandler {
  constructor() {
    this.tools = new Map();
  }

  register(name, func, schema) {
    this.tools.set(name, { func, schema });
  }

  async execute(toolCall) {
    const tool = this.tools.get(toolCall.function.name);
    if (!tool) throw new Error(`Unknown tool: ${toolCall.function.name}`);

    const args = JSON.parse(toolCall.function.arguments);
    const result = await tool.func(args);

    return {
      role: "tool",
      tool_call_id: toolCall.id,
      content: JSON.stringify(result),
    };
  }
}

// Stubs - replace with your actual implementations
const weatherSchema = { type: "object", properties: { location: { type: "string" } }, required: ["location"] };
const searchSchema = { type: "object", properties: { query: { type: "string" } }, required: ["query"] };
const getWeatherAPI = async ({ location }) => ({ temperature: 72, condition: "sunny", location });
const searchWebAPI = async ({ query }) => ({ results: [`Result for: ${query}`] });
const response = { choices: [{ message: { tool_calls: [] } }] };

const handler = new ToolHandler();
handler.register("get_weather", getWeatherAPI, weatherSchema);
handler.register("search_web", searchWebAPI, searchSchema);

// Execute tool calls
const toolResults = await Promise.all(
  response.choices[0].message.tool_calls.map((call) => handler.execute(call)),
);

Parallel Tool Execution

When the model returns multiple tool calls in one turn, execute them concurrently to reduce latency.

async function getWeatherAsync(args: Record<string, unknown>) { return { temperature: 72, condition: "sunny" }; } // replace with your weather API
async function searchProductsAsync(args: Record<string, unknown>) { return { products: [] as unknown[] }; } // replace with your product search
async function checkInventoryAsync(args: Record<string, unknown>) { return { inStock: true }; } // replace with your inventory API

// Replace with your actual response and messages array
const response = { choices: [{ message: { role: "assistant" as const, content: null as string | null, tool_calls: [] as Array<{ function: { name: string; arguments: string }; id: string }> } }] }; // e.g., client.chat.completions.create(...)
let messages: Array<{ role: string; tool_call_id?: string; content: string | null }> = [];

const dispatch: Record<string, (args: Record<string, unknown>) => Promise<unknown>> = {
  get_weather: getWeatherAsync,
  search_products: searchProductsAsync,
  check_inventory: checkInventoryAsync,
};

async function executeToolsParallel(
  toolCalls: Array<{ function: { name: string; arguments: string }; id: string }>,
) {
  return Promise.all(
    toolCalls.map(async (call) => {
      const args = JSON.parse(call.function.arguments) as Record<string, unknown>;
      const fn = dispatch[call.function.name] ?? (() => Promise.resolve({ error: `Unknown: ${call.function.name}` }));
      const result = await fn(args);
      return { role: "tool" as const, tool_call_id: call.id, content: JSON.stringify(result) };
    }),
  );
}

if (response.choices[0].message.tool_calls) {
  messages.push(response.choices[0].message);
  const toolResults = await executeToolsParallel(response.choices[0].message.tool_calls);
  messages = [...messages, ...toolResults];
}

Advanced Use Cases

Database Integration

Expose SQL access as a tool so the model can query data directly. Always sanitize queries before execution.

const sanitizeSql = (q: string) => q; // replace with your SQL sanitizer
const getDbConnection = () => null as unknown as { execute: (q: string) => Promise<{ fetchAll: () => unknown[] }> }; // replace with your DB client

const db = getDbConnection();

const tools = [
  {
    type: "function" as const,
    function: {
      name: "query_database",
      description: "Query the customer database",
      parameters: {
        type: "object",
        properties: {
          query: { type: "string", description: "SQL query to execute" },
          limit: { type: "integer", description: "Maximum number of results" },
        },
        required: ["query"],
      },
    },
  },
];

async function queryDatabase(args: { query: string; limit?: number }) {
  const query = sanitizeSql(args.query);
  const limit = args.limit ?? 10;
  const results = await db.execute(`${query} LIMIT ${limit}`); // limit is integer-typed in the schema, safe to interpolate directly
  return { results: results.fetchAll() };
}

API Integration

Expose external service actions (email, calendar, notifications) as tools the model can invoke during a conversation.

const apiTools = [
  {
    type: "function",
    function: {
      name: "send_email",
      description: "Send an email to a recipient",
      parameters: {
        type: "object",
        properties: {
          to: { type: "string", description: "Email address" },
          subject: { type: "string", description: "Email subject" },
          body: { type: "string", description: "Email content" },
        },
        required: ["to", "subject", "body"],
      },
    },
  },
  {
    type: "function",
    function: {
      name: "create_calendar_event",
      description: "Create a calendar event",
      parameters: {
        type: "object",
        properties: {
          title: { type: "string" },
          start_time: { type: "string", format: "date-time" },
          duration: { type: "integer", description: "Duration in minutes" },
          attendees: { type: "array", items: { type: "string" } },
        },
        required: ["title", "start_time"],
      },
    },
  },
];

const emailAPI = { send: async (args: Record<string, unknown>) => ({ messageId: "msg_001" }) }; // replace with your email client
const calendarAPI = { createEvent: async (args: Record<string, unknown>) => ({ eventId: "evt_001" }) }; // replace with your calendar client

const executeApiTool = async (toolCall) => {
  const { name } = toolCall.function;
  const args = JSON.parse(toolCall.function.arguments);

  switch (name) {
    case "send_email":
      return await emailAPI.send(args);
    case "create_calendar_event":
      return await calendarAPI.createEvent(args);
    default:
      throw new Error(`Unknown API tool: ${name}`);
  }
};

Multi-Steps Workflows

Run the model in a loop until it produces a final text response, enabling multi-step agent behavior without streaming.

import { OpenAI } from "openai";
import type { ChatCompletionMessageParam, ChatCompletionTool } from "openai/resources";

const client = new OpenAI({
  apiKey: process.env.ORQ_API_KEY,
  baseURL: "https://api.orq.ai/v3/router",
});

class WorkflowEngine {
  private tools: Record<string, { func: (args: Record<string, unknown>) => Promise<unknown>; schema: ChatCompletionTool }> = {};

  register(name: string, func: (args: Record<string, unknown>) => Promise<unknown>, schema: ChatCompletionTool) {
    this.tools[name] = { func, schema };
  }

  async executeWorkflow(initialPrompt: string, maxSteps = 10): Promise<string> {
    const conversation: ChatCompletionMessageParam[] = [{ role: "user", content: initialPrompt }];

    for (let step = 0; step < maxSteps; step++) {
      const response = await client.chat.completions.create({
        model: "openai/gpt-5.4",
        messages: conversation,
        tools: Object.values(this.tools).map((t) => t.schema),
        tool_choice: "auto",
      });

      conversation.push(response.choices[0].message);

      if (!response.choices[0].message.tool_calls) {
        return response.choices[0].message.content ?? "";
      }

      for (const call of response.choices[0].message.tool_calls) {
        conversation.push(await this.executeTool(call));
      }
    }
    return "Workflow exceeded maximum steps";
  }

  private async executeTool(toolCall: { function: { name: string; arguments: string }; id: string }): Promise<ChatCompletionMessageParam> {
    const args = JSON.parse(toolCall.function.arguments) as Record<string, unknown>;
    const tool = this.tools[toolCall.function.name];
    const result = tool ? await tool.func(args) : { error: `Unknown tool: ${toolCall.function.name}` };
    return { role: "tool", tool_call_id: toolCall.id, content: JSON.stringify(result) };
  }
}

Error Handling

Return structured error objects in tool outputs so the model can report failures or retry with corrected arguments.

const validateArgs = (name: string, args: Record<string, unknown>) => ({ valid: true, errors: [] as string[] }); // replace with your validator
const executeFunction = async (name: string, args: Record<string, unknown>): Promise<unknown> => ({}); // replace with your dispatcher

const safeToolExecution = async (toolCall) => {
  try {
    const args = JSON.parse(toolCall.function.arguments);

    // Validate arguments
    const validation = validateArgs(toolCall.function.name, args);
    if (!validation.valid) {
      return {
        role: "tool",
        tool_call_id: toolCall.id,
        content: JSON.stringify({
          error: "Invalid arguments",
          details: validation.errors,
        }),
      };
    }

    // Execute with timeout
    const result = await Promise.race([
      executeFunction(toolCall.function.name, args),
      new Promise((_, reject) =>
        setTimeout(() => reject(new Error("Tool execution timeout")), 30000),
      ),
    ]);

    return {
      role: "tool",
      tool_call_id: toolCall.id,
      content: JSON.stringify(result),
    };
  } catch (error) {
    console.error(`Tool execution failed: ${error.message}`);

    return {
      role: "tool",
      tool_call_id: toolCall.id,
      content: JSON.stringify({
        error: "Tool execution failed",
        message: error.message,
      }),
    };
  }
};

Best Practices

Tool design

Use clear, descriptive function names.
Provide detailed parameter descriptions.
Include examples in descriptions.
Make tools idempotent when possible.

Schema design

{
  "type": "object",
  "properties": {
    "location": {
      "type": "string",
      "description": "City and state (e.g., 'San Francisco, CA')",
      "examples": ["New York, NY", "London, UK"]
    },
    "units": {
      "type": "string",
      "enum": ["metric", "imperial"],
      "description": "Temperature unit system",
      "default": "metric"
    }
  },
  "required": ["location"]
}

Security considerations

Never expose destructive operations directly.
Validate all inputs thoroughly.
Use allowlists for sensitive operations.
Implement proper authentication.
Log all tool executions.

Troubleshooting

Tool not being called

Check tool descriptions are clear.
Verify parameter schemas are correct.
Ensure tool_choice is set appropriately.
Try more explicit prompts. Invalid arguments
Validate JSON Schema thoroughly.
Add parameter examples.
Check required fields are marked.
Simplify complex parameter structures.

Execution failures

Implement proper error handling.
Add timeout protection.
Validate inputs before execution.
Return structured error messages.

Limitations

Limitation	Impact	Workaround
Tool limit	Max ~20 tools per request	Group related functions
Parameter size	Large schemas may fail	Simplify parameter structure
Execution time	Tools block response	Use async patterns
Error propagation	Failures can break workflow	Implement error recovery
Model differences	Varying tool calling quality	Test across models

Performance Optimization

Tool caching

Cache results from read-only tools to avoid redundant external calls within a session.

class CachedToolExecutor {
  constructor() {
    this.cache = new Map();
    this.cacheTTL = 300000; // 5 minutes
  }

  getCacheKey(toolCall) {
    return `${toolCall.function.name}:${toolCall.function.arguments}`;
  }

  async executeFunction(toolCall) {
    throw new Error("executeFunction must be implemented in a subclass");
  }

  async execute(toolCall) {
    const key = this.getCacheKey(toolCall);
    const cached = this.cache.get(key);

    if (cached && Date.now() - cached.timestamp < this.cacheTTL) {
      return cached.result;
    }

    const result = await this.executeFunction(toolCall);
    this.cache.set(key, { result, timestamp: Date.now() });

    return result;
  }
}

Batch operations

Accept arrays of inputs in a single tool to reduce the number of model turns required for bulk operations.

const getWeather = (location: string) => ({ temperature: 72, condition: "sunny", location }); // replace with your weather API

const getWeatherBatch = (locations: string[]) =>
  Object.fromEntries(locations.map((loc) => [loc, getWeather(loc)]));

const getWeatherBatchTool = {
  type: "function" as const,
  name: "get_weather_batch",
  description: "Get weather for multiple locations in a single call",
  parameters: {
    type: "object",
    properties: {
      locations: { type: "array", items: { type: "string" }, description: "List of city names" },
    },
    required: ["locations"],
  },
};

​Quick Start

​Configuration

​Tool Definition

​Tool Choice Options

​Tool Message Format

​Code examples

​Function Execution Patterns

​Basic Tool Handler

​Parallel Tool Execution

​Advanced Use Cases

​Database Integration

​API Integration

​Multi-Steps Workflows

​Error Handling

​Best Practices

​Tool design

​Schema design

​Security considerations

​Troubleshooting

​Limitations

​Performance Optimization

​Tool caching

​Batch operations

Quick Start

Configuration

Tool Definition

Tool Choice Options

Tool Message Format

Code examples

Function Execution Patterns

Basic Tool Handler

Parallel Tool Execution

Advanced Use Cases

Database Integration

API Integration

Multi-Steps Workflows

Error Handling

Best Practices

Tool design

Schema design

Security considerations

Troubleshooting

Limitations

Performance Optimization

Tool caching

Batch operations