How to Implement Tool Use

A guide to integrating and utilizing tools with NeosantaraAI models for enhanced capabilities.

Choosing a Model

Generally, models with function_calling capabilities are suitable for tool use. For complex tools and ambiguous queries, models like nusantara-base, gpt-4o-mini, gemini-1.5-pro, or command-r-plus are recommended as they handle tool selection and parameter inference more robustly.

For more straightforward tools, models like bahasa-llm or gemini-1.5-flash can be effective.

Specifying Tools

Tools (functions) are specified in the tools top-level parameter of your API request. Each tool definition includes:

Parameter	Description
type	Must be "function".
function	An object defining the function’s metadata, including its name, description, and parameters.
name	(within function): The name of the tool. Must match the regex ^[a-zA-Z0-9_-]{1,64}$.
description	(within function): A detailed plaintext description of what the tool does, when it should be used, and how it behaves. This is crucial for the model to understand the tool’s purpose.
parameters	(within function): A JSON Schema object defining the expected parameters for the tool. This schema describes the input the tool expects.

Example Simple Tool Definition

{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Get the current weather in a given location. Use this tool when the user asks about weather conditions for a specific city. Ensure to ask for the city if not provided.",
    "parameters": {
      "type": "object",
      "properties": {
        "location": {
          "type": "string",
          "description": "The city and state, e.g. Jakarta, ID or New York, USA"
        },
        "unit": {
          "type": "string",
          "enum": ["celsius", "fahrenheit"],
          "description": "The unit of temperature, either 'celsius' or 'fahrenheit'"
        }
      },
      "required": ["location"]
    }
  }
}

This tool, named get_weather, expects an input object with a required location string and an optional unit string that must be either “celsius” or “fahrenheit”.

Best Practices for Tool Definitions

To get the best performance out of NeosantaraAI when using tools, follow these guidelines:

• Provide extremely detailed descriptions. This is by far the most important factor in tool performance. Your descriptions should explain every detail about the tool, including:
  • What the tool does.
  • When it should be used (and when it shouldn’t).
  • What each parameter means and how it affects the tool’s behavior.
  • Any important caveats or limitations, such as what information the tool does not return if the tool name is unclear. The more context you can give the model about your tools, the better it will be at deciding when and how to use them. Aim for at least 3-4 sentences per tool description, more if the tool is complex.
• Prioritize descriptions over examples. While you can include examples of how to use a tool in its description or in the accompanying prompt, this is less important than having a clear and comprehensive explanation of the tool’s purpose and parameters. Only add examples after you’ve fully fleshed out the description.

Example of a Good Tool Description

{
  "name": "get_stock_price",
  "description": "Retrieves the current stock price for a given ticker symbol. The ticker symbol must be a valid symbol for a publicly traded company on a major stock exchange (e.g., NYSE, NASDAQ, IDX). The tool will return the latest trade price in USD (or IDR if on IDX). It should be used when the user asks about the current or most recent price of a specific stock. It will not provide any other information about the stock or company.",
  "parameters": {
    "type": "object",
    "properties": {
      "ticker": {
        "type": "string",
        "description": "The stock ticker symbol, e.g. AAPL for Apple Inc. or BBCA for Bank Central Asia."
      }
    },
    "required": ["ticker"]
  }
}

Example Poor Tool Description

{
  "name": "get_stock_price",
  "description": "Gets the stock price for a ticker.",
  "parameters": {
    "type": "object",
    "properties": {
      "ticker": {
        "type": "string"
      }
    },
    "required": ["ticker"]
  }
}

The good description clearly explains what the tool does, when to use it, what data it returns, and what the ticker parameter means. The poor description is too brief and leaves the model with many open questions about the tool’s behavior and usage.

Controlling Model’s Output

Forcing Tool Use

In some cases, you may want NeosantaraAI to use a specific tool to answer the user’s question, even if the model thinks it can provide an answer without using a tool. You can do this by specifying the tool in the tool_choice field.

The tool_choice parameter offers four possible options:

• auto: (Default when tools are provided) Allows the model to decide whether to call any provided tools or not.
• any: Tells the model that it must use one of the provided tools, but doesn’t force a particular tool.
• {"type": "function", "function": {"name": "your_tool_name"}}: Allows you to force the model to always use a particular tool.
• none: Prevents the model from using any tools. (Default when no tools are provided).

This diagram illustrates how each option works:

If you would like to keep chain-of-thought (where the model provides its reasoning steps) while still requesting that the model use a specific tool, you can use tool_choice: "auto" (the default) and add explicit instructions in a user message. For example: "What's the weather like in London? Use the get_weather tool in your response."

JSON Output

Tools do not necessarily need to be client functions — you can use tools anytime you want the model to return JSON output that follows a provided schema. For example, you might use a record_summary tool with a particular schema to extract structured data from text or images.

When using tools for JSON output:

• You usually provide a single tool definition.
• You should set tool_choice to {"type": "function", "function": {"name": "your_tool_name"}} to instruct the model to explicitly use that tool.
• The function’s parameters define the exact JSON schema the model should adhere to.

See the Tool use with NeosantaraAI overview for a full working example of JSON output for image description.

Chain of Thought

When using tools, NeosantaraAI models often show their “chain of thought”, i.e., the step-by-step reasoning they use to break down the problem and decide which tools to use. This provides insight into the model’s reasoning process and can help you debug unexpected behavior.

For example, given the prompt “What’s the weather like in Jakarta right now, and what time is it there?”, a model might implicitly reason:

1. Identify intent: User wants weather and time for Jakarta.
2. Identify tools: get_weather and get_time tools are available.
3. Plan execution: Both operations are independent, so they can be called in parallel.
4. Formulate calls: Generate tool_calls for get_weather(location='Jakarta, ID') and get_time(timezone='Asia/Jakarta').

Your code should then execute these tool_calls and feed the results back to the model.

Handling Tool Use and Tool Result Content Blocks

NeosantaraAI’s response for tool use will have a finish_reason of tool_calls and one or more tool_calls content blocks within the assistant’s message. These blocks include:

• id: A unique identifier for this particular tool call. This will be used to match up the tool results later.
• type: Always "function".
• function: An object containing the name of the tool (name) and the arguments (arguments) to be passed to it as a JSON string.

Example API Response with `tool_calls` Content Block

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "created": 1701234567,
  "model": "nusantara-base",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_toolu_01A09q90qw90lq917835lq9",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"location\": \"Jakarta, ID\", \"unit\": \"celsius\"}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ],
  "usage": { /* ... */ },
  "_metadata": { /* ... */ }
}

When you receive a tool use response, you should:

1. Extract the name, id, and arguments (parsing the JSON string) from the tool_calls block.
2. Run the actual tool in your codebase corresponding to that tool name, passing in the parsed tool arguments.
3. Continue the conversation by sending a new message with the role of tool, and a content block containing the result of your tool’s execution. This message must also include the tool_call_id to link it back to the original tool request.
   • tool_call_id: The id of the tool_calls request this is a result for.
   • content: The result of the tool, as a string. For complex results, it’s often a JSON string.
   • is_error (optional): Set to true if the tool execution resulted in an error. NeosantaraAI will then incorporate this error into its response.

Important Formatting Requirements:

• Tool messages ("role": "tool") must always immediately follow their corresponding assistant message that contained the tool_calls. You cannot include any other messages between them.
• When returning results for parallel tool calls, all tool messages must be provided in a single messages array in the subsequent API call.

Example of Successful Tool Result

{
  "role": "tool",
  "tool_call_id": "call_toolu_01A09q90qw90lq917835lq9",
  "content": "{\"temperature\": 28, \"unit\": \"celsius\", \"description\": \"Partly cloudy\"}"
}

Example of Tool Result with Error

If the tool itself throws an error during execution (e.g., a network error when fetching weather data), you can return the error message in the content along with "is_error": true:

{
  "role": "tool",
  "tool_call_id": "call_toolu_01A09q90qw90lq917835lq9",
  "content": "ConnectionError: the weather service API is not available (HTTP 500)",
  "is_error": true
}

NeosantaraAI will then incorporate this error into its response to the user, e.g., “I’m sorry, I was unable to retrieve the current weather because the weather service API is not available. Please try again later.”

After receiving the tool result, NeosantaraAI will use that information to continue generating a response to the original user prompt.

Troubleshooting

Parallel Tool Calls Not Working

If NeosantaraAI isn’t making parallel tool calls when expected, check these common issues:

1. Incorrect Tool Result Formatting

The most common issue is formatting tool results incorrectly in the conversation history. This “teaches” the model to avoid parallel calls. All tool results from a single assistant’s tool_calls message must be sent in a single tool message (each result in its own content block if there were multiple parallel calls with tool_call_id and content).

2. Weak Prompting

While models generally attempt parallel calls, you can use stronger language in your prompts to encourage it, for example: “Please use parallel tool calls to get the weather for Jakarta and Bandung at the same time.”

3. Model-Specific Behavior

Different models may have varying propensities for parallel tool use. Experiment with different models to find one that suits your needs.

Invalid Tool Name or Parameters

If the model’s attempted use of a tool is invalid (e.g., missing required parameters), it usually means that there wasn’t enough information for the model to use the tool correctly, or the tool definition was unclear. Your best bet during development is to try the request again with more-detailed description values in your tool definitions.

If a tool request is invalid or missing parameters, NeosantaraAI will typically retry 2-3 times with corrections before apologizing to the user. You can also return an error in the tool message by setting "is_error": true in its content.

Pricing

Tool use requests are priced based on:

1. The total number of input tokens sent to the model (including the tokens from tool definitions in the tools parameter and tool call/result messages).
2. The number of output tokens generated (including tool calls generated by the model).
3. Additional charges may apply for specific capabilities (e.g., vision processing for image inputs).

These token counts are added to your normal input and output tokens to calculate the total cost of a request.

Refer to our pricing documentation for current per-model prices and usage tiers.

When you send a tool use prompt, just like any other API request, the response will output both input and output token counts as part of the reported usage metrics.

Next Steps

Explore our other capabilities and API references:

Image Generation

Generate high-quality images from text prompts.

Embeddings

Convert text into numerical vectors for semantic search.