Effective tool selection is critical for tool-using agents. It ensures that agents choose the right tool for each task, leading to accurate, efficient, and purposeful interactions. This guide explores strategies and techniques for implementing your own tool selection in Prisme.ai.

These guides provide example YAML files for educational purposes, showcasing the range of possibilities. It is up to the technical teams to implement them according to your internal tools and APIs.

​
Understanding Tool Selection

Tool selection is the process by which an AI agent:

  1. Recognizes when a tool is needed to fulfill a user request
  2. Evaluates which available tool is most appropriate
  3. Prepares the necessary parameters for tool execution
  4. Decides whether to use the tool results or try alternative approaches

Effective tool selection balances accuracy (choosing the right tool) with efficiency (minimizing unnecessary tool usage).

​
The Tool Selection Challenge

Tool selection presents several key challenges:

Ambiguous Requests

User requests may not clearly indicate which tool is needed

Multiple Valid Options

Several tools might satisfy the request in different ways

Parameter Extraction

Identifying and formatting the required parameters from natural language

Decision Complexity

Balancing multiple factors including capabilities, costs, and permissions

​
Tool Selection Strategies

Prisme.ai supports several approaches to tool selection, each with different strengths:

Leverage the language model’s reasoning to select appropriate tools.

How it works:

  • Tools are described to the LLM with clear purposes and capabilities
  • The LLM analyzes user requests and determines tool requirements
  • Function calling capabilities enable structured tool invocation
  • The LLM formats parameters based on tool schemas

Best for:

  • General-purpose agents with diverse tools
  • Complex decision-making with nuanced criteria
  • Adapting to varied user request formulations
  • Scenarios where selection logic is difficult to formalize

​
Implementing LLM-Based Selection

For many applications, LLM-based selection provides the best balance of flexibility and effectiveness:

1

Craft Clear Tool Descriptions

Provide the LLM with detailed information about each tool.

Effective description elements:

name: CustomerLookup
description: |
  Use this tool to find customer information in the CRM system.
  This tool should be used when the user asks about specific customer details,
  account status, or purchase history.
  Do NOT use this tool for general product inquiries or non-customer-specific questions.
  The tool requires either a customer ID or an email address to perform the lookup.

Key components include:

  • Clear purpose statement
  • When to use the tool
  • When NOT to use the tool
  • Required parameters
  • Expected outcomes
2

Provide Selection Guidelines

Include explicit guidance for tool selection in your agent instructions.

Example guidelines:

When choosing between available tools:

1. Always use the most specific tool for the task
2. Prefer internal data sources over external ones when both contain the information
3. Use the ProductSearch tool for general product inquiries and ProductDetails for specific product information
4. Only use the CustomerUpdate tool when explicitly asked to change customer information
5. If multiple tools could work, explain your selection reasoning to the user

These guidelines help the LLM make consistent decisions aligned with business preferences.

3

Implement Parameter Extraction

Guide the LLM in extracting and formatting parameters from user requests.

Example guidance:

When extracting parameters for tools:

1. For customer lookups, identify any customer ID (format: CUS-XXXXX) or email address
2. For product searches, extract product type, category, price range, and features
3. For date ranges, convert relative dates (e.g., "last week") to specific date formats
4. Always confirm ambiguous parameters with the user before tool execution
5. Format numeric parameters according to each tool's requirements

Effective parameter extraction ensures tools receive correctly formatted inputs.

4

Design Error Recovery

Prepare for tool selection errors with recovery strategies.

Example recovery instructions:

If a tool returns an error or no results:

1. First, check if parameters were correctly formatted and try again with adjustments
2. If parameters are correct but no results found, try an alternative tool if available
3. For customer lookups that fail, try searching by alternate identifiers
4. If all tool attempts fail, explain to the user what was tried and ask for additional information
5. Never make up information if tools cannot retrieve it

These strategies ensure resilience when initial tool selections don’t succeed.

​
Example: LLM-Based Tool Selection

Here’s a complete example of custom implementation of LLM-based tool selection:

slug: llm-tool-selection
name: "LLM-Based Tool Selection"
description: "Uses the LLM to select appropriate tools based on user input"
do:
  - AgentAnalyzer.chat-completion:
      prompt: |
        Select the most appropriate tool for this user request:
        User request: "{{input.text}}"

        Available tools:
        1. WeatherTool - Gets current weather information for a location
        2. CalendarTool - Manages calendar events and appointments
        3. StockTool - Retrieves current stock prices and market information
        
        Respond with a JSON object containing:
        - selectedTool: name of the selected tool
        - reasoning: why this tool was selected
        - parameters: parameters to pass to the tool
      output: toolSelection
  
  - conditions:
      '{{toolSelection.selectedTool}} == "WeatherTool"':
        - WeatherTool.execute:
            location: "{{toolSelection.parameters.location}}"
            output: result
      '{{toolSelection.selectedTool}} == "CalendarTool"':
        - CalendarTool.execute:
            action: "{{toolSelection.parameters.action}}"
            date: "{{toolSelection.parameters.date}}"
            output: result
      '{{toolSelection.selectedTool}} == "StockTool"':
        - StockTool.execute:
            symbol: "{{toolSelection.parameters.symbol}}"
            output: result
      default:
        - set:
            name: result
            value:
              error: "No appropriate tool selected"

output: "{{result}}"
when:
  events:
    - user-request

​
Implementing Rule-Based Selection

For more controlled environments, rule-based selection provides predictability and reliability:

​
Example: Hybrid Selection Approach

Most enterprise applications benefit from hybrid approaches that combine LLM flexibility with rule-based governance:

slug: hybrid-tool-selection
name: "Hybrid Tool Selection"
description: "Combines rule-based filtering with LLM ranking for optimal tool selection"
do:
  - EntityExtractor.extract:
      text: "{{input.text}}"
      output: entities
  - set:
      name: eligibleTools
      value: []
  
  - conditions:
      '{{input.text}} matches "weather" && {{entities.locations.length}} > 0':
        - set:
            name: eligibleTools[]
            value:
              name: "WeatherTool"
              parameters:
                location: "{{entities.locations[0]}}"
  
  - conditions:
      '{{input.text}} matches "schedule" && {{entities.dates.length}} > 0':
        - set:
            name: eligibleTools[]
            value:
              name: "CalendarTool"
              parameters:
                action: "{{entities.action}}"
                date: "{{entities.dates[0]}}"
  
  - conditions:
      '{{input.text}} matches "stock" && {{entities.companies.length}} > 0':
        - set:
            name: eligibleTools[]
            value:
              name: "StockTool"
              parameters:
                symbol: "{{entities.companies[0].symbol}}"
  
  - conditions:
      '{{eligibleTools.length}} > 1':
        - Knowledge Client.chat-completion:
            prompt: |
              Rank these tools for the user request: "{{input.text}}"
              
              Available tools:
              {{eligibleTools}}
              - {{tool.name}}              
                Respond with a JSON array of tool names, ranked from most to least appropriate.
            output: rankedTools
        - set:
            name: selectedTool
            value: "{{rankedTools[0]}}"
      '{{eligibleTools.length}} == 1':
        - set:
            name: selectedTool
            value: "{{eligibleTools[0].name}}"
        - set:
            name: toolParameters
            value: "{{eligibleTools[0].parameters}}"
      default:
        - Knowledge Client.chat-completion:
            prompt: |
              Select the most appropriate tool for this user request:
              User request: "{{input.text}}"
              
              Available tools:
              1. WeatherTool - Gets current weather information for a location
              2. CalendarTool - Manages calendar events and appointments
              3. StockTool - Retrieves current stock prices and market information
              
              Respond with a JSON object containing:
              - selectedTool: name of the selected tool
              - parameters: parameters to pass to the tool
            output: llmSelection
        - set:
            name: selectedTool
            value: "{{llmSelection.selectedTool}}"
        - set:
            name: toolParameters
            value: "{{llmSelection.parameters}}"
 
  - conditions:
      '{{selectedTool}} == "WeatherTool"':
        - WeatherTool.execute:
            location: "{{toolParameters.location}}"
            output: result
      '{{selectedTool}} == "CalendarTool"':
        - CalendarTool.execute:
            action: "{{toolParameters.action}}"
            date: "{{toolParameters.date}}"
            output: result
      '{{selectedTool}} == "StockTool"':
        - StockTool.execute:
            symbol: "{{toolParameters.symbol}}"
            output: result
      default:
        - set:
            name: result
            value:
              message: "I couldn't determine which tool would help with your request."

output: "{{result}}"
when:
  events:
    - user-request

​
Example: Tool Definition

Here’s an example of how to define a weather tool in AI Builder:

slug: get-weather
name: "Get Weather Information"
description: "Returns current weather information for a specified location"
do:
  - fetch:
      url: "https://api.weatherservice.com/v1/current"
      method: GET
      headers:
        Authorization: "Bearer {{secret.weatherApiKey}}"
      query:
        location: "{{location}}"
      output: weatherData
  - set:
      name: output
      value:
        temperature: "{{weatherData.current.temperature}}"
        condition: "{{weatherData.current.condition.text}}"
        humidity: "{{weatherData.current.humidity}}"
        location: "{{weatherData.location.name}}, {{weatherData.location.country}}"
        updated: "{{weatherData.current.last_updated}}"

validateArguments: true
arguments:
  location:
    type: string
    description: "The city or location to get weather for"
  format:
    type: string
    enum: 
     - detailed
     - simple
    default: simple
    description: "Level of detail to return"
output: "{{output}}"
when:
  endpoint: true
labels:
  - weather
  - tools

​
Best Practices for Tool Selection

​
Tool Selection in AI Knowledge

For AI Knowledge agents, implement effective tool selection through these configurations:

1

Configure Tool Descriptions

Create detailed, LLM-friendly descriptions for each tool.

Effective descriptions include:

  • Clear purpose and capabilities
  • When to use and when not to use
  • Example scenarios
  • Required parameter information
  • Expected result formats
2

Set Tool Selection Instructions

Add specific guidance for tool selection in agent instructions.

Example instructions:

Available Tools:

1. Web Browsing: Use when the user needs current information not in our knowledge base. Only use for factual information that would be publicly available.

2. Customer Database: Use when the user asks about specific customer records. Requires a customer ID or email address. Never use for general inquiries.

3. Product Catalog: Use when the user needs detailed product specifications or inventory information. Prefer this over Web Browsing for any product-related queries.

When choosing tools:
- First check if the information is available in our knowledge base
- Only use Web Browsing for current events or information that changes frequently
- Always use the most specific tool for the task
- If multiple tools could work, explain your choice to the user
3

Add Usage Examples

Provide concrete examples of correct tool selection.

Example:

Example 1:
User: "What's the current price of the XPS 15 laptop?"
Appropriate tool: Product Catalog
Reasoning: This is a specific product query about current pricing, which is exactly what the Product Catalog tool is designed for.

Example 2:
User: "Did we release any new products this month?"
Appropriate tool: Knowledge Base first, then Product Catalog
Reasoning: Check our knowledge base first for recent product announcements. If not found or if the information might be outdated, use the Product Catalog to check for products with recent release dates.

Example 3:
User: "What were the key announcements at yesterday's tech conference?"
Appropriate tool: Web Browsing
Reasoning: This is about a recent external event, so the information is likely not in our knowledge base and requires current web information.
4

Test and Refine

Evaluate tool selection performance with real-world scenarios.

Testing approaches:

  • Create a test set of diverse queries
  • Evaluate tool selection accuracy
  • Identify patterns in incorrect selections
  • Refine descriptions and instructions
  • A/B test different instruction formats

​
Tool Selection in AI Builder

For advanced tool selection logic in AI Builder, implement these patterns:

​
Next Steps

Ready to implement effective tool selection for your agents? Continue your journey with these resources:

Memory Management

Learn how to maintain state across tool interactions

Execution & Activity

Explore monitoring and debugging tool execution

Error Handling

Implement robust error management for tools

Multi-Agent Systems

Learn about systems with specialized agent collaboration

Was this page helpful?

Raise issue