RAG with Built-in Tools

RAG (Retrieval Augmented Generation) agents with built-in tools represent the most powerful no-code agents available on the Prisme.ai platform. These agents combine access to your organization’s knowledge with specialized capabilities that extend beyond conversation, enabling them to perform complex tasks, access external information, generate media, and process data.

Understanding RAG + Tools Agents

These advanced agents integrate three key capabilities:

Foundation Models

Powerful language models providing reasoning and generation capabilities

Knowledge Retrieval

Access to your organization’s documents and information

Specialized Tools

Built-in capabilities that extend functionality beyond conversation

This combination creates versatile, capable agents that can:

Answer questions using your internal knowledge
Retrieve up-to-date information from the web
Generate visual content based on requirements
Perform data analysis and calculations
Create structured documents and outputs
Execute multi-step processes with diverse requirements

Available Built-in Tools

Prisme.ai provides several powerful tools that can be activated without writing code:

Web Browsing
Image Generation
Code Interpreter

Description: Access current information from the internet using Serper integrationKey Capabilities:

Search for current information beyond the model’s knowledge cutoff
Access specific websites for reference
Verify facts and retrieve up-to-date data
Gather content from public sources

Best For:

Research and information gathering
Validating time-sensitive information
Supplementing internal knowledge with public data
Competitive research and market intelligence

Configuration Options:

Search domains to include/exclude
Number of results to retrieve
Safe search settings
Result depth control

When to Use RAG + Tools Agents

These advanced agents are ideal for scenarios that require a combination of:

Knowledge Access

Situations requiring specific organizational information

Current Information

Needs for up-to-date data beyond internal documentation

Visual Content

Requirements for custom images or visualizations

Data Processing

Calculations, analysis, or data transformation tasks

Multi-capability Workflows

Processes that combine different types of tasks

Complex Problem Solving

Scenarios requiring multiple approaches and resources

Example Use Cases

Research Assistant
Technical Support
Content Creator
Financial Analyst

Purpose: Help users research topics by combining internal knowledge with current information and analysis.Tool Utilization:

Retrieves relevant internal research and documentation
Searches the web for current information and trends
Generates data visualizations to compare findings
Creates structured research summaries

Example Interaction:

User: I need to understand the competitive landscape for cloud storage solutions, focusing on security features.

Agent: I'll help you research this topic. Let me check our internal knowledge first, then supplement with current market information.

[Accesses knowledge base for internal research on cloud storage security]

Based on our internal analysis from Q1 this year, the key security features in cloud storage include:
- End-to-end encryption
- Zero-knowledge architecture
- Multi-factor authentication
- SOC 2 compliance
- Data residency options

Let me check the current market landscape to see if there have been any recent developments.

[Uses web browsing to search for current information]

I've found some updated information. In the last three months, several major providers have introduced:
- Quantum-resistant encryption options
- Biometric authentication integration
- Enhanced ransomware protection features

Would you like me to create a visualization comparing the top competitors based on these security features?

User: Yes, please create a comparison chart.

[Uses code interpreter to generate a visual comparison]

Here's a visual comparison of the top cloud storage providers and their security features:
[Image: Comparative chart of cloud storage security features]

Would you like me to focus on any specific aspect of this competitive landscape?

Purpose: Provide advanced technical assistance by combining product documentation with analytical capabilities.Tool Utilization:

Retrieves relevant product documentation and troubleshooting guides
Generates diagrams to explain technical concepts
Runs code to demonstrate solutions or analyze logs
Searches for updates or known issues

Example Interaction:

User: I'm getting a 503 error when trying to access the admin dashboard, but only when using Chrome. Firefox works fine.

Agent: I'll help you troubleshoot this issue. Let me check our knowledge base for relevant information.

[Accesses knowledge base for information on 503 errors and browser compatibility]

According to our troubleshooting guide, a 503 error specific to one browser could indicate:
1. Browser cache issues
2. Extension conflicts
3. Security settings incompatibility

Let me check if there are any known issues with Chrome specifically.

[Uses web browsing to search for recent Chrome issues]

I've found that Chrome released version 115.0.5790.110 three days ago, which has some known issues with certain web applications. Let me create a diagram to explain what might be happening.

[Uses image generation to create a technical diagram]

Here's a visualization of how the Chrome request might be failing:
[Image: Diagram showing the request flow and error point]

I recommend trying these steps:
1. Clear your Chrome cache
2. Temporarily disable extensions
3. Try using Chrome in incognito mode

If those don't work, I can help you with a more advanced troubleshooting process. Would you like to try these steps first?

Purpose: Assist in creating rich content by combining knowledge, research, and media generation.Tool Utilization:

Retrieves brand guidelines and existing content
Searches for current trends and information
Generates custom images and visualizations
Analyzes content performance data

Example Interaction:

User: I need to create a blog post about sustainable manufacturing for our company website.

Agent: I'll help you create a comprehensive blog post on sustainable manufacturing. Let me start by checking our brand guidelines and existing content on this topic.

[Accesses knowledge base for brand guidelines and related content]

Based on our brand guidelines, sustainability content should:
- Focus on measurable impact
- Include specific examples from our operations
- Use a professional but conversational tone
- Incorporate our commitment to the UN Sustainable Development Goals

Let me check for current trends in sustainable manufacturing.

[Uses web browsing to research current sustainability trends]

Recent trends in sustainable manufacturing include:
- Circular economy implementation
- Carbon-neutral production targets
- AI-optimized resource utilization
- Blockchain for supply chain transparency

I'd like to suggest a structure for your blog post with these elements. I can also generate some custom images to illustrate key concepts.

[Generates outline and sample images]

Here's a suggested outline for your blog post:
[Structured blog post outline]

And here's a custom image illustrating the circular economy principle in manufacturing:
[Image: Custom illustration of circular economy in manufacturing]

Would you like me to expand any section of this outline, or would you prefer different imagery?

Purpose: Help analyze financial data by combining documentation, calculations, and visualizations.Tool Utilization:

Retrieves financial policies and procedures
Runs calculations on financial data
Creates visual representations of financial trends
Searches for market benchmarks and comparisons

Example Interaction:

User: Can you help me analyze our Q2 revenue against projections? I've uploaded the spreadsheet.

Agent: I'll help you analyze the Q2 revenue performance against projections using the spreadsheet you've provided. Let me first check our financial analysis procedures.

[Accesses knowledge base for financial analysis guidelines]

According to our financial analysis framework, we should examine:
- Variance analysis by product line
- Key performance indicators against targets
- Seasonal adjustment factors
- Year-over-year comparison

Let me analyze the spreadsheet you've uploaded.

[Uses code interpreter to perform financial analysis]

I've analyzed the data and found several key insights:

1. Overall revenue is 3.2% below projection, primarily due to underperformance in the Enterprise product line
2. Consumer segment exceeded projections by 7.8%
3. The variance expanded from 1.1% in April to 5.4% in June, indicating a growing trend

Let me create some visualizations to help understand these patterns.

[Generates financial charts and graphs]

Here's a visualization of the quarterly performance:
[Image: Revenue vs. Projection chart]

And here's a breakdown by product line:
[Image: Product line performance comparison]

Would you like me to perform any additional analysis, such as identifying the root causes of the Enterprise segment underperformance?

Creating RAG + Tools Agents

Building these advanced agents involves a structured process:

Define Agent Purpose and Requirements

Clearly specify what your agent needs to accomplish.Key considerations:

Primary use cases and scenarios
Required knowledge sources
Necessary tools and capabilities
User expectations and needs
Success criteria and metrics

Prepare Knowledge Resources

Assemble and organize the information your agent needs.Activities include:

Document collection and curation
Content organization and structuring
Metadata creation for improved retrieval
Validation of information accuracy
Knowledge base configuration

Select and Configure Tools

Determine which built-in tools your agent requires.For each tool:

Enable required capabilities
Configure settings and parameters
Define usage boundaries and limitations
Set up authentication if required
Test individual tool functionality

Create System Instructions

Develop comprehensive guidance for your agent.Key components:

Agent role and purpose definition
Knowledge utilization guidelines
Tool selection and usage criteria
Response formatting requirements
Error handling and limitations

Develop Tool Usage Instructions

Provide specific guidance for when and how to use each tool.Include for each tool:

When to use (clear criteria)
How to use effectively
Result interpretation guidance
Error handling approaches
User communication during tool use

Test and Refine

Validate and iteratively improve your agent.Testing approach:

Scenario-based evaluation
Tool usage verification
Knowledge retrieval assessment
Edge case testing
Real user feedback collection

Implementation in Knowledges

Creating a RAG + Tools agent in Prisme.ai’s Knowledges is straightforward:

1. Create New Agent
2. Connect Knowledge Base
3. Enable Built-in Tools
4. Create System Instructions
5. Test and Deploy

Start by creating a new agent in the Agent Creator.Steps:

Navigate to Knowledges
Click “Create New Agent”
Provide name and basic description

Tool-Specific Instructions

To maximize the effectiveness of each built-in tool, include specific guidance in your system instructions:

Web Browsing Tool Instructions

When to Use:

Use the web browsing tool when:
1. The user asks for current information beyond our internal documentation
2. You need to verify or update potentially outdated internal information
3. The user explicitly requests external research
4. You need specific details not available in our knowledge base

Always tell the user when you're using web browsing and cite the sources of information retrieved.

How to Use Effectively:

When using the web browsing tool:
Form specific, focused search queries
Prioritize authoritative sources
Cross-reference information from multiple sources when possible
Summarize key findings rather than providing entire web pages
Extract only relevant information to the user's query
Always cite sources with title and URL

Example Prompt Section:

For web browsing:
- Formulate clear, specific search queries focused on the user's information need
- Prioritize results from authoritative sources (.gov, .edu, established industry sites)
- Synthesize information from multiple sources rather than relying on a single result
- Always cite your sources by including "According to [source name] ([URL])"
- When information conflicts between sources, acknowledge this and explain which source seems more reliable and why
- If search results are insufficient, try reformulating your query or tell the user what information you were unable to find

Image Generation Tool Instructions

When to Use:

Use the image generation tool when:
1. The user explicitly requests a visual representation
2. A complex concept would benefit from visual explanation
3. Creating diagrams, charts, or illustrations would enhance understanding
4. Visual examples would supplement textual information

Always confirm with the user before generating images unless explicitly requested.

How to Use Effectively:

When generating images:
Create detailed, specific prompts for the image generator
Align visual style with our brand guidelines (professional, clean, modern)
Ensure images are accessible and clear in their meaning
Generate appropriate alternatives if the user is not satisfied
Focus on informative rather than decorative images

Example Prompt Section:

For image generation:
- Create detailed, specific prompts that describe both content and style
- Follow our brand style guidelines: clean, professional, with our color palette (blue #1A73E8, gray #5F6368, green #34A853)
- For diagrams and charts, prioritize clarity and readability over aesthetic complexity
- Ensure generated images are appropriate and professional for business use
- Always ask the user if they'd like modifications to the generated image
- If the user requests a specific style, artistic approach, or reference, incorporate these elements explicitly in your prompt

Code Interpreter Tool Instructions

When to Use:

Use the code interpreter tool when:
Performing calculations or data analysis is required
The user has provided data that needs processing
Creating data visualizations would be valuable
Converting between data formats is needed
The user explicitly requests analysis or computation

Explain your analytical approach before executing code.

How to Use Effectively:

When using code interpreter:
Write clean, well-commented code for transparency
Explain your analysis approach in non-technical terms
Present results with clear interpretations
Create visualizations that highlight key insights
Handle errors gracefully with meaningful explanations

Example Prompt Section:

For code interpretation and data analysis:
- Write clean, well-documented Python code with comments explaining key steps
- Before executing code, explain your analytical approach in non-technical terms
- When analyzing data, first examine its structure and quality before drawing conclusions
- Create informative visualizations with clear titles, labeled axes, and appropriate chart types
- Always interpret results in business terms, focusing on insights rather than just numbers
- When errors occur, provide clear explanations and alternative approaches
- For data visualizations, use our brand colors (#1A73E8 blue, #34A853 green, #FBBC04 yellow, #EA4335 red)

Best Practices for RAG + Tools Agents

Clear Tool Selection Logic

Provide explicit criteria for when to use each toolExample:

Use web browsing when:
- Information needed is likely to change frequently
- The query explicitly mentions current events
- Internal knowledge is insufficient or outdated

Use code interpreter when:
- Calculations are required
- Data analysis is needed
- Visualization would enhance understanding

User Communication During Tool Use

Keep users informed about tool usage and processesExample:

Before using a tool:
"I'll need to search the web to find the most current information on this topic."

During tool use:
"I'm analyzing the data you provided. This will take a moment."

After tool use:
"Based on the web search results from [sources], I found that..."

Integrate Knowledge and Tool Outputs

Combine information from different sources coherentlyExample:

First present information from internal knowledge:
"According to our documentation, our current process is..."

Then supplement with web information:
"Based on recent industry updates from [source], there are new approaches..."

Finally, provide analysis:
"Analyzing these approaches against your requirements shows that..."

Handle Tool Failures Gracefully

Provide alternative approaches when tools don’t work as expectedExample:

"I tried to search for current information on this topic, but couldn't find specific results. Instead, let me:
Share what we know from our internal documentation
Suggest some alternative approaches
Recommend reliable sources you might check directly"

Troubleshooting Common Issues

Knowledge Retrieval Problems

Symptoms:

Agent fails to find relevant information
Responses ignore available knowledge
Retrieved information is incorrect or outdated

Solutions:

Verify knowledge base configuration and content
Improve document chunking and organization
Enhance retrieval instructions in system prompt
Add more specific examples of knowledge utilization
Test with focused queries to validate retrieval

Inappropriate Tool Selection

Symptoms:

Agent uses tools when unnecessary
Agent fails to use tools when needed
Consistent selection of suboptimal tools

Solutions:

Refine tool selection criteria in system instructions
Provide explicit examples of when to use each tool
Add clear guidance for choosing between similar tools
Include decision tree for tool selection logic
Test with scenarios that require tool selection decisions

Poor Tool Usage

Symptoms:

Ineffective web searches returning irrelevant results
Low-quality image generation
Code that errors or produces incorrect results

Solutions:

Improve tool-specific instructions and examples
Add more detailed guidance on effective tool usage
Provide templates for common tool operations
Include error handling instructions for each tool
Test each tool independently with focused scenarios

Response Integration Issues

Symptoms:

Disjointed responses that fail to synthesize information
Tool outputs presented without context or interpretation
Failure to connect retrieved knowledge with tool results

Solutions:

Enhance instructions for information synthesis
Add examples of well-integrated responses
Include specific guidance on combining knowledge and tool outputs
Provide templates for presenting multi-source information
Test with scenarios requiring information integration

Performance Issues

Symptoms:

Slow response times
Tool execution timeouts
Memory errors during complex operations

Solutions:

Optimize tool usage guidelines for efficiency
Add instructions for breaking complex tasks into manageable steps
Provide fallback options for resource-intensive operations
Include guidelines for managing large datasets
Test with progressively complex scenarios to identify limits

Advanced Configuration Techniques

For organizations seeking to maximize the effectiveness of their RAG + Tools agents, consider these advanced approaches:

Tool Chaining

Configure agents to use tools in sequence for complex workflowsImplementation:

For market analysis requests:
First use knowledge retrieval to access internal market research
Then use web browsing to gather current market trends
Next use code interpreter to analyze combined data
Finally use image generation to create visualizations of findings

Tool chaining enables sophisticated multi-step processes that combine different capabilities.

Contextual Tool Configuration

Adapt tool behavior based on user type or request contextImplementation:

When supporting technical users:
- Include more technical detail in responses
- Show code snippets and raw data when relevant
- Use technical terminology in explanations

When supporting business users:
- Focus on insights rather than technical details
- Emphasize visualizations over raw data
- Use business terminology in explanations

Contextual configuration improves relevance for different user groups.

Hybrid Knowledge Approach

Strategically combine internal knowledge with web informationImplementation:

For product information:
- Always use internal knowledge for specifications and features
- Use web browsing only for competitor comparisons
- Supplement with internal competitive analysis when available
- Clearly distinguish between internal and external information

This approach ensures accurate internal information while benefiting from external context.

Progressive Disclosure

Structure responses to present essential information firstImplementation:

When providing analysis results:
Start with key findings and recommendations
Follow with supporting evidence and context
Offer detailed analysis for those who want to explore further
Provide visualization options that users can request

Progressive disclosure improves usability by preventing information overload.

Enterprise Integration Considerations

Governance and Compliance

When deploying RAG + Tools agents in enterprise environments, consider:

Usage Policies: Define clear guidelines for appropriate tool usage
Content Review: Establish processes for reviewing agent outputs
Audit Trails: Implement logging for tool usage and information sources
Access Controls: Limit specific tools to appropriate user groups
Compliance Checks: Ensure tool outputs meet regulatory requirements

Implementation Example:

This agent's web browsing capability is restricted to approved domains and automatically logs all external information sources. Image generation follows brand guidelines and content policies. All data processing complies with our data handling procedures and privacy requirements.

Security Considerations

Protect sensitive information when using tools:

Data Handling: Establish clear rules for handling sensitive information
Tool Boundaries: Define what data can be processed by each tool
Output Scrubbing: Implement checks for sensitive information in outputs
Authentication: Manage service credentials securely
User Permissions: Align tool access with user authorization levels

Implementation Example:

Code interpreter will not process files containing PII or confidential information. Web browsing is restricted to public information only. All generated content is screened for sensitive information before delivery to users.

Scalability Planning

Prepare for widespread adoption and usage:

Resource Management: Monitor and optimize tool usage patterns
Usage Quotas: Implement appropriate limits for resource-intensive tools
Caching Strategies: Cache common tool results when appropriate
Load Balancing: Distribute tool processing across resources
Performance Monitoring: Track response times and resource utilization

Implementation Example:

This agent implements smart caching for web search results and common calculations. Image generation is limited to 10 images per user per day. Resource-intensive code execution has a 60-second timeout with graceful degradation to simpler analysis for complex requests.

Next Steps

Ready to create your own RAG + Tools agent? Continue with these resources:

Agent Creator Guide

Learn how to create agents effectively

Testing Best Practices

Learn how to thoroughly test and validate your agents

Documentation Index

​Understanding RAG + Tools Agents

Foundation Models

Knowledge Retrieval

Specialized Tools

​Available Built-in Tools

​When to Use RAG + Tools Agents

Knowledge Access

Current Information

Visual Content

Data Processing

Multi-capability Workflows

Complex Problem Solving

​Example Use Cases

​Creating RAG + Tools Agents

​Implementation in Knowledges

​Tool-Specific Instructions

​Best Practices for RAG + Tools Agents

Clear Tool Selection Logic

User Communication During Tool Use

Integrate Knowledge and Tool Outputs

Handle Tool Failures Gracefully

​Troubleshooting Common Issues

​Advanced Configuration Techniques

Tool Chaining

Contextual Tool Configuration

Hybrid Knowledge Approach

Progressive Disclosure

​Enterprise Integration Considerations

​Next Steps

Agent Creator Guide

Testing Best Practices

Understanding RAG + Tools Agents

Available Built-in Tools

When to Use RAG + Tools Agents

Example Use Cases

Creating RAG + Tools Agents

Implementation in Knowledges

Tool-Specific Instructions

Best Practices for RAG + Tools Agents

Troubleshooting Common Issues

Advanced Configuration Techniques

Enterprise Integration Considerations

Next Steps