Learn how to create a powerful retrieval-augmented generation agent without writing code using AI Knowledge
This tutorial guides you through creating a powerful Retrieval-Augmented Generation (RAG) agent using Prisme.ai’s AI Knowledge product. You’ll learn how to build an AI that can intelligently access and leverage your organization’s documents to provide accurate, contextual responses—all without writing a single line of code.
A complete RAG system with:
AI Knowledge functions as a RAG-as-a-Service offering, creating a collaborative environment where Data Scientists, Data Engineers, Developers, and Business Teams can efficiently work together. The platform features a unique methodology called Gen.AI Test-Driven Building, ensuring everyone operates from a single source of truth.
Before starting this tutorial, make sure you have:
Let’s familiarize ourselves with the AI Knowledge product’s interface and main components:
Access AI Knowledge
Log in to your Prisme.ai account and navigate to the AI Knowledge product
Explore the Navigation
The interface consists of several key sections:
Now, let’s set up your first RAG project in AI Knowledge:
Access Your Dashboard
Navigate to the AI Knowledge dashboard
Create a New Agent
Click the “Create Agent” button to start building your RAG agent
Configure Project Settings
Enter a name and description for your project that reflects its purpose
AI Knowledge uses a Retriever-Augmented Generation architecture that combines document retrieval with language generation:
This architecture allows your agent to:
AI Knowledge is designed to be model-agnostic, ensuring compatibility across various LLMs for both embedding and generative tasks. This flexibility allows you to switch models as needed without rebuilding your entire knowledge base.
Let’s add documents to your knowledge base:
Access Document Integration
In your project, navigate to the document integration section
Upload Documents
Click the “Add Document” button and upload files from your computer, or provide URLs to web content
Configure Document Processing
Adjust the document processing settings:
Process Documents
Start the document processing, which includes:
Now that your documents are integrated, you can start querying your knowledge base:
Access the Chat Interface
Click on the “Chat” button to access the interactive interface
Ask Questions
Enter questions related to your documents to see how the AI responds
Review Sources and Chunks
Examine the sources and chunks used to generate each response
AI Knowledge provides full transparency by showing which document chunks were used to generate each response. This feature allows you to verify the AI’s reasoning and ensure accuracy.
A unique aspect of AI Knowledge is its support for Test-Driven Building (TDB), which ensures your RAG agent performs consistently over time:
Access the Testing Module
Navigate to the “Test” section in your project
Create a Test Suite
Click “Add Test” to create a new test suite for your RAG agent
Define Test Cases
Add test cases with:
Configure Test Settings
Set how often tests should run (daily, weekly, etc.) and whether they should trigger automatically
Run Tests
Execute the tests to evaluate your agent’s performance
Review Results
Analyze the test results, which include:
Business experts can manually assess responses, context, and instances of hallucination. We recommend running tests after each modification to the model, prompt, AI parameters, chunk size, image inclusion settings, self-query configuration, and other adjustments.
After setting up your basic RAG agent, you can explore advanced customization options:
Access the Builder
Navigate to the Builder product on the Prisme.ai Platform
Implement Custom Logic
Create automations with Python or NodeJS using frameworks like LlamaIndex or Langchain
Connect to AI Knowledge
Copy the automation URL and go to Advanced > API & Webhook in AI Knowledge
Enable Webhook
Enter your automation URL and subscribe to events like questions asked, documents added/deleted/updated, and test runs
Access the Builder
Navigate to the Builder product on the Prisme.ai Platform
Implement Custom Logic
Create automations with Python or NodeJS using frameworks like LlamaIndex or Langchain
Connect to AI Knowledge
Copy the automation URL and go to Advanced > API & Webhook in AI Knowledge
Enable Webhook
Enter your automation URL and subscribe to events like questions asked, documents added/deleted/updated, and test runs
Develop External Service
Build your custom processing logic as an external service with an API endpoint
Configure Webhook
Navigate to Advanced > API & Webhook in AI Knowledge
Connect Your Service
Enter your service’s URL to receive events from AI Knowledge
Set Up Event Handlers
Configure your service to respond appropriately to different event types
For organizations dealing with sensitive information, AI Knowledge offers options for privacy and safety:
Implement PII Controls
Use the Webhook functionality to detect and handle personally identifiable information
Configure Data Anonymization
Decide whether to anonymize personal data based on your use case
Add Custom Safety Controls
Integrate specific safety measures tailored to your organization’s compliance requirements
Keep your RAG agent performing optimally with regular monitoring:
Review Analytics
Check the Analytics section to monitor usage patterns, token consumption, and response quality
Set Up Alerts
Use the Webhook feature with Builder to implement alerts on platforms like Slack, Teams, or Jira
Regular Maintenance
Update your knowledge base as new documents become available or existing information changes
Performance Optimization
Based on analytics and test results, adjust settings to improve response quality and efficiency
To maximize the effectiveness of your RAG agent, consider these recommendations:
Design a well-organized structure for easy access and interpretation by AI systems
Ensure data is accurate, well-formatted, and relevant—this accounts for ~80% of success
Consider combining NLU for tag extraction with LLMs for generation to improve efficiency
Regularly monitor response times, accuracy, and retrieval effectiveness
By following this tutorial, you’ve created a powerful RAG agent that can access and leverage your organization’s knowledge without requiring any coding. Your AI can now provide accurate, contextual responses based on your documents, with full transparency about its sources and reasoning.
The Test-Driven Building approach ensures your agent maintains high performance over time, while the flexible architecture allows you to adapt and extend its capabilities as your needs evolve.
Learn how to automatically classify and organize your documents with AI
Create an intelligent contact form that routes inquiries to the right department
Build advanced integrations between your RAG agent and other systems
Turn website content into a comprehensive knowledge base for your RAG agent
Learn how to create a powerful retrieval-augmented generation agent without writing code using AI Knowledge
This tutorial guides you through creating a powerful Retrieval-Augmented Generation (RAG) agent using Prisme.ai’s AI Knowledge product. You’ll learn how to build an AI that can intelligently access and leverage your organization’s documents to provide accurate, contextual responses—all without writing a single line of code.
A complete RAG system with:
AI Knowledge functions as a RAG-as-a-Service offering, creating a collaborative environment where Data Scientists, Data Engineers, Developers, and Business Teams can efficiently work together. The platform features a unique methodology called Gen.AI Test-Driven Building, ensuring everyone operates from a single source of truth.
Before starting this tutorial, make sure you have:
Let’s familiarize ourselves with the AI Knowledge product’s interface and main components:
Access AI Knowledge
Log in to your Prisme.ai account and navigate to the AI Knowledge product
Explore the Navigation
The interface consists of several key sections:
Now, let’s set up your first RAG project in AI Knowledge:
Access Your Dashboard
Navigate to the AI Knowledge dashboard
Create a New Agent
Click the “Create Agent” button to start building your RAG agent
Configure Project Settings
Enter a name and description for your project that reflects its purpose
AI Knowledge uses a Retriever-Augmented Generation architecture that combines document retrieval with language generation:
This architecture allows your agent to:
AI Knowledge is designed to be model-agnostic, ensuring compatibility across various LLMs for both embedding and generative tasks. This flexibility allows you to switch models as needed without rebuilding your entire knowledge base.
Let’s add documents to your knowledge base:
Access Document Integration
In your project, navigate to the document integration section
Upload Documents
Click the “Add Document” button and upload files from your computer, or provide URLs to web content
Configure Document Processing
Adjust the document processing settings:
Process Documents
Start the document processing, which includes:
Now that your documents are integrated, you can start querying your knowledge base:
Access the Chat Interface
Click on the “Chat” button to access the interactive interface
Ask Questions
Enter questions related to your documents to see how the AI responds
Review Sources and Chunks
Examine the sources and chunks used to generate each response
AI Knowledge provides full transparency by showing which document chunks were used to generate each response. This feature allows you to verify the AI’s reasoning and ensure accuracy.
A unique aspect of AI Knowledge is its support for Test-Driven Building (TDB), which ensures your RAG agent performs consistently over time:
Access the Testing Module
Navigate to the “Test” section in your project
Create a Test Suite
Click “Add Test” to create a new test suite for your RAG agent
Define Test Cases
Add test cases with:
Configure Test Settings
Set how often tests should run (daily, weekly, etc.) and whether they should trigger automatically
Run Tests
Execute the tests to evaluate your agent’s performance
Review Results
Analyze the test results, which include:
Business experts can manually assess responses, context, and instances of hallucination. We recommend running tests after each modification to the model, prompt, AI parameters, chunk size, image inclusion settings, self-query configuration, and other adjustments.
After setting up your basic RAG agent, you can explore advanced customization options:
Access the Builder
Navigate to the Builder product on the Prisme.ai Platform
Implement Custom Logic
Create automations with Python or NodeJS using frameworks like LlamaIndex or Langchain
Connect to AI Knowledge
Copy the automation URL and go to Advanced > API & Webhook in AI Knowledge
Enable Webhook
Enter your automation URL and subscribe to events like questions asked, documents added/deleted/updated, and test runs
Access the Builder
Navigate to the Builder product on the Prisme.ai Platform
Implement Custom Logic
Create automations with Python or NodeJS using frameworks like LlamaIndex or Langchain
Connect to AI Knowledge
Copy the automation URL and go to Advanced > API & Webhook in AI Knowledge
Enable Webhook
Enter your automation URL and subscribe to events like questions asked, documents added/deleted/updated, and test runs
Develop External Service
Build your custom processing logic as an external service with an API endpoint
Configure Webhook
Navigate to Advanced > API & Webhook in AI Knowledge
Connect Your Service
Enter your service’s URL to receive events from AI Knowledge
Set Up Event Handlers
Configure your service to respond appropriately to different event types
For organizations dealing with sensitive information, AI Knowledge offers options for privacy and safety:
Implement PII Controls
Use the Webhook functionality to detect and handle personally identifiable information
Configure Data Anonymization
Decide whether to anonymize personal data based on your use case
Add Custom Safety Controls
Integrate specific safety measures tailored to your organization’s compliance requirements
Keep your RAG agent performing optimally with regular monitoring:
Review Analytics
Check the Analytics section to monitor usage patterns, token consumption, and response quality
Set Up Alerts
Use the Webhook feature with Builder to implement alerts on platforms like Slack, Teams, or Jira
Regular Maintenance
Update your knowledge base as new documents become available or existing information changes
Performance Optimization
Based on analytics and test results, adjust settings to improve response quality and efficiency
To maximize the effectiveness of your RAG agent, consider these recommendations:
Design a well-organized structure for easy access and interpretation by AI systems
Ensure data is accurate, well-formatted, and relevant—this accounts for ~80% of success
Consider combining NLU for tag extraction with LLMs for generation to improve efficiency
Regularly monitor response times, accuracy, and retrieval effectiveness
By following this tutorial, you’ve created a powerful RAG agent that can access and leverage your organization’s knowledge without requiring any coding. Your AI can now provide accurate, contextual responses based on your documents, with full transparency about its sources and reasoning.
The Test-Driven Building approach ensures your agent maintains high performance over time, while the flexible architecture allows you to adapt and extend its capabilities as your needs evolve.
Learn how to automatically classify and organize your documents with AI
Create an intelligent contact form that routes inquiries to the right department
Build advanced integrations between your RAG agent and other systems
Turn website content into a comprehensive knowledge base for your RAG agent