Skip to main content
This tutorial guides you through building a Retrieval-Augmented Generation (RAG) agent on Prisme.ai without writing any code. In v27 the work is split across two products: Knowledges hosts the knowledge base (document ingestion, embeddings, retrieval), and Agent Creator hosts the agent itself (instructions, model, playground, evaluations). You’ll wire them together so your agent answers questions grounded in your documents.

​
What You’ll Build

A complete RAG system with:
  • A Knowledge Base in Knowledges containing your documents
  • Vector embedding and retrieval handled by the Knowledge Base
  • An agent in Agent Creator that consumes that Knowledge Base for context
  • The Agent Creator Playground for testing conversations
  • Evaluations and Analytics in Agent Creator for ongoing quality control
This split lets Data Scientists, Data Engineers, Developers, and Business Teams collaborate from a single source of truth: knowledge stays curated in one place, while agents that depend on it are configured and tested independently.

​
Prerequisites

Before starting this tutorial, make sure you have:
  • Access to a Prisme.ai workspace with Knowledges and Agent Creator enabled
  • Basic understanding of prompting concepts
  • Documents to ingest (web pages, PDFs, Word documents, PowerPoint presentations, etc.)

​
Step 1: Understanding the Knowledges Interface

Let’s start by orienting ourselves inside the Knowledges product.
1

Open Knowledges

From AI Studio, open the Knowledges product (/apps/ai-knowledge).
2

Explore the navigation

The left sidebar gives you four entries:
  • Dashboard: overview with high-level stats: number of Knowledge Bases, Documents, and Connectors.
  • Knowledge Bases: the list of every knowledge base in your workspace, with quick access to each one.
  • Connectors: third-party SaaS integrations (SharePoint, Google Drive, Notion, Confluence) that sync content into Knowledge Bases. Note: web crawling and direct file uploads are not here; they live on each Knowledge Base’s detail page via Add Web Source and Add Documents.
  • Settings: workspace-level Knowledges settings.
3

Skim the dashboard

The dashboard tiles are the fastest way to gauge volume: how many knowledge bases exist, how many documents are indexed across all of them, and how many third-party connectors are wired up.
Knowledges dashboard with Dashboard/Knowledge Bases/Connectors/Settings sidebar and stats tiles

​
Step 2: Create a Knowledge Base

Now create the knowledge base that will hold your documents.
1

Open Knowledge Bases

In the Knowledges sidebar, click Knowledge Bases.
2

Start a new Knowledge Base

Click the + Create button in the top-right (the Dashboard’s + New Knowledge Base tile opens the same dialog).
3

Name and configure it

Give the Knowledge Base a clear name and description that reflect what it covers. Pick the embedding model you want to use for vectorization; the choice affects retrieval quality and cost.
4

Open the Knowledge Base

Once created, open it from the Knowledge Bases list to access its documents and settings.
Knowledges Create Knowledge Base dialog with Name, Description, RAG Configuration (Fast/Balanced/Quality)
Knowledges is model-agnostic for both embedding and generation, so you can swap models later without rebuilding the knowledge base from scratch.

​
Step 3: Add Documents

With the Knowledge Base open, ingest your content.
1

Open the documents tab

Inside the Knowledge Base, navigate to its documents tab.
2

Add documents

Click Add Documents and either upload files from your computer or paste URLs to web content. For full website ingestion, see the Add Web Source flow on the Knowledge Base detail page.
3

Configure document processing

Per Knowledge Base, you can adjust how content is chunked before embedding:
  • Text Splitter: static (fixed-size chunks) or dynamic (content-aware) segmentation
  • Chunk Size: the size of each segment
  • Overlap: how much content overlaps between adjacent chunks to preserve context
See RAG settings for the full reference.
4

Process documents

Start the processing job. Knowledges will load and parse each document, extract and clean the text, segment it into chunks, generate vector embeddings, and capture metadata.

​
Step 4: Create an Agent in Agent Creator and Connect the Knowledge Base

The knowledge base is ready. Now create the agent that will use it.
1

Open Agent Creator

From AI Studio, switch to the Agent Creator product (/apps/agent-factory).
2

Create a new agent

Click + Create agent in the top-right. Give the agent a name and description.
3

Set instructions and model

Write the agent’s system instructions (its role, tone, what it should and shouldn’t do) and pick the generation model.
4

Attach the Knowledge Base

In the agent’s Capabilities section, connect the Knowledge Base you created in Step 2. From now on, the agent will retrieve relevant chunks from that Knowledge Base each time a user asks a question.
Agent Creator Create Agent wizard with From Scratch / Import AGENTS.md options The screenshot shows the agent creation wizard. Once your agent exists, attach a Knowledge Base from its capabilities/configuration view.

​
Step 5: Chat With Your Agent in the Playground

Time to try it.
1

Open the Playground

On the agent’s detail page, open the Playground tab.
2

Ask questions

Enter questions related to your documents and watch the agent respond.
3

Inspect the retrieved sources

Each response surfaces the document chunks that were pulled from the Knowledge Base to ground the answer. Use this to verify the agent’s reasoning and catch retrieval issues early.
Source transparency is a first-class feature: every answer in the Playground (and in production) shows which chunks were used, so you can audit grounding without leaving the agent’s page.

​
Step 6: Evaluations (Test-Driven Building)

Evaluations live alongside the agent in Agent Creator, which keeps quality checks tied to the thing being tested.
1

Open the Evaluate tab

On the agent’s detail page, switch to the Evaluate tab.
2

Create an eval suite

Add a new evaluation suite for the agent.
3

Define test cases

Each test case pairs a question a user might ask with the expected answer the agent should produce.
4

Schedule runs

Configure how often the suite runs (daily, weekly, on demand) and whether it should fire automatically after agent changes.
5

Run the suite

Execute the suite to evaluate the agent’s current behavior.
6

Review results

Each result includes:
  • Prompt and Context: the exact prompt and retrieved context used
  • Generated Response: what the agent actually produced
  • Response Evaluation: rating of response quality (poor, correct, good)
  • Context Evaluation: rating of retrieved-context relevance and accuracy
Business experts can manually grade responses, context, and hallucinations. Re-run the suite after every meaningful change (instructions, model, retrieval parameters, chunking, or attached Knowledge Base) so regressions surface immediately.

​
Step 7: Advanced Customization

Once the basics work, you can extend the system with custom logic that reacts to Knowledge Base activity. The pattern is the same in both cases: a Builder automation (or an external service) listens for platform events emitted by the Knowledge Base and runs your custom logic in response.
1

Open Builder

Open the Builder product (/builder), Prisme.ai’s no-code workspace for automations.
2

Create an event-triggered automation

Create an automation whose when block subscribes to the Knowledge Base events you care about, for example document add/update/delete or query events. See the Automations reference for the trigger syntax.
3

Implement custom logic

In the automation’s do steps, run whatever processing you need: call out to a custom service, write to a Collection, post to Slack, or chain another agent. You can mix YAML instructions with Custom Code (Python or NodeJS) blocks.

​
Step 8: Handling PII and Safety Controls

For sensitive content, route Knowledge Base events through your own safety layer.
1

Implement PII controls

Subscribe a Builder automation to Knowledge Base events (document ingest, query) and run PII detection inside that automation before sensitive content is stored or returned.
2

Configure data anonymization

Decide whether to anonymize personal data based on your use case and compliance requirements.
3

Add custom safety controls

Plug in organization-specific safety measures (content filters, allow/deny lists, audit logging) wherever they belong in the event flow.

​
Step 9: Monitoring and Maintenance

Keep the agent healthy with the right monitoring surface for each concern.
1

Monitor agent conversations in Insights

For cross-agent analytics on real user conversations (adoption, topics, feedback, clusters), use the Insights product. See Insights overview.
2

Check per-agent usage in Agent Creator

Open the Analytics tab on the agent itself for usage metrics specific to that agent (volume, latency, token consumption, response quality).
3

Set up alerts

Use a Builder automation subscribed to Knowledge Base events to push alerts into Slack, Teams, or Jira when something goes wrong.
4

Maintain the Knowledge Base

Refresh documents as content changes, retire stale sources, and keep connectors synced.
5

Optimize

Use evaluations and analytics to drive iterative tuning: instructions, model, chunking, retrieval parameters.

​
Best Practices for High-Performance RAG

To maximize the effectiveness of your RAG agent, consider these recommendations:

Knowledge Architecture

Design a well-organized structure for easy access and interpretation by AI systems

Data Quality

Ensure data is accurate, well-formatted, and relevant; this accounts for ~80% of success

Hybrid Technologies

Consider combining NLU for tag extraction with LLMs for generation to improve efficiency

Continuous Optimization

Regularly monitor response times, accuracy, and retrieval effectiveness

​
Wrapping Up

You’ve built a RAG agent without writing code: documents ingested into a Knowledge Base in Knowledges, an agent configured in Agent Creator that retrieves from that Knowledge Base, and a Playground plus Evaluations workflow for ongoing quality. The clean split between knowledge and agent makes it easy to evolve each side independently (swap models, refine instructions, or replace the Knowledge Base) without rebuilding the rest.

​
Next Steps

Document Classification

Learn how to automatically classify and organize your documents with AI

AI Contact Routing

Create an intelligent contact form that routes inquiries to the right department

Webhook Integration

Build advanced integrations between your RAG agent and other systems

Website to RAG

Turn website content into a comprehensive knowledge base for your RAG agent