> ## Documentation Index
> Fetch the complete documentation index at: https://docs.prisme.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Website to RAG Agent

> Create a website chatbot using Knowledges for ingestion and Agent Creator for the agent and embed widget.

This tutorial guides you through creating an intelligent chatbot for your website using Retrieval-Augmented Generation (RAG). You'll set up a Knowledge Base in **Knowledges** that crawls and embeds your website, build an agent in **Agent Creator** wired to that Knowledge Base, and expose a customizable embed widget so visitors can chat with your content directly from your site.

## What You'll Build

* A **Knowledge Base** in Knowledges that crawls your website and stores the content as vector embeddings
* An **agent** in Agent Creator that answers questions using the Knowledge Base as its source of truth
* A **public chat URL** and an **embed widget** generated from the agent's Share Agent dialog, ready to drop into any website

<Note>
  This solution turns static website content into an interactive knowledge base, giving visitors a conversational way to find information instantly.
</Note>

## Prerequisites

Before starting this tutorial, make sure you have:

* An active Prisme.ai account with access to **Knowledges** and **Agent Creator**
* A website URL with content you want to make conversational
* Basic understanding of HTML (to embed the chat widget on your site)

## Step 1: Create a Knowledge Base

Start by creating the Knowledge Base that will store your website content as embeddings.

<Steps>
  <Step title="Open Knowledges">
    From the Prisme.ai platform, open the **Knowledges** product.
  </Step>

  <Step title="Create a new Knowledge Base">
    From **Knowledge Bases** in the Knowledges sidebar, click **+ Create** (or use the Dashboard's **+ New Knowledge Base** tile — same dialog). Give it a clear name (for example, "Company Website") and pick the embedding model you want to use.
  </Step>

  <Step title="Save">
    Save the Knowledge Base to open its detail view.
  </Step>
</Steps>

<img src="https://mintcdn.com/prismeai/K_v8yhAp7bkcmKW-/images/tutorials/knowledges-new-kb-dialog.png?fit=max&auto=format&n=K_v8yhAp7bkcmKW-&q=85&s=58d03ed33b7da7cd5e9537105a9d974f" alt="Knowledges Create Knowledge Base dialog with Name, Description, RAG Configuration (Fast/Balanced/Quality)" width="1440" height="900" data-path="images/tutorials/knowledges-new-kb-dialog.png" />

## Step 2: Add the website as a Web Source

The Knowledge Base ingests pages directly. Inside the Knowledge Base detail page, the **Add Web Source** action handles both single-page additions and full-site auto-discovery.

<Note>
  The global **Connectors** entry in the Knowledges sidebar is a separate, cross-Knowledge-Base concept. It is **not** how websites are added. Web ingestion lives on a Knowledge Base's own detail page, alongside file uploads.
</Note>

<Steps>
  <Step title="Open the Knowledge Base">
    From Knowledges → **Knowledge Bases**, click the Knowledge Base you just created. You land on its detail page where its files and web sources are listed.
  </Step>

  <Step title="Click Add Web Source">
    Use the **Add Web Source** button next to **Add Documents**. (If the Knowledge Base is still empty, the same action is available from the dashed "Add a web source — Crawl a website or add a single URL" tile.)
  </Step>

  <Step title="Pick the right tab">
    The modal has two tabs:

    * **Single URL** — adds one specific page (placeholder `https://example.com/page`). Use this when you only want to ingest a few hand-picked URLs.
    * **Auto-Discovery** — crawls the whole website starting from a root URL. Use this for full-site ingestion.

    For a website chatbot, pick **Auto-Discovery**.
  </Step>

  <Step title="Enter the website URL and tune the crawler">
    Fill in the **Website URL** (e.g. `https://example.com`). The Auto-Discovery tab exposes per-hostname crawler settings:

    * **Path filter** — restrict the crawl to a subtree (e.g. `/docs`).
    * **Blacklisted patterns** — skip URLs matching given patterns (e.g. `/admin/.*`).
    * **Respect robots.txt** — toggle (default on).
    * **Sitemap only** + **Sitemap URL** — drive the crawl from a sitemap.
    * **XPath filter** — only index nodes matching an XPath, e.g. `//article | //main`.
    * **HTTP headers** — custom request headers (e.g. `Authorization`, `Cookie`) for sites that need auth.

    If the hostname is already configured for another URL on the same Knowledge Base, the form pre-fills with the existing config and warns you that any change applies to all URLs on that hostname.
  </Step>

  <Step title="Submit">
    Click **Start Crawling**. The Knowledge Base PATCHes its `websites` list and `websites_config` map and the crawl runs in the background. Once submitted, the new entry appears under the **Web Sources** list on the Knowledge Base page with a per-source row summarizing its config and indexed page count.
  </Step>

  <Step title="Tune the store-level crawler limits">
    Per-page-count limits and re-crawl interval are **Knowledge-Base-level**, not per-source. Open the Knowledge Base **Settings** tab to set:

    * **Max pages** — overall crawl budget for the Knowledge Base.
    * **Re-crawl interval** — manual or a fixed interval.
  </Step>
</Steps>

<Note>
  Crawling and embedding can take time depending on site size. You can begin testing as soon as some pages are processed. A maximum document quota is often enforced on your instance — contact your internal point of contact if you hit the limit.

  You can re-crawl any web source on demand using the row's **Re-crawl** action, edit its hostname config via **Edit Config**, or remove it via **Remove**.
</Note>

## Step 3: Inspect indexed content

Once crawling starts producing results, verify ingestion directly from the Knowledge Base.

<Steps>
  <Step title="Open the Knowledge Base">
    From Knowledges → **Knowledge Bases**, click your Knowledge Base.
  </Step>

  <Step title="Browse Web Sources and indexed pages">
    The Knowledge Base detail page lists each Web Source with its current **pages indexed** count. Underneath, the file list shows the actual indexed pages. Use the **Source type** filter (`all` / `file` / `web`) to focus on web content.
  </Step>

  <Step title="Spot-check chunking and retrieval">
    Click a row to inspect how a page was chunked. The Knowledge Base supports a built-in query test from the chat-with-KB surface — confirm relevant passages come back for representative questions.
  </Step>
</Steps>

## Step 4: Create the agent in Agent Creator

With the Knowledge Base ready, build the agent your visitors will chat with.

<Steps>
  <Step title="Open Agent Creator">
    From the Prisme.ai platform, open **Agent Creator**.
  </Step>

  <Step title="Create a new agent">
    Click **+ Create agent** and set:

    * **Name** (for example, "Website Assistant")
    * **Model** for generation
    * **Instructions** — for example: *"Answer the user based on the indexed website content. Cite sources when possible. If the answer isn't in the indexed content, say so clearly."*
  </Step>

  <Step title="Attach the Knowledge Base">
    In the agent's configuration, attach the Knowledge Base you created in Step 1 as a knowledge source.
  </Step>

  <Step title="Save the agent">
    Save your changes.
  </Step>
</Steps>

<img src="https://mintcdn.com/prismeai/K_v8yhAp7bkcmKW-/images/tutorials/agent-creator-create.png?fit=max&auto=format&n=K_v8yhAp7bkcmKW-&q=85&s=d9b7d65c193273c62d40811db97f569c" alt="Agent Creator Create Agent wizard with From Scratch / Import AGENTS.md options" width="1440" height="900" data-path="images/tutorials/agent-creator-create.png" />

*The screenshot shows the agent creation wizard. Once the agent exists, attach the Knowledge Base from its capabilities/configuration view.*

## Step 5: Test the agent in the playground

<Steps>
  <Step title="Open the Playground">
    From your agent in Agent Creator, open the **Playground** tab.
  </Step>

  <Step title="Ask questions">
    Try realistic questions about your website's content.
  </Step>

  <Step title="Inspect sources">
    Review the cited sources / passages returned alongside each answer to confirm the agent is grounded in the right pages. Refine your instructions or Knowledge Base settings if needed.
  </Step>
</Steps>

## Step 6: Share the agent

Once the agent answers correctly in the Playground, expose it to your audience.

<Steps>
  <Step title="Open the Share menu">
    From your agent in Agent Creator, open the **Share** menu (in the agent header). The dialog gathers the surfaces available on your instance — for example, an access bindings panel for inviting users or groups, and (depending on instance configuration) a public link or embed snippet.
  </Step>

  <Step title="Copy whatever the dialog exposes">
    Copy the link, snippet, or invitation flow shown in the Share dialog. Available options vary by instance — some configurations expose a hosted chat URL and a `<script>` embed for your website, others ship only access-binding controls. Use what your instance shows; don't paste a URL pattern that isn't surfaced in the dialog.
  </Step>
</Steps>

<Note>
  If your instance doesn't ship a public chat / embed widget out of the box, you can still front the agent with a small page in **Builder** that calls `Agents.sendMessage` from a webhook automation (see the [Webhook Builder tutorial](/resources/tutorials/webhook-builder)) and renders the response in a custom React SPA.
</Note>

## Step 7: Place the agent on your site

<Steps>
  <Step title="Use the Share output">
    If your Share dialog produced a hosted URL, link to it from your website. If it produced a `<script>` embed snippet, paste it just before the closing `</body>` tag of your site.
  </Step>

  <Step title="Verify the integration">
    Open your website and confirm the link or embedded chat works against the agent backed by your Knowledge Base.
  </Step>
</Steps>

## Step 8: Understanding the architecture

Your website chatbot is powered by a Retrieval-Augmented Generation (RAG) pipeline that spans Knowledges and Agent Creator:

<Frame>
  <img src="https://mintcdn.com/prismeai/AXY6wkubsI-xGMVN/images/rag-architecture.png?fit=max&auto=format&n=AXY6wkubsI-xGMVN&q=85&s=635f26196d7ad9bebfbd12af05704dde" alt="RAG Architecture" width="9220" height="5396" data-path="images/rag-architecture.png" />
</Frame>

The system works through several key processes:

1. **Document processing**: The Knowledge Base's web crawler fetches the website you registered as a Web Source, segments content into chunks, and stores vector embeddings
2. **Query understanding**: When a visitor asks a question, the agent processes and enhances the query for better retrieval
3. **Relevant content retrieval**: The Knowledge Base returns the most relevant chunks from your indexed pages
4. **Response generation**: The agent uses those chunks as grounded context to generate a comprehensive, accurate response

<Note>
  Knowledges breaks documents into optimally sized chunks for efficient retrieval. Chunk size and overlap are configurable from the Knowledge Base settings, so you can tune them to your content.
</Note>

## Step 9: Advanced customization

For power users who need to go beyond the default ingestion path:

<Steps>
  <Step title="Tune Knowledge Base settings">
    From the Knowledge Base settings, fine-tune embeddings, chunking, and query enhancement.
  </Step>

  <Step title="React to Knowledge Base events">
    Subscribe a Builder automation to Knowledge Base events (document add/update/delete, query) to trigger custom processing — anonymization, audit logging, downstream notifications. See the [Automations reference](/products/ai-builder/automations) for the trigger syntax.
  </Step>

  <Step title="Custom ingestion in Builder">
    For full control, run a Builder workspace with the **Crawler** app and custom automations to feed documents into your Knowledge Base via its API.
  </Step>

  <Step title="Custom code">
    Implement advanced RAG pipelines (LlamaIndex, LangChain, etc.) using Python Custom Code inside Builder, then push results into the Knowledge Base.
  </Step>
</Steps>

## Step 10: Monitoring

Keep your website chatbot performing at its best:

<Steps>
  <Step title="Agent analytics">
    Open the **Analytics** tab on your agent in Agent Creator to track usage, latency, and popular questions.
  </Step>

  <Step title="Insights product">
    For deeper conversation analysis, topic clustering, and adoption metrics across agents, use the [Insights](/products/ai-insights) product.
  </Step>

  <Step title="Refresh content">
    Let the crawler re-run on its configured periodicity (or trigger **Re-crawl** on the Web Source row manually) to keep the Knowledge Base aligned with your live website.
  </Step>

  <Step title="Iterate on instructions">
    Adjust the agent's instructions and Knowledge Base settings based on observed gaps and user feedback.
  </Step>
</Steps>

## Best Practices for Website RAG Agents

To maximize the effectiveness of your website chatbot:

<CardGroup cols="2">
  <Card title="Content Structure" icon="sitemap">
    Organize your website content with clear headings and logical structure for better chunking
  </Card>

  <Card title="Regular Updates" icon="arrows-rotate">
    Schedule regular recrawling to keep your chatbot's knowledge current
  </Card>

  <Card title="User Guidance" icon="hand-pointer">
    Provide example questions to help users understand what they can ask
  </Card>

  <Card title="Response Tuning" icon="sliders">
    Adjust chunk size and retrieval settings based on the nature of your content
  </Card>
</CardGroup>

## Conclusion

You've turned your static website into an interactive, grounded chatbot: Knowledges handles ingestion and retrieval, Agent Creator handles the conversation and the embed surface, and your visitors get fast, contextual answers to their questions.

Beyond the better visitor experience, you also gain visibility into what people are looking for on your site — feeding directly into how you improve your content.

## Next Steps

<CardGroup cols="2">
  <Card title="Build a Document Classification System" icon="file" href="/resources/tutorials/data-classification-agent">
    Create an AI system that automatically organizes uploaded documents
  </Card>

  <Card title="Create a No-Code RAG Agent" icon="robot" href="/resources/tutorials/no-code-rag-agent">
    Explore more advanced RAG capabilities without writing code
  </Card>

  <Card title="Implement Webhook Integration" icon="webhook" href="/resources/tutorials/webhook-builder">
    Connect your RAG agent to other systems using webhooks
  </Card>

  <Card title="Develop Custom Applications" icon="puzzle-piece" href="/resources/tutorials/ai-builder-custom-apps">
    Build more sophisticated applications using Builder
  </Card>
</CardGroup>