Collection

The Collection app is a powerful infrastructure component in the Prisme.ai ecosystem that provides simplified access to a document database. It enables you to store, retrieve, and manage structured data without the complexity of setting up and maintaining a separate database system.

Overview

Collection serves as a managed database service integrated directly into the Prisme.ai platform:

Data Storage

Store structured data in document collections

Query Capabilities

Retrieve and filter data with powerful query options

Data Management

Create, update, and delete records with simple operations

Integration Ready

Seamlessly connect with automations and workflows

This infrastructure app is particularly valuable for persistently storing information, managing application state, and building data-driven automations without external database dependencies.

Key Features

Store flexible document structures:

Schema-free Design: No rigid schema requirements
Nested Data Support: Store complex, hierarchical data
Data Types: Support for strings, numbers, booleans, arrays, objects, dates
Automatic Indexing: Optimized for fast retrieval
Document Size: Support for documents up to 16MB

This flexible storage model accommodates a wide range of data needs.

How Collection Works

Collection provides a MongoDB-compatible interface integrated directly into the Prisme.ai platform:

Collections Organization

Data is organized into collections, similar to tables in relational databases:

Each collection contains related documents
Collections are created automatically when used
No schema definition is required
Each workspace has its own collection namespace

Document Structure

Documents are stored as JSON-like objects:

Each document has a unique _id field
Documents can have any structure
Fields can contain various data types
Nested objects and arrays are supported
Documents in the same collection can have different structures

Data Operations

Operations are performed through simple, intuitive methods:

Commands follow MongoDB syntax and patterns
Results are returned in standard formats
Operations are executed in a secure environment
Performance is optimized for common use cases

Integration

Collection integrates with the rest of the Prisme.ai ecosystem:

Direct usage in automations
Connection to AI agents through tools
Data exchange with other platform components
Role-based access control

This approach provides the power of a document database with the simplicity of a fully managed service.

Basic Operations

Let’s explore the core operations you can perform with Collection:

Inserting Data

Add documents to a collection:

# Insert a single document
- Collection.insert:
    data:
      name: "John Doe"
      email: "john.doe@example.com"
      age: 30
      active: true
    output: result

# Insert multiple documents
- Collection.insertMany:
    data:
      - name: "Jane Smith"
        email: "jane.smith@example.com"
        age: 28
      - name: "Bob Johnson"
        email: "bob.johnson@example.com"
        age: 35
    output: result

The operation returns information about the inserted documents, including their assigned _id values.

Finding Data

Retrieve documents from a collection:

# Find documents matching criteria
- Collection.find:
    query:
      age: { $gt: 25 }
    output: users

# Find a single document
- Collection.findOne:
    query:
      email: "john.doe@example.com"
    output: user

# Find + sort + pagination
- Collection.find:
    query:
      active: true
    sort:
      age: -1  # Descending
    options:
      limit: 10
      skip: 0
    output: activeUsers

These operations allow you to retrieve documents with precise filtering and control over the results.

Updating Data

By default, updates use mongo $set operator to only update given fields without removing other fields already existing in the matched record :

# Update a single document
- Collection.updateOne:
    query:
      email: "john.doe@example.com"
    data:
      age: 31
      lastUpdated: "{{run.date}}"
    output: updateResult

But you can also use the MongoDB operators you want :

# Update a single document
- Collection.updateOne:
    query:
      email: "john.doe@example.com"
    data:
      $set:
        age: 31
        lastUpdated: "{% now() %}"
    output: updateResult

# Update multiple documents
- Collection.updateMany:
    query:
      active: false
    data:
      $set:
        status: "inactive"
        lastChecked: "{{run.date}}"
    output: updateResult

# Update with advanced operators
- Collection.updateOne:
    query:
      _id: "{{userId}}"
    data:
      $inc:
        loginCount: 1
      $push:
        loginHistory:
          date: "{{run.date}}"
          ip: "{{userIp}}"
    output: updateResult

These operations enable precise updates to documents, including field modifications, additions, and array operations.

Deleting Data

Remove documents from a collection:

# Delete a single document
- Collection.deleteOne:
    query:
      email: "john.doe@example.com"
    output: deleteResult

# Delete multiple documents
- Collection.deleteMany:
    query:
      active: false
      lastLogin: { $lt: "{% dateAdd('now', -90, 'days') %}" }
    output: deleteResult

These operations allow you to remove documents based on specific criteria.

Aggregation Operations

Perform complex data analysis:

# Aggregate data
- Collection.find:
    aggregate:
      - $match:
          active: true
      - $group:
          _id: "$department"
          count: { $sum: 1 }
          avgAge: { $avg: "$age" }
          employees: { $push: { name: "$name", email: "$email" } }
      - $sort:
          count: -1
    output: departmentStats

Aggregation provides powerful data analysis capabilities, including grouping, filtering, and computation.

Find pagination

Pagination is enforced by returning only the first 50 matching entries by default. This number is configurable with options.limit.
You can then choose which page you are interested in using options.page, starting at 1 for the first page :

# First page :  
- Collection.find:
    query: {}
    options:
      limit: 50
      page: 1
    output: firstPage
- Collection.find:
    query: {}
    options:
      limit: 50
      page: 2
    output: secondPage

Alternatively, you can use options.skip to finely select the matching page :

# Retrieve from the 11th to the 51th  record
- Collection.find:
    query: {}
    options:
      limit: 50
      skip: 10
    output: firstPage

Advanced Features

Collection includes several advanced features that enable sophisticated data management:

Indexing

Create indexes to optimize query performance:

Single-field indexes
Compound indexes
Text indexes for full-text search
Unique indexes for constraint enforcement

Transactions

Ensure data consistency with multi-document transactions:

Atomic operations across multiple documents
Rollback on error
Consistent reads within a transaction
Isolation levels

Geospatial

Store and query location data:

GeoJSON format support
Proximity queries
Geospatial indexing
Area containment queries

Schema Validation

Optional schema validation for data consistency:

JSON Schema validation
Custom validation rules
Validation actions (error or warning)
Field restriction

These advanced features provide additional capabilities for specific use cases and requirements.

Common Use Cases

Collection enables a wide range of use cases:

User Management

Store and manage user information:

User profiles
Preferences
Activity history
Authentication data

Content Management

Manage structured content:

Articles and posts
Product information
Media metadata
Categorization and tagging

Workflow State

Track process and workflow state:

Status tracking
Approval flows
Stage information
Audit history

Data Collection

Collect and store form submissions:

Survey responses
Application data
Contact requests
Registration information

Integration with Prisme.ai Products

Collection works seamlessly with other Prisme.ai products:

Enhance knowledge bases with Collection:

Store metadata about knowledge base documents
Track usage patterns and popular queries
Maintain user feedback on responses
Save and manage test results

This integration improves knowledge management and quality assurance.

Example: Contact Management System

Here’s an example of using Collection to build a contact management system:

Define Data Structure

Plan your data organization:

Contacts collection for individual contacts
Companies collection for organization information
Interactions collection for communication history
Tags collection for categorization

Create Storage Operations

Implement data storage automations:

# Add a new contact
slug: add-contact
name: Add Contact
do:
  - Collection.insert:
      data:
        firstName: "{{payload.firstName}}"
        lastName: "{{payload.lastName}}"
        email: "{{payload.email}}"
        phone: "{{payload.phone}}"
        company: "{{payload.company}}"
        title: "{{payload.title}}"
        tags: "{{payload.tags}}"
        createdAt: "{% now() %}"
      output: result
  - emit:
      event: contact-added
      payload:
        contact: "{{result}}"

Implement Query Operations

Create data retrieval automations:

# Search for contacts
slug: search-contacts
name: Search Contacts
do:
  - set:
      name: query
      value: {}
  - conditions:
      '{{payload.searchTerm}}':
        - set:
            name: query
            value:
              $or:
                - firstName: { $regex: "{{payload.searchTerm}}", $options: "i" }
                - lastName: { $regex: "{{payload.searchTerm}}", $options: "i" }
                - email: { $regex: "{{payload.searchTerm}}", $options: "i" }
                - company: { $regex: "{{payload.searchTerm}}", $options: "i" }
      '{{payload.tags}}':
        - set:
            name: query.tags
            value: { $in: "{{payload.tags}}" }
      default: []
  - Collection.find:
      query: "{{query}}"
      sort:
        lastName: 1
        firstName: 1
      options:
        limit: "{{payload.limit || 20}}"
        skip: "{{payload.skip || 0}}"
      output: contacts
  - emit:
      event: search-results
      payload:
        contacts: "{{contacts}}"
        query: "{{query}}"

Create User Interface

Build a UI to interact with your data:

Contact list view
Contact detail view
Add/edit contact forms
Search and filtering

Implement Business Logic

Add specialized functionality:

Duplicate detection
Contact merging
Import/export capabilities
Notification system

This example demonstrates how Collection can serve as the data layer for a complete application.

Best Practices

Follow these recommendations to get the most from Collection:

Data Modeling

Query Optimization

Data Validation

Security Considerations

Limitations and Considerations

When using Collection, be aware of these considerations:

Document Size: Individual documents are limited to 16MB
Nested Depth: Deep nesting of objects can impact performance
Query Complexity: Very complex queries may have performance implications
Transaction Limits: Transactions have time and size limitations
Indexing Overhead: Indexes improve query performance but increase storage requirements and write overhead
Consistency Model: Collection uses an eventually consistent model in some scenarios

Next Steps

API Integrations

Learn about connecting to external APIs

Crawler

Discover web content extraction capabilities

Custom Code

Execute custom logic within your workflows

Extending Apps

Create your own custom integrations

Integration Overview

Apps Marketplace

Overview

Data Storage

Query Capabilities

Data Management

Integration Ready

Key Features

How Collection Works

Basic Operations

Advanced Features

Indexing

Transactions

Geospatial

Schema Validation

Common Use Cases

User Management

Content Management

Workflow State

Data Collection

Integration with Prisme.ai Products

Example: Contact Management System

Best Practices

Limitations and Considerations

Next Steps

API Integrations

Crawler

Custom Code

Extending Apps

Integration Overview

Apps Marketplace

​Overview

Data Storage

Query Capabilities

Data Management

Integration Ready

​Key Features

​How Collection Works

​Basic Operations

​Advanced Features

Indexing

Transactions

Geospatial

Schema Validation

​Common Use Cases

User Management

Content Management

Workflow State

Data Collection

​Integration with Prisme.ai Products

​Example: Contact Management System

​Best Practices

​Limitations and Considerations

​Next Steps

API Integrations

Crawler

Custom Code

Extending Apps

Overview

Key Features

How Collection Works

Basic Operations

Advanced Features

Common Use Cases

Integration with Prisme.ai Products

Example: Contact Management System

Best Practices

Limitations and Considerations

Next Steps