Example of Platform Sizing

Capacity: 50GB Elastic File System for shared storage

Configuration: Can be shared between different environments or isolated for each

Bucket Structure: Environment-based separation

• 1 “models” bucket per environment
• 1 “uploads” bucket per environment
• 1 “uploads-public” bucket per environment (behind CDN)

Data Types: RBAC permissions, users, application data

Configuration:

• 3 nodes in replica set
• 2GB RAM and 2 vCPU per node
• 1,000 IOPS

Disk Space: 10GB total storage requirement

Environment Separation:

• 1 “permissions” database per environment
• 1 “users” database per environment
• 1 “collections” database per environment

Version: MongoDB version 6 with path to version 7

Data Types: Real-time EDA streams, permission cache, OIDC sessions, rate limits, application cache

Configuration:

• 1 master and 2 replicas
• 3GB RAM and 2 vCPU per node

Environment Separation: 1 cluster per environment recommended

Version: Redis version 5 or higher

Data Types: Crawl queue, metadata of known documents, search engine configurations

Configuration:

• 1 master and 2 replicas
• 2GB RAM and 2 vCPU per node

Environment Separation: 1 database per environment

Version: Redis version 5 or higher

Scaling Example: 100,000 documents → 600MB RAM

Data Types: Persisted EDA events for traceability and statistics calculations, text content of crawled documents

Configuration:

• 3 nodes
• 8GB RAM and 4 vCPU per node

Disk Space: 400GB per node (NVMe or SSD storage recommended)

Version: Elasticsearch/OpenSearch version 8 or higher

Implementation: Redis Stack (Redis with vector search capabilities)

Data Types: Text chunks accompanied by their vector embeddings

Configuration:

• 1 master and 2 replicas
• 5GB RAM and 2 vCPU per node

Environment Separation: 1 cluster per environment recommended

Version: Redis version 5 or higher with SEARCH and JSON modules

Scaling Example: 100,000 chunks from 20,000 documents requires approximately 2.6GB RAM