Scaling

As your organization’s usage of Prisme.ai grows, you’ll need to scale your self-hosted platform to maintain performance and reliability. This guide provides strategies and best practices for scaling different components of your Prisme.ai deployment.

Scaling Approaches

Horizontal scaling involves adding more instances (pods, nodes) to distribute load:

Benefits:

Better fault tolerance and availability
Linear capacity scaling
No downtime during scaling operations

Considerations:

Requires stateless application design
More complex networking
Service discovery requirements

When to Scale

Performance Indicators

Monitor these key metrics to identify scaling needs:

API response times exceeding thresholds
CPU utilization consistently above 70%
Memory utilization consistently above 80%
Request queue depth increasing
Database query times growing

Growth Indicators

Business metrics that suggest scaling requirements:

Increasing number of users
Growing document count
More concurrent sessions
Higher query volume
Additional knowledge bases

Preventative Scaling

Proactive scaling for anticipated demands:

Before major rollouts
Ahead of seasonal peaks
Prior to marketing campaigns
In advance of organizational growth

Recovery Objectives

Scaling to meet resilience targets:

Redundancy requirements
High availability goals
Load distribution needs
Geographic distribution objectives

Scaling Core Components

Assess Current Usage

Gather metrics on current performance and resource utilization:

# Check CPU and memory usage of pods
kubectl top pods -n prisme-system

# Review HPA metrics if configured
kubectl get hpa -n prisme-system

Configure HPA (Horizontal Pod Autoscaler)

Set up automatic scaling based on metrics:

# Sample HPA configuration for API
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: prisme-api
  namespace: prisme-system
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: prisme-api
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80

Apply the configuration:

kubectl apply -f api-hpa.yaml

Update Helm Values

Alternatively, configure scaling parameters in your Helm values:

# In values.yaml
api:
  replicaCount: 3
  autoscaling:
    enabled: true
    minReplicas: 2
    maxReplicas: 10
    targetCPUUtilizationPercentage: 70
    targetMemoryUtilizationPercentage: 80
    
worker:
  replicaCount: 3
  autoscaling:
    enabled: true
    minReplicas: 2
    maxReplicas: 10
    targetCPUUtilizationPercentage: 70
    targetMemoryUtilizationPercentage: 80

Apply the configuration:

helm upgrade prisme-core prisme/prisme-core -f values.yaml -n prisme-system

Set Resource Requests and Limits

Define appropriate resource allocations:

# In values.yaml
api:
  resources:
    requests:
      cpu: 500m
      memory: 1Gi
    limits:
      cpu: 2000m
      memory: 4Gi
      
worker:
  resources:
    requests:
      cpu: 500m
      memory: 2Gi
    limits:
      cpu: 2000m
      memory: 6Gi

Set resource requests and limits based on observed usage patterns. Start conservative and adjust based on monitoring data.

Configure Pod Disruption Budgets

Ensure high availability during scaling:

# In values.yaml
api:
  podDisruptionBudget:
    enabled: true
    minAvailable: 1
    # or maxUnavailable: 25%

Assess Current Usage

Gather metrics on current performance and resource utilization:

# Check CPU and memory usage of pods
kubectl top pods -n prisme-system

# Review HPA metrics if configured
kubectl get hpa -n prisme-system

Configure HPA (Horizontal Pod Autoscaler)

Set up automatic scaling based on metrics:

# Sample HPA configuration for API
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: prisme-api
  namespace: prisme-system
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: prisme-api
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80

Apply the configuration:

kubectl apply -f api-hpa.yaml

Update Helm Values

Alternatively, configure scaling parameters in your Helm values:

# In values.yaml
api:
  replicaCount: 3
  autoscaling:
    enabled: true
    minReplicas: 2
    maxReplicas: 10
    targetCPUUtilizationPercentage: 70
    targetMemoryUtilizationPercentage: 80
    
worker:
  replicaCount: 3
  autoscaling:
    enabled: true
    minReplicas: 2
    maxReplicas: 10
    targetCPUUtilizationPercentage: 70
    targetMemoryUtilizationPercentage: 80

Apply the configuration:

helm upgrade prisme-core prisme/prisme-core -f values.yaml -n prisme-system

Set Resource Requests and Limits

Define appropriate resource allocations:

# In values.yaml
api:
  resources:
    requests:
      cpu: 500m
      memory: 1Gi
    limits:
      cpu: 2000m
      memory: 4Gi
      
worker:
  resources:
    requests:
      cpu: 500m
      memory: 2Gi
    limits:
      cpu: 2000m
      memory: 6Gi

Set resource requests and limits based on observed usage patterns. Start conservative and adjust based on monitoring data.

Configure Pod Disruption Budgets

Ensure high availability during scaling:

# In values.yaml
api:
  podDisruptionBudget:
    enabled: true
    minAvailable: 1
    # or maxUnavailable: 25%

Scale Individual Products

Each Prisme.ai product module can be scaled independently:

# Check current deployment status
kubectl get deploy -n prisme-system -l app.kubernetes.io/component=securechat

# Manual scaling
kubectl scale deployment prisme-securechat -n prisme-system --replicas=3

Using Helm values:

# In securechat-values.yaml
replicaCount: 3
autoscaling:
  enabled: true
  minReplicas: 2
  maxReplicas: 8

Apply the configuration:

helm upgrade prisme-securechat prisme/prisme-securechat -f securechat-values.yaml -n prisme-system

Scale Based on Product Usage

Different products may require different scaling approaches:

AI Knowledge

Scale for document processing load
Increase resources for large knowledge bases
Tune based on retrieval volume

AI SecureChat

Scale based on concurrent user sessions
Provision for message throughput
Consider message storage requirements

AI Store

Scale for catalog browsing traffic
Provision for agent deployment operations
Consider metadata storage needs

AI Builder

Scale for concurrent development sessions
Increase resources for complex builds
Consider testing environment requirements

Scale Ingress Controller

Ensure your ingress controller can handle increased traffic:

# Example for NGINX ingress controller
controller:
  replicaCount: 3
  autoscaling:
    enabled: true
    minReplicas: 2
    maxReplicas: 5
  resources:
    requests:
      cpu: 100m
      memory: 256Mi
    limits:
      cpu: 500m
      memory: 512Mi

Configure Connection Pooling

Optimize connection handling for scaled deployments:

# In values.yaml
api:
  config:
    db:
      poolSize: 50
      maxPoolSize: 100
    redis:
      maxClients: 100

Implement Caching

Add Redis caching for frequently accessed data:

# In values.yaml
redis:
  enabled: true
  replica:
    replicaCount: 3
  cache:
    enabled: true

Scaling Database Components

Implement Replica Sets

Deploy MongoDB with replica sets for high availability and read scaling:

# MongoDB Helm values example
architecture: replicaset
replicaCount: 3
arbiter:
  enabled: false

readinessProbe:
  enabled: true
  initialDelaySeconds: 30
  periodSeconds: 10
  timeoutSeconds: 5
  failureThreshold: 6
  successThreshold: 1

Configure Sharding

For very large deployments, implement MongoDB sharding:

Set up config servers (typically 3 nodes)
Deploy shard servers (multiple replica sets)
Configure mongos routers
Define shard keys based on data access patterns

Sharding adds complexity and should only be implemented when dataset size exceeds what a single replica set can handle efficiently.

Optimize Indexes

Ensure proper indexes exist for common queries:

// Execute via MongoDB shell

// For user queries
db.users.createIndex({ "email": 1 }, { unique: true })

// For document searches
db.documents.createIndex({ "title": "text", "content": "text" })
db.documents.createIndex({ "metadata.tags": 1 })

// For agent queries
db.agents.createIndex({ "workspaceId": 1, "type": 1 })

Scale MongoDB Resources

Increase resources for MongoDB instances:

# MongoDB Helm values
resources:
  requests:
    cpu: 2
    memory: 4Gi
  limits:
    cpu: 4
    memory: 8Gi

Implement Replica Sets

Deploy MongoDB with replica sets for high availability and read scaling:

# MongoDB Helm values example
architecture: replicaset
replicaCount: 3
arbiter:
  enabled: false

readinessProbe:
  enabled: true
  initialDelaySeconds: 30
  periodSeconds: 10
  timeoutSeconds: 5
  failureThreshold: 6
  successThreshold: 1

Configure Sharding

For very large deployments, implement MongoDB sharding:

Set up config servers (typically 3 nodes)
Deploy shard servers (multiple replica sets)
Configure mongos routers
Define shard keys based on data access patterns

Sharding adds complexity and should only be implemented when dataset size exceeds what a single replica set can handle efficiently.

Optimize Indexes

Ensure proper indexes exist for common queries:

// Execute via MongoDB shell

// For user queries
db.users.createIndex({ "email": 1 }, { unique: true })

// For document searches
db.documents.createIndex({ "title": "text", "content": "text" })
db.documents.createIndex({ "metadata.tags": 1 })

// For agent queries
db.agents.createIndex({ "workspaceId": 1, "type": 1 })

Scale MongoDB Resources

Increase resources for MongoDB instances:

# MongoDB Helm values
resources:
  requests:
    cpu: 2
    memory: 4Gi
  limits:
    cpu: 4
    memory: 8Gi

Scale Cluster Size

Add more nodes to your Elasticsearch/OpenSearch cluster:

# Elasticsearch Helm values example
master:
  replicaCount: 3
  
data:
  replicaCount: 5
  
coordinating:
  replicaCount: 3

Configure Node Roles

Optimize cluster by separating node roles:

Master nodes: Cluster management
Data nodes: Store and search data
Coordinating nodes: Handle queries and distribute load
Ingest nodes: Pre-process documents

Optimize Index Settings

Configure index settings for optimal performance:

PUT /knowledge_index/_settings
{
  "index": {
    "number_of_shards": 5,
    "number_of_replicas": 1,
    "refresh_interval": "30s"
  }
}

Implement Index Lifecycle Management

Set up ILM policies for managing growing indices:

PUT _ilm/policy/prisme_logs_policy
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_size": "50GB",
            "max_age": "30d"
          }
        }
      },
      "warm": {
        "min_age": "7d",
        "actions": {
          "forcemerge": {
            "max_num_segments": 1
          },
          "shrink": {
            "number_of_shards": 1
          }
        }
      },
      "cold": {
        "min_age": "30d",
        "actions": {
          "allocate": {
            "require": {
              "data": "cold"
            }
          }
        }
      },
      "delete": {
        "min_age": "90d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

Implement Redis Cluster

Deploy Redis in cluster mode for horizontal scaling:

# Redis Helm values example
architecture: replication
replicaCount: 3

cluster:
  enabled: true
  slaveCount: 2
  
master:
  persistence:
    enabled: true
    size: 8Gi

Optimize Redis Configuration

Tune Redis settings for performance:

# Redis configuration
maxmemory 4gb
maxmemory-policy allkeys-lru

# Connection settings
maxclients 10000
timeout 300

# Persistence settings
save 900 1
save 300 10
save 60 10000

Monitor and Scale

Set up monitoring to detect Redis bottlenecks:

# Using redis-cli to check info
redis-cli -h redis-host info | grep used_memory
redis-cli -h redis-host info | grep connected_clients

Scaling Storage

Scale Object Storage

S3 or compatible object storage typically scales automatically, but ensure proper configuration:

Performance Options

Enable transfer acceleration
Use multipart uploads for large files
Implement appropriate file organization
Consider regional deployments for global access

Cost Optimization

Implement lifecycle policies
Use appropriate storage classes
Enable compression where applicable
Monitor usage patterns

Scale Persistent Volumes

Adjust storage for stateful components:

# Example PVC expansion (if storage class supports it)
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: prisme-mongodb
  namespace: prisme-system
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 100Gi  # Increased from previous size
  storageClassName: standard

Not all storage classes support volume expansion. Check your cloud provider or storage system capabilities.

Scaling Infrastructure with Terraform

Scale Kubernetes Nodes

Adjust your node groups in Terraform:

# For AWS EKS
module "eks_node_group" {
  source = "terraform-aws-modules/eks/aws//modules/eks-managed-node-group"
  
  name = "prisme-workers"
  
  min_size     = 3
  max_size     = 10
  desired_size = 5
  
  instance_types = ["m5.xlarge", "m5.2xlarge"]
  capacity_type  = "ON_DEMAND"
  
  # ... other settings
}

# For Azure AKS
resource "azurerm_kubernetes_cluster_node_pool" "prisme_workers" {
  name                  = "prismeworkers"
  kubernetes_cluster_id = azurerm_kubernetes_cluster.prisme.id
  vm_size               = "Standard_D4_v3"
  node_count            = 5
  min_count             = 3
  max_count             = 10
  enable_auto_scaling   = true
  
  # ... other settings
}

Configure Node Autoscaling

Set up cluster autoscaler for automatic node provisioning:

# For AWS EKS
resource "helm_release" "cluster_autoscaler" {
  name       = "cluster-autoscaler"
  repository = "https://kubernetes.github.io/autoscaler"
  chart      = "cluster-autoscaler"
  namespace  = "kube-system"
  
  set {
    name  = "autoDiscovery.clusterName"
    value = module.eks.cluster_id
  }
  
  set {
    name  = "awsRegion"
    value = var.region
  }
  
  # ... other settings
}

Implement Regional Deployments

For global deployments, consider multi-region architecture:

Deploy Prisme.ai in multiple regions
Use global load balancing (e.g., Route53, Azure Traffic Manager)
Replicate databases across regions
Synchronize object storage

Monitoring for Scaling Decisions

Key Metrics to Watch

Core metrics that indicate scaling needs:

API response time > 200ms
CPU utilization > 70% sustained
Memory usage > 80% sustained
Queue depth increasing
Connection timeouts occurring

Monitoring Tools

Tools to implement for scaling insights:

Prometheus + Grafana
Kubernetes metrics server
Custom dashboards for Prisme.ai services
Database-specific monitoring

Alert Thresholds

Set up alerts to trigger scaling actions:

Warning: 60% resource utilization
Critical: 80% resource utilization
Performance degradation > 50%
Error rate increase > 10%

Scaling Dashboards

Create dashboards focused on scaling metrics:

Resource usage trends
Traffic patterns
Database performance
Storage growth rates

Scaling Best Practices

Implement Gradual Scaling

Scale resources incrementally rather than making large changes at once:

Increase replicas by 50-100% at a time
Monitor effects before further scaling
Allow system to stabilize between changes
Document performance impacts

Test Before Production

Validate scaling changes in non-production environments:

Use load testing tools (JMeter, k6, Locust)
Simulate real-world usage patterns
Test both scaling up and scaling down
Verify application behavior during scaling events

Automate Where Possible

Use automation to handle routine scaling:

Implement Horizontal Pod Autoscalers (HPA)
Configure cluster autoscaling
Use scheduled scaling for predictable patterns
Set up anomaly detection for unexpected loads

Document Scaling Procedures

Maintain clear documentation for scaling operations:

Standard operating procedures
Emergency scaling runbooks
Performance baselines
Historical scaling decisions and outcomes

Next Steps

Updates

Keep your platform current with the latest updates

Backup & Restore

Protect your data with comprehensive backup strategies

Overview

Cloud Providers

Docker & Kubernetes Deployment

Entreprise Services

AI Products

Operations

Scaling Approaches

When to Scale

Performance Indicators

Growth Indicators

Preventative Scaling

Recovery Objectives

Scaling Core Components

AI Knowledge

AI SecureChat

AI Store

AI Builder

Scaling Database Components

Scaling Storage

Performance Options

Cost Optimization

Scaling Infrastructure with Terraform

Monitoring for Scaling Decisions

Key Metrics to Watch

Monitoring Tools

Alert Thresholds

Scaling Dashboards

Scaling Best Practices

Next Steps

Updates

Backup & Restore

Overview

Cloud Providers

Docker & Kubernetes Deployment

Entreprise Services

AI Products

Operations

​Scaling Approaches

​When to Scale

Performance Indicators

Growth Indicators

Preventative Scaling

Recovery Objectives

​Scaling Core Components

AI Knowledge

AI SecureChat

AI Store

AI Builder

​Scaling Database Components

​Scaling Storage

Performance Options

Cost Optimization

​Scaling Infrastructure with Terraform

​Monitoring for Scaling Decisions

Key Metrics to Watch

Monitoring Tools

Alert Thresholds

Scaling Dashboards

​Scaling Best Practices

​Next Steps

Updates

Backup & Restore

Scaling Approaches

When to Scale

Scaling Core Components

Scaling Database Components

Scaling Storage

Scaling Infrastructure with Terraform

Monitoring for Scaling Decisions

Scaling Best Practices

Next Steps