A robust backup and restore strategy is essential for protecting your Prisme.ai platform data and ensuring business continuity. This guide provides detailed instructions for backing up and restoring all components of your self-hosted Prisme.ai environment.

​
Backup Strategy

Your Prisme.ai platform requires backing up several components:

Client-Managed Databases

  • MongoDB/compatible database
  • Elasticsearch/OpenSearch
  • Redis

Object Storage

  • S3 or compatible object storage
  • Document files and attachments

Configuration

  • Kubernetes manifests
  • Helm values
  • Terraform state files

Secrets

  • Kubernetes secrets
  • Certificate files
  • API keys and credentials

​
Database Backup Procedures

1

Create MongoDB Backup

Use mongodump to create a full backup of your MongoDB database:
# For standalone MongoDB
mongodump --uri="mongodb://username:password@hostname:port/database" \
  --out=/path/to/backup/mongo/$(date +%Y-%m-%d)

# For MongoDB Atlas
mongodump --uri="mongodb+srv://username:password@cluster.mongodb.net/database" \
  --out=/path/to/backup/mongo/$(date +%Y-%m-%d)
2

Verify MongoDB Backup

Ensure the backup contains all expected data:
# List databases in the backup
find /path/to/backup/mongo/$(date +%Y-%m-%d) -type d -depth 1

# Count documents in a specific collection
mongorestore --uri="mongodb://username:password@hostname:port/database" \
  --nsInclude="database.collection" \
  --dryRun /path/to/backup/mongo/$(date +%Y-%m-%d) | grep "documents to restore"
3

Schedule Regular Backups

Create a cron job to automate daily backups:
# Add to crontab
0 1 * * * /path/to/mongodb-backup-script.sh > /path/to/logs/mongodb-backup-$(date +\%Y-\%m-\%d).log 2>&1
Example backup script (mongodb-backup-script.sh):
#!/bin/bash

BACKUP_DIR="/path/to/backup/mongo"
DATE=$(date +%Y-%m-%d)
RETENTION_DAYS=30

# Create backup
mongodump --uri="mongodb://username:password@hostname:port/database" \
  --out=${BACKUP_DIR}/${DATE} --gzip

# Clean up old backups
find ${BACKUP_DIR} -type d -mtime +${RETENTION_DAYS} -exec rm -rf {} \;

​
Configuration Backup

1

Back Up Kubernetes Resources

Save your Kubernetes configuration resources:
# Back up all resources in the Prisme namespace
mkdir -p /path/to/backup/kubernetes/$(date +%Y-%m-%d)

# Export deployments, services, configmaps, secrets, etc.
kubectl get all,pvc,cm,secret -n prisme-system -o yaml > \
  /path/to/backup/kubernetes/$(date +%Y-%m-%d)/prisme-resources.yaml
2

Back Up Helm Values

Save your Helm chart values for each release:
# Get values for each Helm release
helm get values prisme-core -n prisme-system -o yaml > \
  /path/to/backup/helm/$(date +%Y-%m-%d)/prisme-core-values.yaml

helm get values prisme-securechat -n prisme-system -o yaml > \
  /path/to/backup/helm/$(date +%Y-%m-%d)/prisme-securechat-values.yaml

# Repeat for each product module
3

Back Up Terraform State

If using Terraform, back up your state files:
# If using local state
cp terraform.tfstate terraform.tfstate.backup
cp -r terraform.tfstate.d /path/to/backup/terraform/$(date +%Y-%m-%d)

# If using remote state (recommended), ensure your remote backend has proper backup
Using remote state in Terraform (like S3 with versioning or Terraform Cloud) provides built-in backup capabilities.

​
Restore Procedures

1

Prepare for Restore

Before restoring, stop services that interact with the database:
# Scale down Prisme.ai deployments
kubectl scale deployment -n prisme-system --replicas=0 \
  prisme-api prisme-worker prisme-securechat prisme-knowledge
2

Restore MongoDB Data

Use mongorestore to restore from your backup:
# Full restore
mongorestore --uri="mongodb://username:password@hostname:port/database" \
  --nsFrom="database.*" --nsTo="database.*" \
  /path/to/backup/mongo/YYYY-MM-DD

# Or restore specific collections
mongorestore --uri="mongodb://username:password@hostname:port/database" \
  --nsInclude="database.users" --nsInclude="database.agents" \
  /path/to/backup/mongo/YYYY-MM-DD
Restoring will overwrite existing data. Be sure to validate your backup before proceeding.
3

Restart Services

After restore is complete, scale the services back up:
# Scale up Prisme.ai deployments
kubectl scale deployment -n prisme-system --replicas=1 \
  prisme-api prisme-worker prisme-securechat prisme-knowledge

# Verify pods are running
kubectl get pods -n prisme-system

​
Configuration Restore

1

Restore Kubernetes Resources

Apply your backed-up Kubernetes configurations:
# First, clean up the namespace if necessary
# WARNING: This will delete all resources! Use with caution.
# kubectl delete namespace prisme-system
# kubectl create namespace prisme-system

# Apply backed-up resources
kubectl apply -f /path/to/backup/kubernetes/YYYY-MM-DD/prisme-resources.yaml
Be cautious when restoring resources. Consider restoring specific resource types instead of everything at once:
# Restore only ConfigMaps and Secrets first
kubectl apply -f /path/to/backup/kubernetes/YYYY-MM-DD/configmaps.yaml
kubectl apply -f /path/to/backup/kubernetes/YYYY-MM-DD/secrets.yaml

# Then restore other resources
kubectl apply -f /path/to/backup/kubernetes/YYYY-MM-DD/deployments.yaml
2

Restore Helm Releases

Use your backed-up values to reinstall or upgrade Helm releases:
# Reinstall core
helm upgrade --install prisme-core prisme/prisme-core \
  -f /path/to/backup/helm/YYYY-MM-DD/prisme-core-values.yaml \
  --namespace prisme-system

# Reinstall product modules
helm upgrade --install prisme-securechat prisme/prisme-securechat \
  -f /path/to/backup/helm/YYYY-MM-DD/prisme-securechat-values.yaml \
  --namespace prisme-system

# Repeat for other product modules
3

Restore Terraform State

If you need to restore Terraform state:
# For local state
cp /path/to/backup/terraform/YYYY-MM-DD/terraform.tfstate .
cp -r /path/to/backup/terraform/YYYY-MM-DD/terraform.tfstate.d .

# Verify state
terraform state list
For remote state, follow your backend provider’s restoration process.

​
Disaster Recovery Planning

Define your recovery objectives to guide your backup strategy:

RPO (Recovery Point Objective)

Maximum acceptable data loss in time:
  • Critical data: RPO < 1 hour
  • Important data: RPO < 24 hours
  • Regular data: RPO < 1 week

RTO (Recovery Time Objective)

Maximum acceptable time to restore service:
  • Critical services: RTO < 4 hours
  • Important services: RTO < 24 hours
  • Regular services: RTO < 3 days

​
Testing and Validation

1

Verify Backup Integrity

Regularly test your backups to ensure they can be restored:
# For MongoDB
mongorestore --uri="mongodb://username:password@hostname:port/test_db" \
  --nsFrom="database.*" --nsTo="test_db.*" \
  --dryRun /path/to/backup/mongo/YYYY-MM-DD

# For S3
aws s3 cp s3://your-backup-bucket/sample-file.pdf /tmp/test-restore.pdf
2

Validation Checkpoints

Establish validation points for successful restoration:

Data Validation

  • Record counts match pre-backup state
  • Sample record content is intact
  • Relationships between data are preserved
  • Application-specific data tests pass

Functionality Validation

  • Core services start successfully
  • API endpoints respond correctly
  • Authentication and authorization work
  • Data processing functions operate properly
  • UI elements display and function as expected
3

Document Restoration Procedures

Maintain detailed, tested restoration runbooks:
  • Step-by-step instructions
  • Required credentials and access
  • Validation checkpoints
  • Troubleshooting guidance
  • Contact information for support

​
Next Steps

Updates

Keep your platform current with the latest updates

Scaling

Scale your platform to meet growing demands