Skip to content

Backup Configuration

Jump to bottom
Nikita Ganzikov edited this page Jul 25, 2025 · 2 revisions

This example demonstrates how to configure SeaweedFS with backup capabilities using various storage backends.

Overview

SeaweedFS backup functionality allows you to:

  • Automatically backup data to external storage
  • Support multiple backup destinations (S3, GCS, Azure, etc.)
  • Configure incremental and full backups
  • Set backup schedules and retention policies

Basic Backup Configuration

Simple S3 Backup

Configure basic backup to S3:

apiVersion: seaweed.seaweedfs.com/v1
kind: Seaweed
metadata:
  name: seaweed-with-backup
  namespace: default
spec:
  version: "3.67"
  
  # Core components
  master:
    replicas: 1
  volume:
    replicas: 3
  filer:
    replicas: 1
    s3: true
  
  # Backup configuration
  filerBackup:
    replicas: 1
    sink:
      s3:
        enabled: true
        region: "us-west-2"
        bucket: "seaweedfs-backup"
        directory: "backups"
        awsAccessKeyID: "your-access-key"
        awsSecretAccessKey: "your-secret-key"

Production Backup Configuration

A more robust backup setup with multiple options:

apiVersion: seaweed.seaweedfs.com/v1
kind: Seaweed
metadata:
  name: production-backup-cluster
  namespace: seaweedfs-system
  labels:
    environment: production
    app: seaweedfs
spec:
  version: "3.67"
  
  # Core components
  master:
    replicas: 3
    resources:
      requests:
        cpu: 250m
        memory: 256Mi
      limits:
        cpu: 500m
        memory: 512Mi
  
  volume:
    replicas: 5
    resources:
      requests:
        cpu: 500m
        memory: 1Gi
      limits:
        cpu: 1000m
        memory: 2Gi
    storageClassName: "fast-ssd"
  
  filer:
    replicas: 2
    resources:
      requests:
        cpu: 250m
        memory: 512Mi
      limits:
        cpu: 500m
        memory: 1Gi
    s3: true
    persistence:
      enabled: true
      mountPath: "/data"
      resources:
        requests:
          storage: 10Gi
  
  # Enhanced backup configuration
  filerBackup:
    replicas: 1
    resources:
      requests:
        cpu: 200m
        memory: 256Mi
      limits:
        cpu: 400m
        memory: 512Mi
    
    # Multiple backup destinations
    sink:
      s3:
        enabled: true
        region: "us-west-2"
        bucket: "seaweedfs-backup-primary"
        directory: "daily-backups"
        isIncremental: true
        awsCredentialsSecretRef:
          name: aws-backup-credentials
          mapping:
            awsAccessKeyID: access-key
            awsSecretAccessKey: secret-key

Backup Sink Types

S3 Backup Configuration

filerBackup:
  sink:
    s3:
      enabled: true
      region: "us-west-2"
      bucket: "my-backup-bucket"
      directory: "seaweedfs-backups"
      awsAccessKeyID: "AKIAIOSFODNN7EXAMPLE"
      awsSecretAccessKey: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
      endpoint: "https://s3.amazonaws.com"  # Optional custom endpoint
      isIncremental: true

Google Cloud Storage Backup

filerBackup:
  sink:
    googleCloudStorage:
      enabled: true
      bucket: "my-gcs-backup-bucket"
      directory: "seaweedfs-backups"
      googleApplicationCredentials: "/path/to/credentials.json"
      isIncremental: true

Azure Blob Storage Backup

filerBackup:
  sink:
    azure:
      enabled: true
      accountName: "myazureaccount"
      accountKey: "my-account-key"
      container: "backup-container"
      directory: "seaweedfs-backups"
      isIncremental: true

Backblaze B2 Backup

filerBackup:
  sink:
    backblaze:
      enabled: true
      b2AccountID: "my-account-id"
      b2MasterApplicationKey: "my-master-key"
      bucket: "my-backup-bucket"
      directory: "seaweedfs-backups"
      b2Region: "us-west-002"
      isIncremental: true

Local File System Backup

filerBackup:
  sink:
    local:
      enabled: true
      directory: "/backups/seaweedfs"
      isIncremental: false

Filer-to-Filer Backup

filerBackup:
  sink:
    filer:
      enabled: true
      grpcAddress: "backup-filer:8880"
      directory: "/backups"
      replication: "001"
      collection: "backups"
      ttLSec: 2592000  # 30 days
      isIncremental: true

Deployment Steps

1. Create Credentials

# Create AWS credentials secret
kubectl create secret generic aws-backup-credentials \
  --from-literal=access-key=YOUR_ACCESS_KEY \
  --from-literal=secret-key=YOUR_SECRET_KEY \
  --namespace=default

2. Deploy Cluster with Backup

# Apply the configuration
kubectl apply -f seaweed-with-backup.yaml

3. Monitor Backup Status

# Check backup pod status
kubectl get pods -l app.kubernetes.io/component=filer-backup

# Check backup logs
kubectl logs -l app.kubernetes.io/component=filer-backup -f

# Check backup service
kubectl get svc -l app.kubernetes.io/component=filer-backup

Backup Verification

# Verify S3 backup
aws s3 ls s3://seaweedfs-backup/backups/

# Verify GCS backup
gsutil ls gs://my-gcs-backup-bucket/seaweedfs-backups/

# Verify Azure backup
az storage blob list --container-name backup-container --account-name myazureaccount

Troubleshooting

Common Issues

1. Backup Pod Not Starting

# Check pod events
kubectl describe pod -l app.kubernetes.io/component=filer-backup

# Check resource limits
kubectl top pods -l app.kubernetes.io/component=filer-backup

2. Credential Issues

# Check secret exists
kubectl get secret aws-backup-credentials

# Verify secret content
kubectl get secret aws-backup-credentials -o yaml

3. Network Issues

# Test connectivity to backup destination
kubectl run test-pod --image=busybox --rm -it --restart=Never -- wget -O- https://s3.amazonaws.com

# Check DNS resolution
kubectl run test-pod --image=busybox --rm -it --restart=Never -- nslookup s3.amazonaws.com

Debug Commands

# Check backup configuration
kubectl describe seaweed seaweed-with-backup

# Check backup events
kubectl get events --field-selector involvedObject.name=seaweed-with-backup

# Check backup logs with timestamps
kubectl logs -l app.kubernetes.io/component=filer-backup --timestamps

# Check backup metrics
kubectl port-forward svc/seaweed-with-backup-filer-backup 5555:5555
curl http://localhost:5555/metrics | grep backup

Performance Optimization

Resource Allocation

filerBackup:
  resources:
    requests:
      cpu: 500m
      memory: 1Gi
    limits:
      cpu: 1000m
      memory: 2Gi

Storage Optimization

filerBackup:
  persistence:
    enabled: true
    mountPath: "/data"
    resources:
      requests:
        storage: 10Gi
    storageClassName: "fast-ssd"

Network Optimization

filerBackup:
  sink:
    s3:
      enabled: true
      region: "us-west-2"  # Choose region close to your cluster
      bucket: "seaweedfs-backup"

Security Considerations

Network Security

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: backup-network-policy
  namespace: default
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/component: filer-backup
  policyTypes:
  - Egress
  egress:
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
    ports:
    - protocol: TCP
      port: 443  # HTTPS for cloud storage

Cleanup

# Delete the cluster
kubectl delete seaweed seaweed-with-backup

# Delete credentials
kubectl delete secret aws-backup-credentials
kubectl delete secret gcs-backup-credentials
kubectl delete secret azure-backup-credentials

# Clean up backup data (optional)
aws s3 rm s3://seaweedfs-backup/backups/ --recursive

Next Steps

Related Resources

Clone this wiki locally