Skip to content

Batch re-encryption CLI tool #17

@ServerSideHannes

Description

@ServerSideHannes

Summary

Provide a CLI tool to batch re-encrypt objects — for key rotation, migrating legacy single-blob objects to multipart format, or changing encryption parameters.

Problem

Several operations require re-processing existing encrypted objects:

There's no tooling to do this safely at scale.

Proposal

# Re-wrap DEKs with new master key
s3proxy batch rewrap --bucket my-bucket --old-key <old> --new-key <new>

# Convert single-blob objects to multipart
s3proxy batch migrate --bucket my-bucket --prefix data/

# Full re-encrypt (download, decrypt, re-encrypt, upload)
s3proxy batch reencrypt --bucket my-bucket --concurrency 4

Features:

  • Progress tracking with resume support (store checkpoint in a state file)
  • Dry-run mode
  • Concurrency control (respect memory limits)
  • Filters: by prefix, size range, last-modified date
  • Report: objects processed, skipped, failed

Considerations

  • Must be idempotent and safe to interrupt/resume
  • Should not interfere with live reads/writes
  • Verify integrity after re-encryption (compare plaintext hash)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions