db2rclone

Database backups to any rclone-supported storage — a spiritual successor to db2fog.

db2rclone dumps your MySQL/MariaDB or PostgreSQL database and ships the compressed dump to S3, Rackspace Cloud Files (Swift), Backblaze B2, Google Cloud Storage, or any of the many other backends rclone supports. It has no runtime gem dependencies: the heavy lifting is done by mysqldump/pg_dump and the actively maintained rclone binary, so the gem stays tiny and stable while storage support keeps growing upstream.

Requirements

• Ruby >= 3.1
• rclone on the PATH (apt install rclone / brew install rclone)
• mysqldump/mariadb-dump or pg_dump + psql for your database

Installation

# Gemfile
gem "db2rclone"

In a Rails app the rake tasks are loaded automatically via a Railtie:

rake db2rclone:backup           # dump + upload
rake db2rclone:list             # list remote backups
rake db2rclone:prune            # apply the retention policy (DRY_RUN=1 to preview)
rake db2rclone:restore[key]     # restore a backup (requires CONFIRM=yes)

Only the app's primary database is backed up. If you run a multi-database app and need the other databases covered too, contributions are very welcome.

Dumps are written to a temporary directory before upload (and restores download there first), so the host needs enough free disk space for one compressed dump. The directory honours the TMPDIR environment variable — worth setting on systems where /tmp is a RAM-backed tmpfs.

Configuration

Create config/db2rclone.yml (see db2rclone.yml.example for full AWS and Rackspace examples). Every key under rclone: is passed straight through to rclone as an environment variable (RCLONE_CONFIG_BACKUP_<KEY>), so all options from the rclone documentation for your backend are available:

• All backends: https://rclone.org/overview/
• AWS S3: https://rclone.org/s3/
• Rackspace / Swift: https://rclone.org/swift/

production:
  bucket: myapp-backups
  prefix: db
  retention:
    keep_daily: 7    # newest backup of each of the 7 most recent days
    keep_weekly: 4   # newest backup of each of the 4 most recent ISO weeks
  rclone:
    type: s3
    provider: AWS
    access_key_id: <%= ENV["AWS_ACCESS_KEY_ID"] %>
    secret_access_key: <%= ENV["AWS_SECRET_ACCESS_KEY"] %>
    region: us-east-1

Retention counts days (and weeks) that have backups, not calendar days: keep_daily: 7 keeps the newest backup of each of the 7 most recent days on which a backup exists. This is a deliberate failsafe — if your backup job silently stops running, prune will never slowly delete your last remaining backups. A policy with both values set to 0 is rejected, since it would prune everything. Preview what prune would delete with rake db2rclone:prune DRY_RUN=1.

Backup filenames carry a second-resolution timestamp. Should two backups ever start within the same second, the second upload is skipped (rclone --ignore-existing) instead of overwriting the first — two dumps taken in the same second are practically identical.

Backups are named after the app and only files matching the configured name: are considered by list and prune, so several apps can safely share a bucket and prefix.

Database credentials are read from your app's database.yml automatically. Add a database: section only to override individual keys, e.g. to dump from a replica:

production:
  # ...
  database:
    host: replica.internal

Credentials never appear on the command line or in the process list: MySQL credentials are passed via a temporary --defaults-extra-file, PostgreSQL credentials via PGPASSWORD, and rclone credentials via environment variables.

Dump options

The flags passed to the dump tool can be overridden with dump_options:.

• MySQL/MariaDB default: --single-transaction --quick (consistent InnoDB snapshot without table locks). Triggers are included by mysqldump by default. Add --routines (stored procedures/functions) or --events (scheduled events) only if your app uses them — both require extra privileges for the backup user: --routines needs SELECT on mysql.proc (MariaDB, MySQL < 8.0) or the global SHOW_ROUTINE/SELECT privilege (MySQL 8.0+), and --events needs the EVENT privilege.
• PostgreSQL default: --clean --if-exists --no-owner --no-privileges (a plain SQL dump that restores cleanly into an existing database). Ownership and GRANT/REVOKE statements are excluded on purpose: they reference roles that may not exist on the restore target — for example a freshly provisioned server — and a single missing role would abort the entire restore. Note that pg_dump never includes role definitions; if you need them, back up pg_dumpall --globals-only separately.

Encrypted backups

The YAML config defines a single rclone remote, so an rclone crypt backend (which wraps a second remote) can't be expressed in the file alone. It still works by defining the inner remote through real environment variables:

production:
  bucket: encrypted           # path within the crypt remote
  rclone:
    type: crypt
    remote: <%= ENV["RCLONE_CRYPT_REMOTE"] %>   # e.g. inner:myapp-backups
    password: <%= ENV["RCLONE_CRYPT_PASSWORD"] %>  # obscured, see rclone docs

with RCLONE_CONFIG_INNER_TYPE=s3, RCLONE_CONFIG_INNER_ACCESS_KEY_ID=... etc. exported in the environment that runs the rake tasks.

Usage outside Rails

The Railtie is optional. Any Ruby script can drive the same operations:

require "db2rclone"

config = Db2Rclone::Config.load(
  path: "db2rclone.yml",
  env: "production",
  database: {
    "adapter" => "postgresql",
    "host" => "localhost",
    "username" => "backup",
    "password" => ENV["DB_PASSWORD"],
    "database" => "myapp",
  },
)

runner = Db2Rclone::Runner.new(config)
runner.backup
runner.prune

Scheduling

Any scheduler works — cron, systemd timers, or an in-app job runner. A SolidQueue recurring task, for example:

# config/recurring.yml
production:
  db_backup:
    command: "Db2Rclone::Runner.for_rails.backup && Db2Rclone::Runner.for_rails.prune"
    schedule: every day at 3am

Restoring

rake db2rclone:list
rake db2rclone:restore[myapp-production-20260715T030000.sql.gz] CONFIRM=yes

The backup is first downloaded to a temporary file and verified (gzip -t); only then is it imported. Not a single statement reaches the database unless the download is complete and intact, so a network failure or a damaged object in storage can never leave you with a half-restored database. Like backup, restore therefore needs enough local disk for one compressed dump (see TMPDIR below). A restore overwrites the current database, hence the explicit CONFIRM=yes.

PostgreSQL restores additionally run inside a single transaction (psql --single-transaction --set ON_ERROR_STOP=1), so even a failed import rolls back cleanly. MySQL/MariaDB has no such transaction: a mid-import SQL error still leaves the database partially restored — the local verification step removes the most likely cause, but restore into a scratch database first if you want to be certain.

The dump's DROP statements will block behind any active connections — stop the app (or terminate its connections) before restoring.

Development

bundle install
bundle exec rspec

The spec suite runs no real database or network commands; all shell interaction goes through a single seam (Db2Rclone::Shell) that specs replace with a recorder.

An integration smoke test (excluded by default) exercises the full dump → upload → restore → prune cycle against a real database, using rclone's local backend as the storage target. CI runs it against MariaDB and PostgreSQL service containers; to run it locally:

DB2RCLONE_INTEGRATION=1 DB2RCLONE_TEST_ADAPTER=postgresql \
  DB2RCLONE_TEST_USERNAME=postgres DB2RCLONE_TEST_PASSWORD=password \
  bundle exec rspec --tag integration

See spec/integration/round_trip_spec.rb for all DB2RCLONE_TEST_* connection variables.

License

MIT — see LICENSE.txt.