Backup to two locations

[holocronology]

Is it possible to configure so that backups are generated in two locations?

[Minituff]

Hmm, not by default. I think you could add the Additional Folders variable and use that to go to a second location.

You could also use the Lifecycle hooks to run a simple copy command to another location once the backup is complete.

[Minituff]

Hi @holocronology ,

I did some work on this today and this feature is now native. Update to v2.5.0 to test it out.

I'm going to close the issue for now (assuming it works), but if it doesn't feel free to re-open this issue or make a new one.

Closed in #298

[holocronology]

I will test it out. Currently I have two instances of the container running. :-)

[Minituff]

That's really smart honestly. Maybe it would have been easier for me to just recommend that 😎.

I also just noticed an issue with version 2.5.0... the secondary locations are being skipped in favor of the Additional Folders variable. I'd hold off on testing it out until I get a patch in.

[holocronology]

It was the way... ;-)

One container would be better, still, IMO. But yes, as best I can tell it isn't working. Granted, my container still indicates version 2.4.3, so that might be part of it. I tried to update it every which way to Sunday, but it wouldn't. Could it be that the version tad internally is still at 2.4.3?

[holocronology]

Just for reference, this is what I basically do now:

  nautical-backup-ext:
    image: minituff/nautical-backup:2
    container_name: nautical-backup-ext
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock 
      - /volume2/docker:/app/source 
      - /volumeUSB1/usbshare/backup/nbu-docker:/app/destination 
    environment: # Optional variables 
      - TZ=America/New_York 
      - CRON_SCHEDULE=33 3 * * * 
      - SKIP_CONTAINERS=xxxxx
      - REPORT_FILE=true
      - BACKUP_ON_START=false
      - OVERRIDE_SOURCE_DIR=xxxxx
    restart: unless-stopped

  nautical-backup:
    image: minituff/nautical-backup:2
    container_name: nautical-backup
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock 
      - /volume2/docker:/app/source 
      - /volume1/backups/nbu-docker:/app/destination 
    environment: # Optional variables 
      - TZ=America/New_York 
      - CRON_SCHEDULE=0 3 * * * 
      - SKIP_CONTAINERS=xxxxx
      - REPORT_FILE=true
      - BACKUP_ON_START=false
      - OVERRIDE_SOURCE_DIR=xxxxx
      #- SECONDARY_DEST_DIRS=/volumeUSB1/usbshare/backup/nbu-docker
    restart: unless-stopped

[Minituff]

Could it be that the version tad internally is still at 2.4.3?

It shouldn't, no--atleast I've never seen that before. I was able to pull down v.2.5.1 from DockerHub, so I'm guessing you haven't gotten the latest image to pull yet.

You should be seeing INFO: Nautical Backup Version: 2.5.1 in the logs otherwise you wont be getting the new features.

Have you checked out the Updating Nautical part of the documentation? You should be able to hard-code it to 2.5.1 which will force Docker to pull that version for now.

---

Also, looking at the this part of your YML:

#- SECONDARY_DEST_DIRS=/volumeUSB1/usbshare/backup/nbu-docker

I don't think this part of the config will work since the Nautical container won't have access to /volumeUSB1/usbshare without you mounting it first. I think it should look like this:

  nautical-backup:
    image: minituff/nautical-backup:2.5.1
    container_name: nautical-backup
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock 
      - /volume2/docker:/app/source 
      - /volume1/backups/nbu-docker:/app/destination
      - /volumeUSB1/usbshare/backup/nbu-docker:/nbu-docker  # <--- Here we mount the secondary location
    environment: # Optional variables 
      - TZ=America/New_York 
      - CRON_SCHEDULE=0 3 * * * 
      - SKIP_CONTAINERS=xxxxx
      - REPORT_FILE=true
      - BACKUP_ON_START=false
      - OVERRIDE_SOURCE_DIR=xxxxx
      - SECONDARY_DEST_DIRS=/nbu-docker  # <---- This is the path inside Nautical
    restart: unless-stopped

I do really like the idea of backing up your volumes to a USB, I might start doing that too.

[holocronology]

It eventually updated on it's own.

I don't think this part of the config will work since the Nautical container won't have access to /volumeUSB1/usbshare without you mounting it first. I think it should look like this:

Yup, exactly and this was the result.

I was just thinking about that as I saw the error and I see your edits above. Testing now.

[holocronology]

Okay, ran it again and it appears to have worked.

I did note the following in the backup report:

2024-08-27 07:07:50.512609 - INFO: Stopping flame...
2024-08-27 07:08:01.058109 - INFO: Backing up flame...
2024-08-27 07:08:01.440469 - ERROR: Destination directory '/external/nbu-test/flame' does not exist
2024-08-27 07:08:01.440663 - INFO: Backing up flame...
2024-08-27 07:08:01.491654 - INFO: Starting flame...
2024-08-27 07:08:02.251897 - INFO: Backup of flame complete!

I assume that the first "backing up flame" was the existing location. Then it moved to the secondary destination, found no folder for flame, hence the error. Then it created the folder and commenced backing up, which was the second "backing up flame" line.

I think the one thing that I would note is that, ideally, there would be a backup report in each destination. However I think that is trivial and just my preference. If you keep this feature, the location of the backup report should be noted in your help documentation. Maybe the report can indicated which "backing up" is going to which destination?

[Minituff]

I assume that the first "backing up flame" was the existing location. Then it moved to the secondary destination, found no folder for flame, hence the error. Then it created the folder and commenced backing up, which was the second "backing up flame" line.

I think this is exactly right.

ideally, there would be a backup report in each destination

I can add this as a feature as well.

Maybe the report can indicated which "backing up" is going to which destination?

Yeah, makes sense as well. I can update the logs as well as the report file to tell you which destination dir it's backing up to.

---

Besides these small tweaks, it sounds like the main functionality is working?

[MMaximuss]

For me the main functionality is not working. My backup report is the same as @holocronology with a second backup line after 'Destination directory does not exist'. But no backup-directories created. The secondary directory is mounted in the right way.

2025-01-12 19:42:35.374957 - INFO: Backing up XXX...
2025-01-12 19:42:35.375322 - DEBUG: RUNNING: 'rsync -raq  /app/source/XXX/ /app/destination/20250112_backup/XXX/'
2025-01-12 19:42:35.515871 - ERROR: Destination directory '/destination1/20250112_backup/XXX' does not exist
2025-01-12 19:42:35.516214 - INFO: Backing up XXX...

But a second container running is a good alternative.

[Minituff]

Destination directory '/destination1/20250112_backup/XXX' does not exist

Im wondering if I just need to have a mkdir -p before it attempts to backup. I can take a look at this.