Skipped Containers

[alexkiddddd]

Hi there, I have created a simple nautical container to backup my docker containers but some containers are skipped, how can I find out why?

Thanks

[alexkiddddd]

I have added the debug log and found out that the container name does not matched the folder.
"Normally a container is backed up only when the container-name is the exact same as the source folder name."

[khallingstad]

@alexkiddddd Thank you for the fix. I can confirm that this fixed the issue for me. In my case my container names are different from my docker data volume names, once I set the docker data volume name to the same name as the container it worked right away with require label true.

[Minituff]

Hey @khallingstad, so all is working now? Is there anything I need to fix?

[khallingstad]

Hey @Minituff yes all working good now! In hindsight it makes total sense that docker data name needs to be the same as the container name. Thank you for a great tool!

[mixpc]

Hi, little remark here. In the case of a docker compose file with more than one service (eg. wordpress) there would be two container names, one of which would not match the folder name... as in this case:

docker-compose.yml for nautical backup
version: '3'
services:
  nautical-backup:
    image: minituff/nautical-backup:latest
    container_name: nautical-backup
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /home/user/docker:/app/source
      - /home/user/dockerbackup:/app/destination
    environment: # Optional variables
      - LOG_LEVEL=DEBUG
      - CRON_SCHEDULE=05 21 * * *
      - SKIP_CONTAINERS=nautical-backup
    restart: unless-stopped

and here using the new group labels:

docker-compose.yml for wordpress
version: '3.1'

services:

  wordpress:
    image: wordpress:6.5
    container_name: wordpress-blog
    restart: unless-stopped
    ports:
      - 80:80
    environment:
      WORDPRESS_DB_HOST: xxx
      WORDPRESS_DB_USER: xxx
      WORDPRESS_DB_PASSWORD: xxx
      WORDPRESS_DB_NAME: xxx
    volumes:
      - ./html:/var/www/html
    labels:
      - "nautical-backup.group=wp"

  db:
    image: mysql:8.0
    container_name: wordpress-db
    restart: always
    environment:
      MYSQL_DATABASE: xxx
      MYSQL_USER: uxxx
      MYSQL_PASSWORD: xxx
      MYSQL_RANDOM_ROOT_PASSWORD: xxx
    volumes:
      - ./db:/var/lib/mysql

    labels:
      - "nautical-backup.group=wp"

@Minituff after several attempts I can confirm the group label does not seem to work as expected (unless I have not configured it correctly, of course)

nautical backup log shows:

DEBUG: Containers completed: many here!
DEBUG: Containers skipped: wordpress-blog, wordpress-db
INFO: Success. 5 containers backed up! 2 skipped.

[Minituff]

Hey, just taking a look at this now.

• Are you trying to use the nautical-backup.group label to have the wordpress-blog and wordpress-db stopped, backed up, and then started at the same time?

• Or are you just trying to back up the volumes? I see in the logs they are being skipped.

  • If so, which paths would you like Nautical to backup? I only see ./html and ./db

[mixpc]

Hi, thank you for your interest.

• Are you trying to use the nautical-backup.group label to have the wordpress-blog and wordpress-db stopped, backed up, and then started at the same time?

Correct

I would expect the volumes ./html and ./db to be backed up as well. So all in all, I would expect the whole
/home/user/docker/worpdress folder to be backed up. As suggested, I may have it wrongly configured, fyi.

[Minituff]

Gotcha, I'm guessing the nautical-backup.group would work, but the containers are getting skipped because of a name mismatch.

• The container wordpress-db has a folder named db so Nautical is skipping it since it doesn't match the name.
• The container wordpress-blog has a folder named html so Nautical is skipping it since it doesn't match the name.

You have two options that can fix it:

Option 1

Use overrides.

 wordpress:
   # Rest of config
    volumes:
      - ./html:/var/www/html
   #       /\       We are matching this 
    labels:
      - "nautical-backup.group=wp"
      - "nautical-backup.override-source-dir=html"

db: 
   # Rest of config
    volumes:
      - ./db:/var/lib/mysql
   #      /\       We are matching this 
    labels:
      - "nautical-backup.group=wp"
      - "nautical-backup.override-source-dir=db"

Option 2

Change the mounted directory in for your containers to match their names

Of course you will need to move the data over from the old folder to the new one.

# Example command (not exact)
cd <where your source dir is>
mv db wordpress-db
mv html wordpress-blog

 wordpress:
   # Rest of config
    container_name: wordpress-blog
    volumes:
      - ./wordpress-blog:/var/www/html  # <--- Change this
    #           /\       Now we match the "container_name"
    labels:
      - "nautical-backup.group=wp"

db: 
   # Rest of config
    container_name: wordpress-db
    volumes:
      - ./wordpress-db:/var/lib/mysql  # <--- Change this
    #           /\       Now we match the "container_name"
    labels:
      - "nautical-backup.group=wp"

Hopefully this help :)

[mixpc]

Thank you for your feedback. Firstly I have tried Option 1 overrides so...

nautical backup docker-compose.yml
version: '3'
services:
  nautical-backup:
    image: minituff/nautical-backup:latest
    container_name: nautical-backup
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /home/user/docker:/app/source
      - /home/user/dockerbackup:/app/destination
    environment: # Optional variables
      - LOG_LEVEL=DEBUG
      - CRON_SCHEDULE=21 9 * * *
      - SKIP_CONTAINERS=nautical-backup
    restart: unless-stopped

and

wordpress docker-compose.yml
version: '3.1'

services:

  wordpress:
    image: wordpress:6.5
    container_name: wordpress-blog
    restart: unless-stopped
    ports:
      - 80:80
    environment:
      WORDPRESS_DB_HOST: xxxx
      WORDPRESS_DB_USER: xxxx
      WORDPRESS_DB_PASSWORD: xxxx
      WORDPRESS_DB_NAME: xxxx
    volumes:
      - ./html:/var/www/html
    labels:
      - "nautical-backup.group=wp"
      - "nautical-backup.override-source-dir=html"

  db:
    image: mysql:8.0
    container_name: wordpress-db
    restart: always
    environment:
      MYSQL_DATABASE: xxxx
      MYSQL_USER: mxxxx
      MYSQL_PASSWORD: xxxx
      MYSQL_RANDOM_ROOT_PASSWORD: xxxx
    volumes:
      - ./db:/var/lib/mysql
    labels:
      - "nautical-backup.group=wp"
      - "nautical-backup.override-source-dir=db"

The group wp is detected:
INFO: Backing up group: wp
However, log output seems to indicate nautical backup is not detecting folders db or html and so not backing up. Longer log below:

DEBUG: REPORT_FILE: true
INFO: Nautical Backup Version: 2.3.3
DEBUG: Built for the platform: linux/arm64
DEBUG: Perparing enviornment variables...
DEBUG: Found defaults.env
DEBUG: CRON_SCHEDULE_ENABLED: true
DEBUG: BACKUP_ON_START: false
DEBUG: USE_DEST_DATE_FOLDER: false
DEBUG: DEST_DATE_PATH_FORMAT: date/container
DEBUG: DEST_DATE_FORMAT: %Y-%m-%d
DEBUG: USE_DEFAULT_RSYNC_ARGS: true
DEBUG: REQUIRE_LABEL: false
DEBUG: REPORT_FILE_LOG_LEVEL: INFO
DEBUG: REPORT_FILE_ON_BACKUP_ONLY: true
DEBUG: KEEP_SRC_DIR_NAME: true
DEBUG: EXIT_AFTER_INIT: false
DEBUG: LOG_RSYNC_COMMANDS: false
DEBUG: RUN_ONCE: false
DEBUG: SOURCE_LOCATION: /app/source
DEBUG: DEST_LOCATION: /app/destination
DEBUG: TEST_MODE: -1
DEBUG: HTTP_REST_API_ENABLED: true
DEBUG: HTTP_REST_API_USERNAME: admin
DEBUG: HTTP_REST_API_PASSWORD: xxxx
DEBUG: ADDITIONAL_FOLDERS_WHEN: before
DEBUG: NAUTICAL_DB_PATH: /config
DEBUG: NAUTICAL_DB_NAME: nautical-db.json
DEBUG: SELF_CONTAINER_ID: xxxxxxxxxxxx
DEBUG: Installing CRON schedule: 21 9 * * *
DEBUG: Verifying source directory '/app/source'...
DEBUG: Verifying destination directory '/app/destination'...
INFO: Initializing database at '/config/nautical-db.json'...
INFO: Creating Database at path: '/config/nautical-db.json'...
INFO: Database initialized at '/config/nautical-db.json'...
DEBUG: Installing nautical backup script...
INFO: API listening on port 8069...
INFO: Initialization complete. Awaiting CRON schedule: 21 9 * * *
DEBUG: Verifying source directory '/app/source'...
DEBUG: Verifying destination directory '/app/destination'...
INFO: Starting backup...
INFO: Processing 10 containers...
DEBUG: Containers: nautical-backup, wordpress-blog, wordpress-db [...]
DEBUG: Skipping nautical-backup based on name
INFO: Backing up group: wp
DEBUG: wordpress-db - Source directory '/app/source/db' does not exist. Skipping
DEBUG: wordpress-blog - Source directory '/app/source/html' does not exist. Skipping
DEBUG: Container wordpress-db was not stopped. No need to start.
DEBUG: Container wordpress was not stopped. No need to start.

So it wasn't backed up,source folder
/home/user/docker/wordpress
is not showing under
/home/user/dockerbackup
as
/home/user/dockerbackup/wordpress

I will try Option 2 later today and report. Thank you for your time and effort.

Note: folder db is owned by 999:root and folder html by www-data:www-data just in case

[mixpc]

Second test with Option 2

Same nautical backup docke-compose.yml

wordpress docker-compose.yml
version: '3.1'

services:

  wordpress:
    image: wordpress:6.5
    container_name: wordpress-blog
    restart: unless-stopped
    ports:
      - 80:80
    environment:
      WORDPRESS_DB_HOST: xxxx
      WORDPRESS_DB_USER: xxxx
      WORDPRESS_DB_PASSWORD: xxxx
      WORDPRESS_DB_NAME: xxxx
    volumes:
      - ./wordpress-blog:/var/www/html
    labels:
      - "nautical-backup.group=wp"

  db:
    image: mysql:8.0
    container_name: wordpress-db
    restart: always
    environment:
      MYSQL_DATABASE: xxxx
      MYSQL_USER: mxxxx
      MYSQL_PASSWORD: xxxx
      MYSQL_RANDOM_ROOT_PASSWORD: xxxx
    volumes:
      - ./wordpress-db:/var/lib/mysql
    labels:
      - "nautical-backup.group=wp"

user@server:~/docker/wordpress $ ls -alh
-rw-r--r--  1 user   user   1.4K Jul 30 22:13 docker-compose.yml
drwxr-xr-x  6 www-data www-data 4.0K Jul 30 09:16 wordpress-blog
drwxr-xr-x  9      999 root     4.0K Jul 30 20:22 wordpress-db

This time all containers stopped before having nautical-backup run, so only nautical backup and the 2 wodpress containers were running. Log:

DEBUG: REPORT_FILE: true
INFO: Nautical Backup Version: 2.3.3
DEBUG: Built for the platform: linux/arm64
DEBUG: Perparing enviornment variables...
DEBUG: Found defaults.env
DEBUG: CRON_SCHEDULE_ENABLED: true
DEBUG: BACKUP_ON_START: false
DEBUG: USE_DEST_DATE_FOLDER: false
DEBUG: DEST_DATE_PATH_FORMAT: date/container
DEBUG: DEST_DATE_FORMAT: %Y-%m-%d
DEBUG: USE_DEFAULT_RSYNC_ARGS: true
DEBUG: REQUIRE_LABEL: false
DEBUG: REPORT_FILE_LOG_LEVEL: INFO
DEBUG: REPORT_FILE_ON_BACKUP_ONLY: true
DEBUG: KEEP_SRC_DIR_NAME: true
DEBUG: EXIT_AFTER_INIT: false
DEBUG: LOG_RSYNC_COMMANDS: false
DEBUG: RUN_ONCE: false
DEBUG: SOURCE_LOCATION: /app/source
DEBUG: DEST_LOCATION: /app/destination
DEBUG: TEST_MODE: -1
DEBUG: HTTP_REST_API_ENABLED: true
DEBUG: HTTP_REST_API_USERNAME: admin
DEBUG: HTTP_REST_API_PASSWORD: xxxx
DEBUG: ADDITIONAL_FOLDERS_WHEN: before
DEBUG: NAUTICAL_DB_PATH: /config
DEBUG: NAUTICAL_DB_NAME: nautical-db.json
DEBUG: SELF_CONTAINER_ID: xxxxxxx
DEBUG: Installing CRON schedule: 17 22 * * *
DEBUG: Verifying source directory '/app/source'...
DEBUG: Verifying destination directory '/app/destination'...
INFO: Initializing database at '/config/nautical-db.json'...
INFO: Creating Database at path: '/config/nautical-db.json'...
INFO: Database initialized at '/config/nautical-db.json'...
DEBUG: Installing nautical backup script...
INFO: API listening on port 8069...
INFO: Initialization complete. Awaiting CRON schedule: 17 22 * * *
DEBUG: Verifying source directory '/app/source'...
DEBUG: Verifying destination directory '/app/destination'...
INFO: Starting backup...
INFO: Processing 3 containers...
DEBUG: Containers: nautical-backup, wordpress-blog, wordpress-db
DEBUG: Skipping nautical-backup based on name
INFO: Backing up group: wp
DEBUG: wordpress-db - Source directory '/app/source/wordpress-db' does not exist. Skipping
DEBUG: wordpress-blog - Source directory '/app/source/wordpress-blog' does not exist. Skipping
DEBUG: Container wordpress-db was not stopped. No need to start.
DEBUG: Container wordpress-blog was not stopped. No need to start.
DEBUG: Containers completed:
DEBUG: Containers skipped: wordpress-db, wordpress-blog, nautical-backup
INFO: Success. 0 containers backed up! 3 skipped.

So wordpress-db and wordpress-blog were not backed up. I may run more tests, as needed.

[Minituff]

Hey @mixpc, thanks for testing this out.

For both options 1 and 2, did you mount the directories inside the Nautical contianer too?

# docker-compose.yml for nautical backup
version: '3'
services:
  nautical-backup:
    image: minituff/nautical-backup:latest
    container_name: nautical-backup
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /home/user/docker:/app/source
      - /home/user/dockerbackup:/app/destination
      - ./wordpress-blog:/app/source/wordpress-blog    # Option 2
      - ./wordpress-db:/app/source/wordpress-db        # Option 2
    environment: # Optional variables
      - LOG_LEVEL=DEBUG
      - CRON_SCHEDULE=05 21 * * *
      - SKIP_CONTAINERS=nautical-backup
    restart: unless-stopped

What is the absolute path of ./wordpress-blog and ./wordpress-db? I can give you a better recommendation with that information.

[mixpc]

Ah, no. Here are the full paths:

/home/user/docker/wordpress/wordpress-blog
/home/user/docker/wordpress/wordpress-db

All docker containers are located within
/home/user/docker

[Minituff]

Ohhh, I think I get it, its because you have wordpress-blog and wordpress-db nested inside the wordpress folder.

Can you move them up to the root of the /home/user/docker folder like this:

/home/user/docker/wordpress-blog
/home/user/docker/wordpress-db

OR

You could update the mounts in the Nautical config like this:

# docker-compose.yml for nautical backup
version: '3'
services:
  nautical-backup:
    image: minituff/nautical-backup:latest
    container_name: nautical-backup
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /home/user/docker:/app/source
      - /home/user/dockerbackup:/app/destination
      - /home/user/docker/wordpress/wordpress-blog:/app/source/wordpress-blog    # HERE
      - /home/user/docker/wordpress/wordpress-db:/app/source/wordpress-db        # AND HERE
    environment: # Optional variables
      - LOG_LEVEL=DEBUG
      - CRON_SCHEDULE=05 21 * * *
      - SKIP_CONTAINERS=nautical-backup
    restart: unless-stopped