docs: Documentation incorrect or misleading?

[Ziris85]

Hello! At the bottom of the linked docs page where it instructs the user to run all of those commands on every cluster member to create images and backups storage volumes, this is of course not possible. Since Ceph is shared storage, once the storage volumes are created on one server, they will be visible on all, and subsequent attempts at creating them (again) will fail:

Error: Volume by that name already exists

Additionally, attempting to use the existing ones in the lxd config is only partially successful:

:~# lxc config get storage.images_volume
remote-fs/images
:~# lxc config set storage.backups_volume=remote-fs/backups
Error: Failed validation of "storage.backups_volume": Storage volume "remote-fs/backups" isn't empty

Should these steps be instructing to create unique names for these volumes for each cluster member, perhaps? Such that there is no name conflict? If so, can I suggest that the docs suggest this by using something like lxc storage volume create remote-fs host1-backups?

Thank you!

---

Reported from: https://documentation.ubuntu.com/microcloud/default/how-to/initialize/

[roosterfish]

Hi @Ziris85 and thanks for reporting this.
The docs do say this bit at the bottom:

Repeat these steps on each cluster member.

However it might not be clear that the lxc config set command needs to be executed from each cluster member (if you don't set --target).

I have pushed a fix in #1418.

Regarding your comment

Since Ceph is shared storage, once the storage volumes are created on one server, they will be visible on all

This is why we explicitly instruct to use CephFS as the volume can be used on multiple cluster members simultaneously. So there only ever needs to be one backups and one images vol.
The error you are seeing is likely caused by configuring storage.backups_volume for one cluster member, storing something on the vol, and the trying to configure it for another member which will fail as there are already contents on the disk.
To prevent this configure both config keys (for all members) right after provisioning the MicroCloud or use new volumes.

[Ziris85]

Thanks for the quick reply @roosterfish ! Yes, I think that does help clarify things a bit, though the docs actually instruct at both the beginning and end of that block to run ALL of those commands on every cluster member (including creating the volumes):

So if the point is to create the two volumes on one member, and then configure LXD on each member to use them, maybe that should be reorganized?

Also, for the error, there was very little time in between me running the lxc config set commands between cluster members on my new deployment - if I'm being generous, maybe 30 seconds. Maybe it wrote data because I tried to create the volume again? But anyway, I can try removing and re-adding the volume and setting the config later today to see if it's any better.

[Ziris85]

Following up on the error, I was never able to get it working from the CLI. Even if I delete the volume and then run this copy/paste command series in my terminal (thus ensuring that the time between volume creation and lxd config set is reduced to a second or two), it still fails due to the volume somehow gaining data in that span of time:

lxc storage volume create remote-fs backups
lxc config set storage.images_volume=remote-fs/images
lxc config set storage.backups_volume=remote-fs/backups
for h in host2 host3 host4; do 
echo "Working on ${h}...";
lxc config set storage.images_volume=remote-fs/images --target ${h};
lxc config set storage.backups_volume=remote-fs/backups --target ${h};
done

The only way I was able to get it to work was via the web UI, by first unsetting the value on all members, and then setting it on all cluster members at the same time.

[roosterfish]

What version of LXD are you using? Can you please share the output of snap list. I was trying to reproduce this yesterday with a two member MicroCloud using LXD 6.8 and I was able to set the config key without issues across the cluster members.