Does the mapper work with systemd cgroup driver

[cf-sewe]

I am running Ubuntu 22 with Docker 24.0.6.
Trying to get /dev/zfs access to work in the container (for ZFS filesystem management).

Docker Info:

Server:
 Server Version: 24.0.6
 Cgroup Driver: systemd
 Cgroup Version: 2
 Swarm: active
  Is Manager: true
  Managers: 3
  Nodes: 3
...

I have installed the plugin:

ID             NAME                                   DESCRIPTION                                  ENABLED
072779dcdbe8   ndouba/device-mapping-manager:latest   A device mapping plugin for swarm clusters   true

But when I run my container, I am unable to get the device access to work.

docker run -it --rm   --cap-add=SYS_ADMIN   --mount='type=bind,source=/dev/zfs,destination=/dev/zfs'   --entrypoint /bin/sh
/ # ls -al /dev/zfs
crw-rw-rw-    1 root     root       10, 249 Oct  3 15:01 /dev/zfs
/ # zfs list
Failed to initialize the libzfs library.

My questions are:

1. Is systemd cgroup driver supported by DMM?
2. Where can I find the logs of the DMM plugin? I dont see the plugin container running nor any relevant logs in the docker logs.

I am glad for any hints getting this to work!
Other workarounds mentioned in the moby#1244 issue (like running a docker run --device=XXX in a Swarm service) I unfortunately cannot use.

[cf-sewe]

When I run the dmm via the docker-compose.yml provided in this repo, I managed to get it working:

2023/10/04 09:54:52 The cgroup version for process 13638 is: 2
2023/10/04 09:54:52 Checking mounts for process 13638
2023/10/04 09:54:52 1d232abf4e805b71be288817b5210ec1795b84614c4d76782e58a15ecdd17e41/13638 requested a volume mount for /dev/zfs at /dev/zfs
2023/10/04 09:54:52 The cgroup path for process 13638 is at /host/sys/fs/cgroup/system.slice/docker-1d232abf4e805b71be288817b5210ec1795b84614c4d76782e58a15ecdd17e41.scope
2023/10/04 09:54:52 Found device: /dev/zfs c 10:249
2023/10/04 09:54:52 Adding device rule for process 13638 at /host/sys/fs/cgroup/system.slice/docker-1d232abf4e805b71be288817b5210ec1795b84614c4d76782e58a15ecdd17e41.scope

So I assume the plugin ndouba/device-mapping-manager:latest is not needed and it is sufficient to execute the dmm manually on each Docker Swarm node?

Does someone have an idea regarding an issue, that the mount is not working inside the container?
Could be related to the docker-default AppArmor profile used on Ubuntu machines.

root@5d5b8540a337:/# zfs create rpool/ENCROOT/cplace/test2
cannot mount 'rpool/ENCROOT/cplace/test2': Permission denied
filesystem successfully created, but not mounted

[allfro]

It sounds like App Armor is getting in the way. Try running the dmm binary natively on the system. Don’t use the docker plugin.