Use OPEN_TREE_NAMESPACE

[EmilyShepherd]

Linux recently added the OPEN_TREE_NAMESPACE option to the open_tree() syscall which does two things:

• Combines unshare(CLONE_NEWNS) and pivot_root() into a single syscall rather than two
• Only clones submounts within the specified newrootfs, reducing the need to recursively umount the oldrootfs during container setup.

The benefit of the first point is unclear for crun as it looks like its unshare and pivot root logic sits in separate areas, however the second point could potentially reduce the amount of setup crun needs to do as well taking the load of the kernel in needlessly copying lots of mounts that are destined to be umounted immediately.

afaik OPEN_TREE_NAMESPACE is only in Linux 7.0 so it is quite early to support.

[giuseppe]

I am playing with it, but I am struggling to find a way to get it working with user namespaces.

It looks like a chicken&egg problem because I can't create the mounts before entering the new mount namespace (the user namespace doesn't own the current mount namespace) and I cannot create them after I join the user namespace because the mount_too_revealing check kicks in as there are no other proc mounts in the current mount namespace.

@brauner am I missing something? :-)

[giuseppe]

I could workaround it with an intermediate mount namespace (CLONE_NEWNS) and replace only pivot_root, but we lose some of the benefits on the way:

                                         avg      min      max       ok
unshare(NEWNS)+pivot_root:            0.343ms   0.147ms   0.735ms   100/100
unshare(NEWNS) only:                  0.069ms   0.025ms   0.132ms   100/100
unshare(NEWNS)+OPEN_TREE_NS:          0.153ms   0.055ms   0.274ms   100/100
open_tree(OPEN_TREE_NS) only:         0.044ms   0.010ms   0.090ms   100/100

Speedup vs unshare+pivot: 7.8x
Speedup vs unshare only:  1.6x
Speedup vs unshare+OTNS:  3.5x

and if I create a bunch of mounts on the host:

                                         avg      min      max       ok
unshare(NEWNS)+pivot_root:            0.917ms   0.588ms   2.403ms   100/100
unshare(NEWNS) only:                  0.549ms   0.352ms   1.380ms   100/100
unshare(NEWNS)+OPEN_TREE_NS:          0.911ms   0.564ms   3.109ms   100/100
open_tree(OPEN_TREE_NS) only:         0.043ms   0.027ms   0.085ms   100/100

Speedup vs unshare+pivot: 21.5x
Speedup vs unshare only:  12.9x
Speedup vs unshare+OTNS:  21.3x

[brauner]

I am playing with it, but I am struggling to find a way to get it working with user namespaces.

It looks like a chicken&egg problem because I can't create the mounts before entering the new mount namespace (the user namespace doesn't own the current mount namespace) and I cannot create them after I join the user namespace because the mount_too_revealing check kicks in as there are no other proc mounts in the current mount namespace.

@brauner am I missing something? :-)

Unshare the userns and then point it to the mount that you want to have as the roots mount. For procfs I merged support to mount with PID only but that's for the current cycle.

[giuseppe]

thanks, yes the only problem I am facing is with filesystems that go through mount_too_revealing since there are no such mounts in the new mount namespace

[EmilyShepherd]

@giuseppe I see - if I am understanding this right, would it work if the process looked like:

1. First mount procfs sysfs into the container rootfs directory from host namespaces
2. Unshare the userns + OPEN_TREE_NAMESPACE to rootfs
3. Remount / mask the procfs and sysfs as usual

Apologies if I have misunderstood - the exact behaviour of mount_too_revealing is not familiar to me and I am presuming OPEN_TREE_NAMESPACE will copy any mounts currently in the target directory into the new mount namespace.

Failing that, it is nice to see support for this when user namespaces are not in use at least 🎉

[giuseppe]

That won't work because the namespaces (e.g. PID and network) won't be owned by the correct user namespace, so I think we really need to create the user namespace as first step. For procfs and sysfs, AFAICS, we need some help from the kernel to allow opening them when we don't own the mount namespace.

From my understanding it would look like:

• create userns & other namespaces except mount.
• pre-create the mounts we need in the target rootfs.
• create the new mount namespace.
• setns(new_mount_namespace), the pivot_root happens as part of this step.
• move the mounts we created to the right place in the rootfs.

There are other cases where the optimization is not usable, e.g. hooks that expect to find the rootfs at a specific path and still access the host file system, so we need to keep the two mechanisms anyway, unless there will be a breaking change in the OCI runtime specs and how hooks work.