Proposal for support for runtime composefs validation

[alexlarsson]

In the automotive sphere, we're interested in having some level of runtime validation. For the rootfs we already get this from bootc using composefs. However, if we're also using containers in /var/lib/containers, those are not protected by this. However, containers/storage already (optionally) supports composefs, so we should be able to do something similar for podman.

Here is what this MR, and the related changes in container-libs does:

• Add option --security-opt signature=[check,require]. If this is set, then the signature for the manifest is validated at podman run time, which allows us to trust the manifest data, like the annotations.
• Add support in the overlay driver to pass in a set of expected composefs fs-verity digests for the layers. When these are used, it will validate at mount time that all the layers have an expected digest, and ensures the overlayfs mount is mounted with verity=require (which requires all files in the mount to be backed by a file with a fs-verity digest from the composefs blob).
• If --security-opt verity=enforce is passed to podman run, then podman looks at the per-layer annotations in the image manifest (which we ideally trust due to a signature) for io.containers.composefs.digest keys, where you can give a list of allowable fs-verity digests. These are then forwarded to the overlayfs driver which validates this at mount time.

With the above, we can have a pretty robust validation of the container at runtime. There are some weak points:

• An attacker could replace the podman binary or some dependency to not do the checks
• An attacker could change the signature policy
• You have to start podman with the right arguments

All of these are fixable in a controlled environment. For example, if you have a read-only /usr and /etc like bootc, and you ship a quadlet file in /usr that has the right arguments, then you can have some trust in that the right code is running, and you can do "podman pull" to get a new image version, keeping this trust.

I have an example signed image with annotations at https://quay.io/repository/alexl42/centos-verity. See the description there for the public key used. With I can run a validated image:

# podman run --security-opt signature=require --security-opt verity=enforce -ti quay.io/alexl42/centos-verity:10 echo foo

You can also see how it works with an unsigned and no-verity image:

# podman run --security-opt signature=require --security-opt verity=enforce -ti quay.io/centos/centos:9 echo foo
Error: manifest signature verification failed: No signature verification policy found for image docker:quay.io/centos/centos:9
# podman run --security-opt verity=enforce -ti quay.io/centos/centos:9 echo foo
Error: verity enforcement: layer 0 missing io.containers.composefs.digest annotation

Or if i tweak the composefs blob:

# podman run --security-opt signature=require --security-opt verity=enforce -ti quay.io/alexl42/centos-verity:10 echo foo
Error: mounting storage for container 45b43778da976b9997ace9eda98b17ac4c87c4e2f820c59a07f9c4f7429e884d: composefs blob /mnt/data/containers/storage/overlay/433ca53ed3a7208fc5834c1ee92839b01ed5ff7fccef4e80c0437992c4806267/composefs-data/composefs.blob has fs-verity digest "sha256:f86904b89e9d4b5c5436a657647c3d0309be742c5f9460b786027fb2968ea167", not in allowed list

I don't expect this PR to just be necessarily merged as is, but I'd like to bring this up for discussion. We'd like a feature like this in automotive, is that reasonable? Is the approach reasonable? Is the interface reasonable?

Discussion points:

• Do we maybe also want config options for making these checks default?
• Is using layer annotations right? They are sort of a pain to set.

Some minor notes:

• We need to support multiple for forward compatibility if we support e.g. other fs-verity hashes or composefs versions.
• This depends on composefs: Properly add verity info to generated dump file container-libs#816
• In the work on OCI sealing in the next gen composefs-rs there is some similar annotations for this, although those are in a separate artifact. We should ensure that these don't conflict, and are properly versioned for future use of this in podman.

[pypingou]

Do we maybe also want config options for making these checks default?

I think we would like this yes, especially as we know that not everyone will use quadlet files in /usr