GNU-All-permissive-Copying-License

[sergiomb2]

is here where we need fill in the License tag ?

Using detector: trivy
Traceback (most recent call last):
File "/usr/bin/go_vendor_license", line 8, in <module>
sys.exit(main())
^^^^^^
File "/usr/lib/python3.11/site-packages/go_vendor_tools/cli/go_vendor_license.py", line 607, in main
install_command(args)
File "/usr/lib/python3.11/site-packages/go_vendor_tools/cli/go_vendor_license.py", line 529, in install_command
license_data: LicenseData = detector.detect(directory)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/go_vendor_tools/license_detection/trivy.py", line 139, in detect
return TrivyLicenseData(
^^^^^^^^^^^^^^^^^
File "<string>", line 9, in __init__
File "/usr/lib/python3.11/site-packages/go_vendor_tools/license_detection/base.py", line 149, in __post_init__
combine_licenses(*self.license_set) if self.license_map else None
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/go_vendor_tools/licensing.py", line 28, in combine_licenses
return simplify_license(
^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/go_vendor_tools/licensing.py", line 53, in simplify_license
parsed = licensing.parse(str(expression), validate=validate, strict=strict)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/license_expression/__init__.py", line 560, in parse
self.validate_license_keys(expression)
File "/usr/lib/python3.11/site-packages/license_expression/__init__.py", line 467, in validate_license_keys
raise ExpressionError(msg)
license_expression.ExpressionError: Unknown license key(s): GNU-All-permissive-Copying-License
error: Bad exit status from /var/tmp/rpm-tmp.fs55Ke (%install)
Please fill in the License tag!
Bad exit status from /var/tmp/rpm-tmp.fs55Ke (%install)

[pombredanne]

@sergiomb2 this is the place. My hunch is that trivy does not use proper SPDX licenses?
Can you explain how to reproduce this error end to end?
I get you are using https://pypi.org/project/go-vendor-tools/ ... but then what?

[sergiomb2]

Hi,
on Fedora 38 / 39 or 40 under an chroot , unpack https://github.com/moby/buildkit/archive/refs/tags/v0.13.2.tar.gz and cd
buildkit-0.13.2
I run go_vendor_license --config go-vendor-tools.toml install --destdir /builddir/build/BUILDROOT/buildkit-0.13.2-1.fc40.x86_64 --install-directory /usr/share/licenses/buildkit --filelist licenses.list

but I think maybe it misses one package , since in my shell seems I don't have the problem

[sergiomb2]

cat go-vendor-tools.toml

[archive]

[licensing]
[[licensing.licenses]]
path = "vendor/github.com/spdx/tools-golang/LICENSE.code"
sha256sum = "e914fb1f3927226e04b0438e0b541b3c6e3c65de4d64aa8f5cdaa803f05448fd"
expression = "MIT"

[[licensing.licenses]]
path = "vendor/gopkg.in/yaml.v3/LICENSE"
sha256sum = "d18f6323b71b0b768bb5e9616e36da390fbd39369a81807cca352de4e4e6aa0b"
expression = "Apache-2.0"
[[licensing.licenses]]
path = "vendor/kernel.org/pub/linux/libs/security/libcap/psx/License"
sha256sum = "279fa656c62857d42952b86a0789b98669ca1cb4b324a8d9e91397c174af4c14"
expression = "GPL-2.0-only"
[[licensing.licenses]]
path = "vendor/kernel.org/pub/linux/libs/security/libcap/cap/License"
sha256sum = "18eb30b662ed0abeef6ad6ded90a99b30332418d2f71e63dcb4646bbb23b9acc"
expression = "GPL-2.0-only"

[sergiomb2]

I reproduced the error in ipython

import license_expression
licensing = license_expression.get_spdx_licensing()
expression = 'GNU-All-permissive-Copying-License'
licensing.parse(str(expression), validate=True, strict=True)

ExpressionError: Unknown license key(s): GNU-All-permissive-Copying-License

[pombredanne]

So if we unwind the twine ball:

• GNU-All-permissive-Copying-License is not a known license id at SPDX, and the go-vendor-tools library is requesting a strict SPDX validation
• GNU-All-permissive-Copying-License is reported by the Go tool trivy
• trivy https://github.com/aquasecurity/trivy is reusing the obsolete, out-of-date https://github.com/google/licenseclassifier
• https://github.com/google/licenseclassifier reports a proper FSFAP license https://scancode-licensedb.aboutcode.org/fsf-ap.html as GNU-All-permissive-Copying-License which is not a valid SPDX license id.

A solution would be to fix trivy and to fix google/licenseclassifier, but this is IMHO a long winding road with not much bright light at the end.

Another solution would be to fix go-vendor-tools to use scancode-toolkit instead for a proper and correct license detection.

There are many many other things to unpack as the license detection by https://github.com/google/licenseclassifier in https://github.com/moby/buildkit/archive/refs/tags/v0.13.2.tar.gz seems incomplete and incorrect, and the other Go tool https://github.com/google/licensecheck/ which is used otherwise for the official go.pkg.dev is also incorrect/wrong.

For instance https://pkg.go.dev/kernel.org/pub/linux/libs/security/libcap/psx is NOT under the GPL but under a choice of BSD-3-Clause OR GPL-2.0-only.

If you are packaging for Fedora you must report the license choice and not pick one. (This is the Fedora way)

On the side, I noted a few smaller license detection issue in ScanCode and I will file and fix them there.

It looks like Go is a somewhat of a mess wrt. to licensing clarity :]

[sergiomb2]

many thanks for all the info , I will report soon to Fedora golang SIG , yes we are packaging go packages in Fedora

[gotmax23]

Thanks for your insight, @pombredanne.

Another solution would be to fix go-vendor-tools to use scancode-toolkit instead for a proper and correct license detection.

For the record, support for scancode in go-vendor-tools is blocked on aboutcode-org/scancode-toolkit#3720 and/or me rewritting my WIP patch to use the scancode python API instead.

In any case, I plan to make the following changes in the go-vendor-tools package:

1. Fix error handling for invalid licenses. The code should print a clear error message with a list of filenames mapped to invalid licenses instead of just emitting a license_expression traceback so it's clear where the problem/bug lies.
2. Consider changing the default license detector backend back to askalono. In Fedora, we patch the package to use an updated version of the SPDX data.

For instance https://pkg.go.dev/kernel.org/pub/linux/libs/security/libcap/psx is NOT under the GPL but under a choice of BSD-3-Clause OR GPL-2.0-only.

That license was not determined by trivy. Licenses in go-vendor-tools.toml that @sergiomb2 pasted in are manually specified by the user when the go-vendor-tools license detector cannot find them. Packagers need to ensure that they are inputting the proper license expression when using that feature.

It looks like Go is a somewhat of a mess wrt. to licensing clarity :]

For the most part, it's not terrible, but there's some packages that do atypical things.

---

As for the original issue, would it be possible to add a GNU-All-permissive-Copying-License -> FSFAP alias to the scancode license data that license-expression uses? Otherwise, feel free to close this as not a license-expression bug.