refactor: replace source modification scripts with TOML overlays

[Tonisal-byte]

Replace source-modification scripts with TOML archive overlays

Migrates 10 components from bespoke modify_source.sh scripts to declarative azldev archive overlays. This keeps source sanitization in component configuration while preserving the existing filtered source contents and hashes.

• apache-commons-compress
• espeak-ng
• exfatprogs
• firefox
• gdal
• kf6-karchive
• libabigail
• libkml
• rubygem-pdf-reader
• yara

[Copilot]

Pull request overview

Migrates ten components from custom source-repacking scripts to declarative azldev archive overlays.

Changes:

• Adds archive-scoped fixture removal and source hash tracking.
• Preserves required dependent test adjustments.
• Deletes obsolete modify_source.sh scripts.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
base/comps/apache-commons-compress/apache-commons-compress.comp.toml	Adds archive fixture removals.
base/comps/apache-commons-compress/modify_source.sh	Removes repacking script.
base/comps/espeak-ng/espeak-ng.comp.toml	Removes flagged demo payload.
base/comps/espeak-ng/modify_source.sh	Removes repacking script.
base/comps/exfatprogs/exfatprogs.comp.toml	Removes unused test tree.
base/comps/exfatprogs/modify_source.sh	Removes repacking script.
base/comps/firefox/firefox.comp.toml	Removes flagged test fixtures.
base/comps/firefox/modify_source.sh	Removes repacking script.
base/comps/gdal/gdal.comp.toml	Removes nested archive fixture.
base/comps/gdal/modify_source.sh	Removes repacking script.
base/comps/kf6-karchive/kf6-karchive.comp.toml	Removes flagged archive tests.
base/comps/kf6-karchive/modify_source.sh	Removes repacking script.
base/comps/libabigail/libabigail.comp.toml	Removes PR30329 fixtures.
base/comps/libabigail/modify_source.sh	Removes repacking script.
base/comps/libkml/libkml.comp.toml	Removes KMZ fixture and dependent test.
base/comps/libkml/modify_source.sh	Removes repacking script.
base/comps/rubygem-pdf-reader/rubygem-pdf-reader.comp.toml	Removes PDFs and skips affected tests.
base/comps/rubygem-pdf-reader/modify_source.sh	Removes repacking script.
base/comps/yara/yara.comp.toml	Removes flagged corpora and PE test entries.
base/comps/yara/modify_source.sh	Removes repacking script.

Comments suppressed due to low confidence (2)

base/comps/firefox/firefox.comp.toml:169

• The repository is pinned at .azldev-version:1 to azldev 661c69b…, which predates support for origin.type = "overlay" (added by azure-linux-dev-tools #276). With the pinned tool this source declaration is unsupported, and the checked-in schema also rejects it, so this component cannot be prepared. Please update the azldev pin to a revision containing #276, regenerate the schema/docs as needed, then refresh this component's lock and rendered spec sources (which still contain the old hash).

origin = { type = "overlay" }

base/comps/gdal/gdal.comp.toml:29

• The repository is pinned at .azldev-version:1 to azldev 661c69b…, which predates support for origin.type = "overlay" (added by azure-linux-dev-tools #276). With the pinned tool this source declaration is unsupported, and the checked-in schema also rejects it, so this component cannot be prepared. Please update the azldev pin to a revision containing #276, regenerate the schema/docs as needed, then refresh this component's lock and rendered spec sources (which still contain the old hash).

origin = { type = "overlay" }

[Copilot]

Pull request overview

Copilot reviewed 40 out of 50 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (5)

base/comps/firefox/firefox.comp.toml:150

• This section still says the authoritative list is in modify_source.sh and refers to REMOVE_PATHS, but this PR deletes that script. Make the overlays themselves authoritative so future version bumps do not follow a nonexistent workflow.

# Archive overlays strip the scanner-flagged fixtures from upstream Source0
# during source preparation, avoiding a separate modified-source tarball.
#
# When bumping the firefox version (or changing REMOVE_PATHS):
#   1. Update the overlay paths below if upstream fixture paths change.

base/comps/gdal/gdal.comp.toml:23

• The overlay exists because the nested archive trips signing-pipeline scanning, not merely to reduce source size. Since overlay descriptions are used in diagnostics and must explain why the divergence exists, retain the scanner-specific rationale stated above.

description = "Remove large test fixture to reduce source size"

base/comps/firefox/firefox.comp.toml:153

• Adding these archive overlays changes Firefox's source payload, but this component uses manual release calculation and azl_release is still 5. The component's own lines 12-14 require this counter to be bumped for a new overlay; otherwise the rendered RPM remains Release 6 and collides with the existing build. Increment azl_release to 6, then refresh the lock and rendered output.

[[components.firefox.overlays]]

base/comps/libabigail/libabigail.comp.toml:18

• The companion patch still documents an “AZL-repacked source tarball” generated by base/comps/libabigail/modify_source.sh, which this PR deletes; the rendered patch carries the same stale instructions. Update the patch header to identify this archive overlay, then re-render so both copies stay consistent.

# `make check` green. This avoids a separate modified-source tarball.

specs/a/apache-commons-compress/sources:1

• The PR description says the existing filtered-source hashes are preserved, but this SHA512—and the hashes for all other migrated archives—changes. If archive-overlay repacking is intentionally byte-different, update the description; if byte-identical hashes are an acceptance criterion, regenerate the archives accordingly.

[Copilot]

Pull request overview

Copilot reviewed 41 out of 51 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

base/comps/gdal/gdal.comp.toml:23

• The overlay rationale above and replace-reason say this file is removed because the nested archive trips signing-pipeline scanning, but this description reports an unrelated size optimization. Since azldev surfaces this text in overlay diagnostics, keep the actual reason here.

description = "Remove large test fixture to reduce source size"

base/comps/firefox/firefox.comp.toml:147

• The migration deletes modify_source.sh, but this maintenance guidance still calls that deleted file and REMOVE_PATHS authoritative. Point maintainers at the overlays instead so the next Firefox bump does not follow a dead workflow.

# Archive overlays strip the scanner-flagged fixtures from upstream Source0

base/comps/libabigail/libabigail.comp.toml:25

• The companion tests-drop-PR30329-fixture-entries.patch still documents base/comps/libabigail/modify_source.sh as the mechanism, but this PR deletes that script in favor of this archive overlay. Update both the source patch header and its rendered copy so the migration does not leave a dangling reference.

[[components.libabigail.overlays]]
type = "file-remove"
file = "tests/data/test-abidiff-exit/PR30329/**"
archive = "libabigail-2.9.tar.xz"
description = "Remove PR30329 fixture directory (two libsqlite3.so.0.8.6.debug files flagged as packer_high_entropy:eod by AZL signing-pipeline AV scanner)"

[Copilot]

Pull request overview

Copilot reviewed 42 out of 52 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

base/comps/firefox/firefox.comp.toml:150

• The preceding guidance still names the deleted modify_source.sh/REMOVE_PATHS as authoritative. That makes the new version-bump instructions point to a file that no longer exists; make the overlays below the documented source of truth instead.

# Archive overlays strip the scanner-flagged fixtures from upstream Source0
# during source preparation, avoiding a separate modified-source tarball.
#
# When bumping the firefox version (or changing REMOVE_PATHS):

base/comps/gdal/gdal.comp.toml:23

• This description gives source-size reduction as the reason, while the surrounding rationale and replace-reason say the overlay exists because the nested archive is rejected by malware scanning. Use the actual reason so overlay diagnostics remain accurate.

description = "Remove large test fixture to reduce source size"

base/comps/libabigail/libabigail.comp.toml:39

• The companion patch header still says modify_source.sh creates an AZL-repacked tarball, but this migration deletes that script and now uses the archive overlay. Update the source patch header and re-render its generated copy so the retained rationale does not point maintainers to a nonexistent file.

replace-reason = "The PR30329 test fixture directory contains debuginfo files flagged by signing-pipeline scanning; the archive overlay removes it and a companion patch removes its tests."

[Tonisal-byte]

/azp run

[azure-pipelines]

Azure Pipelines:
3 pipeline(s) were filtered out due to trigger conditions.