Fix autotools source-origin migration under cmake-re and add test matrix#57
Open
Orphis wants to merge 6 commits into
Open
Fix autotools source-origin migration under cmake-re and add test matrix#57Orphis wants to merge 6 commits into
Orphis wants to merge 6 commits into
Conversation
CHANGELOG:
- HERMETIC_DEFER_NATIVE_ROOTED_FIND_PACKAGE_FOR: forward find_package() calls to cmake's
native implementation rooted at the HFC install prefix instead of serving from the target cache
- HERMETIC_<content-name>_FIND_PACKAGE_EXTRA_CODE: inject arbitrary cmake code into the
provider when find_package() is handled for a dependency (e.g. to set <Pkg>_VERSION)
- Proxy toolchains embed dependency state fingerprints for transitive invalidation when an
upstream dependency changes
- HERMETIC_<content-name>_FIND_PACKAGE_EXTRA_CODE: inject arbitrary cmake code into the
provider when find_package() is handled for a dependency (e.g. to set <Pkg>_VERSION)
- Proxy toolchains embed dependency fingerprints for transitive invalidation when an upstream
dependency changes
- Transient content aliases now resolve correctly through find_package() in sub-builds
- Alias deduplication: multiple aliases resolving to the same canonical content produce a
single registration in the proxy toolchain (e.g. HERMETIC_FIND_PACKAGES "jemalloc;jemallocAlt")
Change-Id: I3a656d3f93f1e4ae8abcfa877f65db238aa2a59a
A focused, parallelisable matrix that exercises how HFC handles changing a dependency's FetchContent origin in place (same build folder + central source cache) and reports which source ends up compiled in. HFC keys its on-disk clone on <content_name>-<8 chars of GIT_TAG|URL_HASH>-src and deliberately omits the origin URL. Each scenario rebuilds a trivial library whose installed variant id doubles as the executable's exit code, so a migration is classified CLEAN (new sources), STALE (old sources reused) or BROKEN (build failed) from observed behaviour. - hfc_source_migration_test.cpp: every scenario is its own BOOST_DATA_TEST_CASE_F parameterised over the build drivers (/_0 = native cmake, /_1 = cmake-re), for both cmake and autotools dependencies: git tag/repo/archive migrations, a moving branch, and .git removal. Clean migrations assert CLEAN; same-cache-key edge cases assert their exact current outcome so nothing silently passes. The .git-removal cases locate the clone on disk (find_samplelib_clone, robust for native cmake and the cmake-re mirror copy) and hard-require the removal, so the precondition can never silently no-op. - test_project_templates/autotools_samplelib: dependency-free fake autotools scaffold (configure + Makefile.rules) for the autotools rows. - The expectations encode current behaviour, including two real limitations the follow-up commits fix: the same-repo tag bump is STALE under cmake-re, and a fresh-build re-populate after .git removal is BROKEN under native cmake (for both cmake and autotools deps). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Change-Id: I9f9b26dbef61018adb4409451c01be7cba8f6908
In-source build systems (autotools/openssl) build a dependency inside its cmake-re mirror worktree, so the dependency's source tree itself lives in that worktree. The worktree identity is derived from the proxy-toolchain hash (the ABI key) and is otherwise revision-independent, so two revisions of the same origin share one worktree. When the pinned revision changes in place, cmake-re restores the previously-built revision's cached source snapshot over the freshly-fetched sources and the dependency is silently rebuilt/installed from stale sources (e.g. bumping GIT_TAG to a newer commit keeps serving the old build). Out-of-source cmake builds read sources from a revision-specific directory and recompile from there, so they are unaffected and keep their incremental mirror reuse. For in-source builds we now fold the content revision (GIT_TAG or URL_HASH) into the proxy toolchain so each revision gets its own mirror worktree. - hfc_make_available_single.cmake: append the in-source revision key to the proxy toolchain extension for autotools/openssl dependencies. - hfc_source_migration_test.cpp: at_git_tag_to_git_tag now expects CLEAN on both drivers (previously STALE under cmake-re). - check_autotools_tag_update.cpp: online regression for the same-repo tag bump using a real autotools dependency, confirming the fix holds with the genuine toolchain (libtool/config.status) and not just the fake scaffold. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Change-Id: I83807508a004d3601f49f3c0f07014fba355eeea
When a git dependency's source-cache clone loses its .git directory (e.g.
manually removed, or a partially-written clone) and the build folder is wiped so
no FetchContent stamps remain, populating fails: FetchContent's git clone
refuses the non-empty, non-git destination, and even after emptying it the
surviving subbuild stamps make ExternalProject skip the re-clone and run its
git-update step in the empty dir ("fatal: not a git repository").
Detect this corrupt state before populating a git dependency: if the source dir
exists, is non-empty and is not a git repository, clear it and invalidate the
populate stamps so ExternalProject re-clones from scratch. This covers both
out-of-source (cmake) deps and in-source build systems (autotools/openssl),
whose first populate clones into the same source cache dir, so the corruption
and the fix are identical. Valid clones (including ones with intentional local
modifications) stay on the existing trust-the-stamps path and are untouched.
- hfc_populate_project.cmake: clear + invalidate a non-git source cache dir
before FetchContent_Populate.
- hfc_source_migration_test.cpp: the .git-removal fresh-build cases (cmake and
autotools deps) now expect CLEAN on both drivers (previously BROKEN for native
cmake).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Change-Id: I22b8237f4f881a466b4493e99013cf9310b24dbf
Orphis
force-pushed
the
feature/source_migration_fixes
branch
from
fe0f84a to
3ff5cc2
Compare
pysco68
requested changes
Jun 29, 2026
pysco68
left a comment
pysco68
left a comment
Contributor
There was a problem hiding this comment.
LGTM other than the presence of the stale comment
Lambourl
requested changes
Jun 30, 2026
Lambourl
left a comment
Lambourl
left a comment
Contributor
There was a problem hiding this comment.
Please follow these guidelines for your commits :
- https://github.com/tipi-build/hfc/blob/main/.github/PULL_REQUEST_TEMPLATE/HFC-RELEASING.md
You can make multiple commits—that's fine—but you must follow the naming convention.
But we can't merge it because it depends on version v1.0.25 and for now, given the feedback we've received on this matter, we will not be releasing version v1.0.25 before mid-July. (https://github.com/tipi-build/specs-cmake-re/issues/463#issuecomment-4798211087)
If this issue is a priority, could you please rebase your work on version 1.0.24?
Contributor
I propose we use the moment to adapt our guidelines, lets keep the guideline that we want a Change-Id but lets get rid of the versioning in commits message (and if you agree @Orphis @pysco68 @Lambourl we can push to main a change of the guidelines and also fix them so that they appear on pull-request creation) |
…igin revert)
When an autotools dependency's origin is switched and later reverted in place
under cmake-re, the build failed with:
make: *** No targets specified and no makefile found. Stop.
PROJECT_BINARY_DIR is a symlink in cmake-re mode. On the revert the caller
decides the dependency needs configuring (dep_need_configure = ON, the configure
marker is not found through the symlink, still pointing at the fork's mirror),
then the mirror preparation re-points the symlink back to the original origin's
mirror -- which still holds the first build's configure marker. The redundant
inner `if(NOT EXISTS already_configured_file)` guard in hfc_autootols_configure
then re-checked the marker through the switched symlink, found that stale marker
and skipped ./configure, leaving the freshly-repopulated sources with no
Makefile. Removing the guard (the caller already gates on dep_need_configure)
makes configure always run when invoked.
- hfc_autotools_restore_or_configure.cmake: drop the redundant inner guard.
- check_autotools_origin_switch.cpp: online regression with a real autotools
dependency, switching origin orig -> fork -> orig in place. Discriminating:
FAILS without the fix (reproduces "No targets" on the revert) and PASSES with
it. Reproduction needs the genuine toolchain, the revert step, a stable
cmake-re host worktree, and TIPI_CACHE_CONSUME_ONLY=ON.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Change-Id: I99ad41f1b147fadc2c498b305337244a9a0f5304
Relocating the central source cache via HERMETIC_FETCHCONTENT_SOURCE_CACHE_DIR is HFC's core "relocate the download folder" capability, but nothing in the test suite exercised it. Add a focused matrix scenario: build a dependency with the default cache, then reconfigure/rebuild the SAME origin with the source cache relocated to a new directory, and assert both that the build still works (the binary reflects the dependency) and that the dependency's clone actually moved to the relocated directory. - hfc_source_migration_test.cpp: write_consumer gains an optional cache-dir argument; run_relocate_scenario + git_relocate_source_cache / at_git_relocate_source_cache cases (cmake and autotools, native cmake and cmake-re). resolve_real_source locates the relocated dir under the mirror in cmake-re mode, where a project-relative path resolves against the mirror copy. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Change-Id: I413153dda8fda4997eb3103152c12c8b04038f29
Orphis
force-pushed
the
feature/source_migration_fixes
branch
from
3ff5cc2 to
b07ecd5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CHANGELOG
under cmake-re (the revision is now folded into the proxy toolchain so each
gets its own mirror worktree).
place under cmake-re ("make: No targets specified and no makefile found").
folder wiped (the non-git clone dir is now cleared and re-cloned).
changes, moving branch, .git removal; cmake and cmake-re; cmake and autotools
deps) plus a HERMETIC_FETCHCONTENT_SOURCE_CACHE_DIR relocation scenario.
revision bump and the origin revert.
Change-Id: I9f9b26dbef61018adb4409451c01be7cba8f6908