Skip to content

[GLUTEN-12599][VL] Fall back to actions/cache when Apache Stash misses#12623

Draft
Smallfu666 wants to merge 3 commits into
apache:mainfrom
Smallfu666:contrib/issue-12599-stash-fallback
Draft

[GLUTEN-12599][VL] Fall back to actions/cache when Apache Stash misses#12623
Smallfu666 wants to merge 3 commits into
apache:mainfrom
Smallfu666:contrib/issue-12599-stash-fallback

Conversation

@Smallfu666

@Smallfu666 Smallfu666 commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

What changes are proposed in this pull request?

Follow-up to #12610 (merged as 86df856). Apache Stash only supports exact-key restore and has no restore-keys mechanism. When hashFiles('ep/build-velox/src/**') changes for a PR branch, the exact Stash key misses and the native build starts cold. This PR adds a fallback to the legacy actions/cache/restore with the original restore-keys prefix, which can still pull the latest warm cache seeded on main.

Design

Stash exact key hit → native build (fast)
Stash miss → actions/cache/restore with restore-keys → native build (still warm)

Each consumer restore now works in two steps:

  1. stash/restore first (exact ccache-<variant>-${{ hashFiles('ep/build-velox/src/**') }} key);
  2. if stash-hit is not 'true', fall back to actions/cache/restore with the pre-migration restore-keys, which pulls the latest cache seeded on main.

The existing actions/cache/save steps are kept, so both the Stash and actions/cache paths stay populated.

Scope

  • Modified (3 files, 57 additions): velox_backend_arm.yml, velox_backend_enhanced.yml, velox_backend_x86.yml
  • Not modified: velox_backend_cache.yml (producer), velox_nightly.yml, build_bundle_package.yml, flink.yml, velox_backend_ansi.yml

The cache producer, nightly and bundle workflows, and self-contained caches (flink, ansi) are untouched — they keep the legacy actions/cache method as-is.

How was this patch tested?

Static validation

  • YAML parsing passes for all three modified workflow files.
  • All 6 fallback restore keys and their restore-keys prefixes have been verified against the producer's actions/cache/save entries in velox_backend_cache.yml.
  • No trailing whitespace, no unrelated formatting changes, no modifications outside the three scope files.

Forced-miss CI validation: PASS

A temporary commit appended -forced-miss-validation to the six Stash restore keys to force Stash to miss. All three Velox Backend workflows completed successfully:

Workflow Stash result Fallback executed Cache hit rate Build time
x86 (centos7 x86_64) stash-hit=false ✅ actions/cache restored (prefix: ccache-centos7-release-default) 95.36 % ~2.5 min
ARM (centos8 aarch64) stash-hit=false ✅ actions/cache restored (prefix: ccache-centos8-release-default-ARM64, 443 MB, ~3.2 s) 94.49 % ~2 min
Enhanced (centos7 x86_64) stash-hit=false ✅ actions/cache restored (prefix: ccache-enhanced-centos7-release-default) 92.52 % ~27 min

Key evidence from each workflow:

  • The Get Ccache from Apache Stash step logged stash-hit=false — the key suffix -forced-miss-validation was present.
  • The Get Ccache from actions/cache (fallback on Stash miss) step executed and restored a warm cache via the restore-keys prefix.
  • All native builds reported "Successfully built Velox from Source." with ccache hit rates above 92%.
  • The forced-miss commit has been reverted; the final branch contains only the clean hybrid diff.

Normal path (Stash hit)

The normal Stash-first path was validated and merged in #12610. On an earlier revision of that PR, build-native-lib-centos-8 (ARM64) showed:

  • stash-hit=true
  • ccache Hits: 101618 / 107649 (94.40%)
  • Native build ~1h → 2m35s

Relationship to other PRs

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude

Smallfu666 and others added 2 commits July 24, 2026 22:49
Per review: Stash has no restore-keys, so a changed ep/build-velox/src
tree misses its exact key and cold-builds. Keep the Stash restore first
on the x86/arm/enhanced consumers, and fall back to actions/cache/restore
(with restore-keys, warm from main) when stash-hit is false; the
actions/cache save is retained so both paths stay populated. Producers,
nightly and bundle keep the legacy actions/cache method unchanged. Also
add a default branch to the gh/jq arch case per review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Appends -forced-miss-validation to all 6 consumer Stash restore keys
so the fallback actions/cache/restore is exercised during CI.
Will be reverted before the PR review.
@github-actions github-actions Bot added the INFRA label Jul 24, 2026
@zhouyuan

Copy link
Copy Markdown
Member

@Smallfu666 thanks a lot for the contribution! I have included this commit in (#12615) to speed up a bit on the rebase work

@Smallfu666

Copy link
Copy Markdown
Contributor Author

@zhouyuan Thanks a lot for the review and for including the commit in #12615 — glad it was useful.

I’m still new to Gluten and currently learning the project through the Velox CI/cache work and the Spark–Velox correctness side. I’d also like to understand the project’s current direction better. Are there particular Velox backend areas or ongoing priorities where more contributor help would be especially useful?

@zhouyuan

Copy link
Copy Markdown
Member

@Smallfu666
Welcome to the team! We’ve put together a great list of good first issues to help you get started:: https://github.com/apache/gluten/issues?q=is%3Aissue%20state%3Aopen%20label%3A%22good%20first%20issue%22

If you're looking for a specific place to dive in, the ANSI support issue (#10134) is a good choice. You can start by reading the Spark code for those functions and then implementing them in the Velox backend.
#10134

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants