Skip to content

feat(providers): use vx-org/mirrors as stable download source for all providers#858

Open
loonghao wants to merge 5 commits into
mainfrom
feat/use-vx-org-mirrors
Open

feat(providers): use vx-org/mirrors as stable download source for all providers#858
loonghao wants to merge 5 commits into
mainfrom
feat/use-vx-org-mirrors

Conversation

@loonghao
Copy link
Copy Markdown
Owner

@loonghao loonghao commented May 6, 2026

Summary

Switch all vx providers to use vx-org/mirrors as the primary version list and download source.

Why

Some upstream sources are unreliable or ephemeral:

  • Third-party mirrors (johnvansickle.com, evermeet.cx) go offline
  • GitHub releases from small projects can be deleted
  • Old versions disappear when upstream does rolling releases
  • Official CDNs (nodejs.org, go.dev) are stable but we now have permanent local mirrors too

What Changed

vx-org/mirrors (new repo)

  • 130 tools configured with permanent archive sync
  • All historical versions preserved forever (keep_versions: 0)
  • Syncs nightly via GitHub Actions
  • Supports: github_release, btbn_ffmpeg, multi_version, nodejs_org, go_dev, rust_rustup, hashicorp, adoptium, dotnet_microsoft

Provider changes (64 files)

All fetch_versions now point to vx-org/mirrors tags instead of upstream:

# Before
fetch_versions = make_fetch_versions("BurntSushi", "ripgrep")

# After  
fetch_versions = make_fetch_versions("vx-org", "mirrors", tag_prefix = "ripgrep-")

Custom download_url functions updated to download from mirrors releases.

Verification

New tools

loonghao added 3 commits May 6, 2026 16:42
All providers now fetch version lists from vx-org/mirrors GitHub Releases
instead of upstream sources directly. Download URLs for providers with
custom download_url are also updated to use vx-org/mirrors assets.

Affected: 64 providers across all ecosystems
Static check: 137/137 pass
make_fetch_versions() now accepts an optional tag_prefix parameter that
is passed through to the Rust runtime's version resolver.

This allows providers to fetch versions from mirror repos where tags
follow the pattern '{tool}-{version}' instead of 'v{version}'.

Example:
  fetch_versions = make_fetch_versions("vx-org", "mirrors", tag_prefix = "ripgrep-")
  # tag 'ripgrep-15.1.0' → version '15.1.0'

The Rust runtime (versions.rs) already supports tag_prefix in the
github_versions descriptor - this change exposes it at the Starlark level.
… on mirrors

mirrors sync is still in progress - switching fetch_versions to mirrors
caused 'No versions found' errors for all providers not yet synced.

Strategy change:
- fetch_versions: restored to original upstream sources (immediate fix)
- download_url: still points to vx-org/mirrors (stability benefit)

Once mirrors sync is complete, fetch_versions can be switched to mirrors
in a follow-up PR.
@codecov
Copy link
Copy Markdown

codecov Bot commented May 6, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 54.23%. Comparing base (363d986) to head (a4cbf91).
⚠️ Report is 71 commits behind head on main.

❌ Your project check has failed because the head coverage (54.23%) is below the target coverage (80.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #858      +/-   ##
==========================================
- Coverage   54.26%   54.23%   -0.03%     
==========================================
  Files         377      382       +5     
  Lines       66876    67670     +794     
==========================================
+ Hits        36289    36703     +414     
- Misses      30587    30967     +380     
Flag Coverage Δ
unittests 54.23% <ø> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 70 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

loonghao added 2 commits May 7, 2026 01:12
Mirror vx-org/mirrors is not fully synced yet.
Revert all provider.star to use original GitHub URLs to fix CI 404 errors.

Keep github.star tag_prefix support for future use.
Zig download URL format changed at version 0.14.1:
- 0.14.0 and earlier: zig-{os}-{arch}-{version}.{ext}
- 0.14.1 and later: zig-{arch}-{os}-{version}.{ext}

Added _zig_use_new_url_format() to detect format based on version
and use the correct URL format in download_url().

Fixes 'error decoding response body' CI failure (404 on old URL format).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant