Updated build for external dependencies (mods-latest) by mathgeekcoder · Pull Request #2989 · ERGO-Code/HiGHS

mathgeekcoder · 2026-04-23T00:04:05Z

Note: this PR is for mods-latest branch

Includes significant improvement to how we use vcpkg
Currently uses vcpkg for highspy build - I want to test this on CI/CD
NOTE: sdist packages shouldn't use vcpkg unless the user already has it installed

Other changes

Added RPATH for linux/osx to find library
Support for vcpkg BLAS on linux (OpenBLAS) and osx (no vcpkg needed, uses accelerate)
Fixed type redefinition warning on metis
Fixed template-depth compile warning on C (not C++) files
Minor code cleanup (e.g., removed debug messages)
Added thread safety for dependency loading
Added highspy._core.getExtrasLoadStatus for debugging the library load
Added highspy COMPONENT to avoid pointless copy of header files
Tested with BUILD_SHARED_LIBS=ON/OFF and BUILD_SHARED_EXTRAS_LIB=ON/OFF and HIPO=ON/OFF (windows and linux)
Tested highspy and highspy-extras (windows and linux)

* Added RPATH for linux/osx to find library * Support for vcpkg BLAS on linux (OpenBLAS) and osx (accelerate, not tested - might not actually work as is) * Fixed type redefinition warning on metis * Fixed `template-depth` compile warning on C (not C++) files * Minor code cleanup (e.g., removed debug messages) * Added thread safety for dependency loading * Added `highspy._core.getExtrasLoadStatus` for debugging the library load * Added highspy COMPONENT to avoid pointless copy of header files * Tested with BUILD_SHARED_LIBS=ON/OFF and BUILD_SHARED_EXTRAS_LIB=ON/OFF and HIPO=ON/OFF (windows and linux) * Tested highspy and highspy-extras (windows and linux)

* Entirely optional for regular build (only uses if vcpkg installed) * Currently used by highspy build (can be easily reverted) * Uses github commit SHA for specific openblas version (reproducibility) * highspy-extras build seems faster on windows (fresh 20mins -> 5mins) and 30s with cached openblas * Should also work when creating highs as a vcpkg package

…ssues with highspy_extras build.

codecov · 2026-04-23T01:20:21Z

Codecov Report

❌ Patch coverage is 83.72093% with 7 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (mods-latest@9aaf141). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
check/TestHighsExternalDeps.cpp	82.35%	3 Missing ⚠️
highs/HighsExternalDeps.cpp	50.00%	3 Missing ⚠️
highs/HighsExternalDeps.h	80.00%	1 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff               @@
##             mods-latest    #2989   +/-   ##
==============================================
  Coverage               ?   75.07%           
==============================================
  Files                  ?      376           
  Lines                  ?    97122           
  Branches               ?    15521           
==============================================
  Hits                   ?    72914           
  Misses                 ?    23932           
  Partials               ?      276

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…uld be github.workspace) * Fixed issue with HIPO=OFF and copyright linking * Hopefully fixed issue with vcpkg for highspy-extras

Since cibuildwheel on linux uses a container, need to point to host system's vcpkg instead.

* Updated sdist workflow to build highspy_extras in ./dist * Updated vcpkg.cmake to use VCPKG_INSTALLATION_ROOT if VCPKG_ROOT not set

… fine

* fixed formatting issue * fixed sdist build for highspy-extras

…extras * changed test-python-win workflow to point to correct tests directory * Updated vcpkg to treat VCPKG_ROOT="" as undefined * Removed vcpkg support for i686 and musllinux, uses BUILD_OPENBLAS instead

* Fixed path issue with nuget tests

* Static link libhighs_extras into libhighs for julia build * Fix venv paths for macos python * Fix path warning for win pipx * Removed vcpkg for workflow that specifically shouldn't use it * Added verbose logging for debugging * Fixed openblas path issue

* Fixed julia build * Fixed macos issue with @loader_path

…uild times * Removed vcpkg from highspy build (not extras) * Fixed issue with openblas_set_num_threads not included in build

…s of the new recommended approach

* Added some more tests for better coverage

* Formatting HighsExternalDeps * Attempt at making highspy_extras build faster by building all on same VM (per image) and caching the vcpkg installs - should save hours of compute effort.

* Fix issue with vcpkg linux not using cache (missing zip/unzip) * Fixed linker warnings with highs_extras on windows * Improve hipo workflow build times

…e build times for highspy-extras

… openblas build

…kg does not build due to libcurl

mathgeekcoder added 3 commits April 22, 2026 11:45

Fix issues building highspy-extras for i686

0743f90

mathgeekcoder mentioned this pull request Apr 23, 2026

Updated build for external dependencies #2987

Closed

mathgeekcoder had a problem deploying to testpypi April 23, 2026 00:26 — with GitHub Actions Failure

Reverted some HIPO checks for FAST_BUILD=OFF. Attempted to fix some i…

1d570de

…ssues with highspy_extras build.

mathgeekcoder had a problem deploying to testpypi April 23, 2026 01:18 — with GitHub Actions Failure

mathgeekcoder marked this pull request as draft April 23, 2026 04:44

jajhall assigned mathgeekcoder Apr 23, 2026

galabovaa self-requested a review April 23, 2026 09:57

galabovaa approved these changes Apr 23, 2026

View reviewed changes

* Fixed yml linting issues (there is no runner.workspace defined, sho…

c7d7aa0

…uld be github.workspace) * Fixed issue with HIPO=OFF and copyright linking * Hopefully fixed issue with vcpkg for highspy-extras

mathgeekcoder had a problem deploying to testpypi April 23, 2026 17:24 — with GitHub Actions Failure

Simplified vcpkg, since all github runners already have it installed.

0c04b9c

Since cibuildwheel on linux uses a container, need to point to host system's vcpkg instead.

mathgeekcoder had a problem deploying to testpypi April 23, 2026 19:16 — with GitHub Actions Failure

mathgeekcoder added 3 commits April 23, 2026 12:34

* Update the vcpkg git repo if needed

77b15dd

* Updated sdist workflow to build highspy_extras in ./dist * Updated vcpkg.cmake to use VCPKG_INSTALLATION_ROOT if VCPKG_ROOT not set

Don't need the absolutely latest vcpkg version, the runner version is…

c41d85a

… fine

Just get latest version instead of specific

c7212eb

mathgeekcoder had a problem deploying to testpypi April 23, 2026 20:16 — with GitHub Actions Failure

* fixed verification CI/CD on windows

df99795

* fixed formatting issue * fixed sdist build for highspy-extras

mathgeekcoder had a problem deploying to testpypi April 23, 2026 22:29 — with GitHub Actions Failure

* changed sdist windows workflow to install both highspy and highspy_…

1ab29e1

…extras * changed test-python-win workflow to point to correct tests directory * Updated vcpkg to treat VCPKG_ROOT="" as undefined * Removed vcpkg support for i686 and musllinux, uses BUILD_OPENBLAS instead

mathgeekcoder had a problem deploying to testpypi April 23, 2026 23:27 — with GitHub Actions Failure

mathgeekcoder had a problem deploying to testpypi April 23, 2026 23:29 — with GitHub Actions Failure

* Added debug logging to test sdist on windows

b71c8ec

* Fixed path issue with nuget tests

mathgeekcoder had a problem deploying to testpypi April 24, 2026 00:12 — with GitHub Actions Failure

mathgeekcoder had a problem deploying to testpypi April 24, 2026 00:18 — with GitHub Actions Failure

Workflow fixes:

88b5f4c

* Static link libhighs_extras into libhighs for julia build * Fix venv paths for macos python * Fix path warning for win pipx * Removed vcpkg for workflow that specifically shouldn't use it * Added verbose logging for debugging * Fixed openblas path issue

mathgeekcoder had a problem deploying to testpypi April 24, 2026 16:56 — with GitHub Actions Failure

mathgeekcoder added 2 commits April 24, 2026 09:58

* Added simple dependency unit test for coverage

88b6c1d

* Fixed julia build * Fixed macos issue with @loader_path

Fixed missing HighsExternalDeps::tryLoad when HIPO=OFF

13ed9a3

mathgeekcoder had a problem deploying to testpypi April 24, 2026 17:25 — with GitHub Actions Failure

mathgeekcoder had a problem deploying to testpypi April 24, 2026 17:27 — with GitHub Actions Failure

* Added vcpkg caching across workflows, should dramatically improve b…

e041d4c

…uild times * Removed vcpkg from highspy build (not extras) * Fixed issue with openblas_set_num_threads not included in build

mathgeekcoder had a problem deploying to testpypi April 24, 2026 18:42 — with GitHub Actions Failure

mathgeekcoder added 3 commits April 24, 2026 12:02

Removed vcpkg github cache. Approach was deprecated and I'm suspiciou…

70187f2

…s of the new recommended approach

* Fixed issue with macos openblas

506f2d2

* Added some more tests for better coverage

Include missing header definition for openblas_set_num_threads

8fb4c55

mathgeekcoder had a problem deploying to testpypi April 24, 2026 19:30 — with GitHub Actions Failure

mathgeekcoder had a problem deploying to testpypi April 24, 2026 19:36 — with GitHub Actions Failure

* Fixed unit_test link with highs_extras (when needed)

b164007

* Formatting HighsExternalDeps * Attempt at making highspy_extras build faster by building all on same VM (per image) and caching the vcpkg installs - should save hours of compute effort.

mathgeekcoder had a problem deploying to testpypi April 24, 2026 20:18 — with GitHub Actions Failure

mathgeekcoder added 3 commits April 24, 2026 13:46

* Avoid building both dynamic and static versions of vcpkg openblas

1389c70

* Fix issue with vcpkg linux not using cache (missing zip/unzip) * Fixed linker warnings with highs_extras on windows * Improve hipo workflow build times

Fix zip/unzip issue on linux containers

cbdd366

Ignore zip for linux, just causing issues for now

4fb4608

mathgeekcoder had a problem deploying to testpypi April 24, 2026 21:09 — with GitHub Actions Failure

mathgeekcoder had a problem deploying to testpypi April 24, 2026 22:12 — with GitHub Actions Failure

mathgeekcoder added 2 commits April 24, 2026 16:38

Another attempt at caching the openblas build to significantly improv…

03ca1b3

…e build times for highspy-extras

Fixed missing package for musllinux_aarch64

dc0b8e7

mathgeekcoder had a problem deploying to testpypi April 24, 2026 23:52 — with GitHub Actions Failure

Specify host target to be windows-static to avoid an additional 10min…

2120e29

… openblas build

mathgeekcoder had a problem deploying to testpypi April 25, 2026 00:22 — with GitHub Actions Failure

Hopefully a fix (but feels like a hack), but on musllinux_aarch64 vcp…

6374885

…kg does not build due to libcurl

mathgeekcoder had a problem deploying to testpypi April 25, 2026 00:43 — with GitHub Actions Failure

mathgeekcoder had a problem deploying to testpypi April 25, 2026 00:48 — with GitHub Actions Failure

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updated build for external dependencies (mods-latest)#2989

Updated build for external dependencies (mods-latest)#2989
mathgeekcoder wants to merge 27 commits intoERGO-Code:mods-latestfrom
mathgeekcoder:mods-latest-luke

mathgeekcoder commented Apr 23, 2026

Uh oh!

codecov Bot commented Apr 23, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

mathgeekcoder commented Apr 23, 2026

Note: this PR is for mods-latest branch

Other changes

Uh oh!

codecov Bot commented Apr 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov Bot commented Apr 23, 2026 •

edited

Loading