Add GH Action workflow to update downstream repos#73
Merged
joverlee521 merged 6 commits intomainfrom Apr 20, 2026
Merged
Conversation
victorlin
reviewed
Apr 14, 2026
joverlee521
changed the title
joverlee521
force-pushed
the
update-vendored
branch
4 times, most recently
from
e3df43a to
ada3160
Compare
This comment was marked as outdated.
This comment was marked as outdated.
joverlee521
added a commit
that referenced
this pull request
Apr 15, 2026
We should only be keeping a single copy of the vendored repo in each downstream repo, so deduplicate the matrix by repo. In cases where there are multiple copies, we are prioritizing the `nextstrain/shared` remote since that is the newer version. This is prompted by the error in the workflow when avian-flu had two paths to update. <#73 (comment)>
victorlin
reviewed
Apr 15, 2026
Searches the Nextstrain GitHub org to find repos that have the `.gitrepo` file with the nextstrain/shared remote to create a matrix of repos to potentially update. Installs and uses `git subrepo` to pull in the latest changes with the `--force` flag to avoid merge conflicts to due rebasing in the downstream repo. If there are changes pulled down, then `git subrepo` will create a single commit. If there is a single commit, then push up the changes to a branch and create or update the PR in the downstream repo. Nothing happens if there were no changes and workflow exits with error if it encounters more than one commit.
I was unable to get the search/code API to work with the 'OR' syntax so just added a separate query for nextstrain/ingest and concatenated the two arrays. Deduplicated the final array to guard against potential overlap.
We should only be keeping a single copy of the vendored repo in each downstream repo, so deduplicate the matrix by repo. In cases where there are multiple copies, we are prioritizing the `nextstrain/shared` remote since that is the newer version. This is prompted by the error in the workflow when avian-flu had two paths to update. <#73 (comment)>
2 tasks
joverlee521
added a commit
that referenced
this pull request
Apr 16, 2026
Prevent the workflow from overwriting additional work on the remote branch. This allows us to add manual updates directly to the PRs without having to worry about them being overwritten by the next update run. Prompted by feedback from @victorlin in review <#73 (comment)>
joverlee521
added a commit
that referenced
this pull request
Apr 16, 2026
Do not allow `git subrepo` to overwrite local changes in automatic updates. This will increase failures in the automatic updates but will flag merge conflicts with local changes in downstream repos for manual fixes. Prompted by feedback from @jameshadfield and @victorlin in review <#73 (comment)>
joverlee521
force-pushed
the
update-vendored
branch
from
ab5399f to
2d0fca4
Compare
GH API calls return 30 items by default¹ so use `--paginate` + `--slurp` to return all pages and get all items.² ¹ <https://docs.github.com/en/rest/using-the-rest-api/using-pagination-in-the-rest-api?apiVersion=2026-03-10> ² <https://cli.github.com/manual/gh_api>
joverlee521
added a commit
that referenced
this pull request
Apr 16, 2026
Prevent the workflow from overwriting additional work on the remote branch. This allows us to add manual updates directly to the PRs without having to worry about them being overwritten by the next update run. Prompted by feedback from @victorlin in review <#73 (comment)>
joverlee521
added a commit
that referenced
this pull request
Apr 16, 2026
Do not allow `git subrepo` to overwrite local changes in automatic updates. This will increase failures in the automatic updates but will flag merge conflicts with local changes in downstream repos for manual fixes. Prompted by feedback from @jameshadfield and @victorlin in review <#73 (comment)>
joverlee521
force-pushed
the
update-vendored
branch
from
2d0fca4 to
cdb0903
Compare
joverlee521
added a commit
that referenced
this pull request
Apr 16, 2026
Do not allow `git subrepo` to overwrite local changes in automatic updates. This will increase failures in the automatic updates but will flag merge conflicts with local changes in downstream repos for manual fixes. Note this requires us to fetch all git history for the downstream repos since `git subrepo pull` now needs to validate the parent commit. Prompted by feedback from @jameshadfield and @victorlin in review <#73 (comment)>
joverlee521
force-pushed
the
update-vendored
branch
from
cdb0903 to
2b38fbc
Compare
Do not allow `git subrepo` to overwrite local changes in automatic updates. This will increase failures in the automatic updates but will flag merge conflicts with local changes in downstream repos for manual fixes. Note this requires us to fetch all git history for the downstream repos since `git subrepo pull` now needs to validate the parent commit. Prompted by feedback from @jameshadfield and @victorlin in review <#73 (comment)>
joverlee521
force-pushed
the
update-vendored
branch
from
2b38fbc to
0e170d1
Compare
Try to fetch remote branch and add subrepo changes to prevent automatic updates from overwriting manual updates. This also allows us to skip unnecessary pushes to the PR if the remote branch is already up-to-date.
joverlee521
force-pushed
the
update-vendored
branch
from
0e170d1 to
23acdb4
Compare
victorlin
approved these changes
Apr 16, 2026
Contributor
Author
|
If there's no other feedback, I'll merge this Monday. The merge of this PR should automatically trigger a workflow run to update downstream repos. Since this is a new change, the existing PRs in downstream repos should get updated. Then I'll go through those PRs to fix and merge them, which should finally allow us to get back to nextstrain/public#39 |
Contributor
Author
|
Workflow failed after merge with 404 error, likely due to ongoing GitHub incident. I'll fix up pathogen workflows first, then come back and re-run the workflow later today. |
Contributor
Author
|
Just noting for posterity that I decided to merge the pathogen PRs that were created by the test runs so that we could unblock nextstrain/public#39. I'm not going to re-run the failed workflow since it will only open a bunch of PRs that only has changes from this PR which don't affect the downstream repos. I can just keep an eye on the next PR in this repo. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of proposed changes
Searches the Nextstrain GitHub org to find repos that have the
.gitrepofile with the nextstrain/shared or nextstrain/ingest remote to create a matrix of repos to potentially update. Installs and usesgit subrepoto pull in the latest changes with the--forceflag to avoid merge conflicts to due rebasing in the downstream repo. If there are changes pulled down, thengit subrepowill create a single commit. If there is a single commit, then push up the changes to a branch and create or update the PR in the downstream repo. Nothing happens if there were no changes and workflow exits with error if it encounters more than one commit.Related issue(s)
Prompted by nextstrain/public#39 (comment)
Checklist
If adding a script, add an entry for it in the README.