Review/reconcile upgrade by lbeckman314 · Pull Request #1438 · calypr/funnel

lbeckman314 · 2026-06-22T02:51:19Z

Important

All credit in this PR belongs to @nss10!

Overview 🌀

This draft PR is an copy/clone of PR #1410 with the following steps to allow it to run GitHub Actions checks (credit: @nss10):

➜ gh pr checkout 1410
remote: Enumerating objects: 160, done.
remote: Counting objects: 100% (92/92), done.
remote: Compressing objects: 100% (28/28), done.
remote: Total 160 (delta 75), reused 72 (delta 64), pack-reused 68 (from 2)
Receiving objects: 100% (160/160), 66.96 KiB | 1.12 MiB/s, done.
Resolving deltas: 100% (103/103), completed with 19 local objects.
From github.com:uc-cdis/funnel
   598528df..2fbbb4d1  chore/reconcile_upgrade -> uc-cdis/chore/reconcile_upgrade
branch 'chore/reconcile_upgrade' set up to track 'uc-cdis/chore/reconcile_upgrade'.
Switched to a new branch 'chore/reconcile_upgrade'

➜ git show --summary --oneline
2fbbb4d1 (HEAD -> chore/reconcile_upgrade, uc-cdis/chore/reconcile_upgrade) Merge branch 'develop' into chore/reconcile_upgrade
 delete mode 100644 .github/workflows/integration.yaml
 delete mode 100644 .github/workflows/k8s.yaml
 delete mode 100644 .github/workflows/s3.yaml

➜ git checkout -b review/reconcile_upgrade
Switched to a new branch 'review/reconcile_upgrade'

➜ git push
Enumerating objects: 264, done.
Counting objects: 100% (215/215), done.
Delta compression using up to 10 threads
Compressing objects: 100% (72/72), done.
Writing objects: 100% (164/164), 27.01 KiB | 1.93 MiB/s, done.
Total 164 (delta 125), reused 128 (delta 89), pack-reused 0 (from 0)
remote: Resolving deltas: 100% (125/125), completed with 27 local objects.
remote:
remote: Create a pull request for 'review/reconcile_upgrade' on GitHub by visiting:
remote:      https://github.com/calypr/funnel/pull/new/review/reconcile_upgrade
remote:
remote: GitHub found 53 vulnerabilities on calypr/funnel's default branch (2 critical, 20 high, 25 moderate, 6 low). To find out more, visit:
remote:      https://github.com/calypr/funnel/security/dependabot
remote:
To github.com:calypr/funnel
 * [new branch]        review/reconcile_upgrade -> review/reconcile_upgrade
branch 'review/reconcile_upgrade' set up to track 'origin/review/reconcile_upgrade'.

…b backof limit is not exhausted

…/reconcile_upgrade

…of this.

…funnel into chore/reconcile_upgrade

…/reconcile_upgrade and rename updated monolith reconcile function as `reconcile_monolith`.

…k in Running as System error

891ea9e was the funnel merge commit, not gen3-code-vigil, so the checkout failed with 'not our ref'. 69c620a (Gen3SDK Tests #558) is the gen3-code-vigil commit from the last green run, and its conftest.py predates the get_navigation_url() call that crashes on master. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Assisted-by: Claude Code:claude [Claude Code]

…pgrade

Signed-off-by: Liam Beckman <lbeckman314@gmail.com>

github-actions · 2026-06-22T21:32:05Z

Gen3 Integration Tests

filepath	passed	skipped	SUBTOTAL
tests/test_gen3_workflow.py	25	4	29
TOTAL	25	4	29

Signed-off-by: Liam Beckman <lbeckman314@gmail.com>

github-actions · 2026-06-22T23:17:08Z

Gen3 Integration Tests

filepath	passed	skipped	SUBTOTAL
tests/test_gen3_workflow.py	25	4	29
TOTAL	25	4	29

github-actions · 2026-06-26T04:26:13Z

Gen3 Integration Tests

filepath	passed	skipped	SUBTOTAL
tests/test_gen3_workflow.py	25	4	29
TOTAL	25	4	29

github-actions · 2026-06-26T18:43:41Z

Gen3 Integration Tests

filepath	passed	skipped	SUBTOTAL
tests/test_gen3_workflow.py	25	4	29
TOTAL	25	4	29

nss10 and others added 30 commits May 7, 2026 14:29

Add reconcile_modular draft code

21c1206

Improve reconciler modular code

d84fa3a

switch to reconclie_modular

9e4854d

Add Debug logs for PVC deletiojn

661f61e

handle reconciler when job restarts happen

e847336

cause exec_error to maje a sys_error

6953a4f

Revert exec_err to sys_err, exec pod crash is no longer sys_err if jo…

697843a

…b backof limit is not exhausted

Remove lastAttempt check

37ea995

Merge tag '2026-05-12' of https://github.com/calypr/funnel into chore…

98d0866

…/reconcile_upgrade

Remove TTLSecondsAfterFinished from worker pod.

7e43576

Set quay to cdis

5d296f9

Remove old code and WIP comments

ffa4030

TTLSecondsAfterFinished to worker job removed. Jobcleanup takes care …

adb1fe8

…of this.

Print SQL logs to identify bug

777acfd

Add state filter in ListTasks

7f60650

Add state in the ListTasks tests

ede3f33

Merge branch 'develop' into chore/reconcile_upgrade

7f3b110

Remove sql debug logs

88b3390

Merge branch 'chore/reconcile_upgrade' of https://github.com/uc-cdis/…

598528d

…funnel into chore/reconcile_upgrade

Merge branch 'develop' of https://github.com/calypr/funnel into chore…

c9b414a

…/reconcile_upgrade and rename updated monolith reconcile function as `reconcile_monolith`.

Add updated reconcile_monolith and modular reconciler

97009c7

Retry pods when job backoff limit is not reached yet

082a1a1

Fix compiler error in reconciler_monolith

0c3f225

Replace inner function to class level function and mark completed tas…

60ba572

…k in Running as System error

Remove tests linked to TTLSecondsAfterFinished

ee31f49

Update backofflimit check when task is in terminal state.

e9da4c6

task: Revert cancel response coe back to 200

a6d75e4

Update integration_tests_on_kind.yaml

e12f2de

Fix isJobMarkedAsFailed for tasks that are completed.

a4f4fbc

Add debug logs to understand the jobConditions for successful tasks

72b368f

nss10 added 3 commits June 20, 2026 23:33

isJobDone

6dda082

Final commits and comment improvements before making PR.

fd848b3

Merge branch 'develop' into chore/reconcile_upgrade

2fbbb4d

lbeckman314 self-assigned this Jun 22, 2026

lbeckman314 added bug blocker blocks user interactions labels Jun 22, 2026

lbeckman314 added this to the Gen3Workflow Production-Ready Support milestone Jun 22, 2026

lbeckman314 added 7 commits June 22, 2026 12:55

debug: Integration tests

9cb2bd9

ci: re-trigger integration tests

544446a

debug: CI

3968358

Merge branch 'task/revert-200' into review/reconcile_upgrade

a3af8e1

Merge remote-tracking branch 'origin/develop' into review/reconcile_u…

2119401

…pgrade

fix: Cleanup code

f2895a0

Signed-off-by: Liam Beckman <lbeckman314@gmail.com>

lbeckman314 marked this pull request as ready for review June 22, 2026 21:13

task: Revert changes to better match PR #1410

098fb86

Signed-off-by: Liam Beckman <lbeckman314@gmail.com>

chore: Unpin integration tests action

60c1bf4

lbeckman314 mentioned this pull request Jun 26, 2026

Chore/reconcile upgrade #1410

Open

8 tasks

Merge branch 'develop' into review/reconcile_upgrade

c28787b

lbeckman314 mentioned this pull request Jun 27, 2026

feat: Add initial funnel k8s cleanup command #1439

Open

8 tasks

lbeckman314 modified the milestones: Gen3Workflow Production-Ready Support, Helm Chart v0.1.99-rc.39 Jul 3, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Review/reconcile upgrade#1438

Review/reconcile upgrade#1438
lbeckman314 wants to merge 43 commits into
developfrom
review/reconcile_upgrade

lbeckman314 commented Jun 22, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 22, 2026

Uh oh!

github-actions Bot commented Jun 22, 2026

Uh oh!

github-actions Bot commented Jun 26, 2026

Uh oh!

github-actions Bot commented Jun 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

lbeckman314 commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview 🌀

Uh oh!

github-actions Bot commented Jun 22, 2026

Gen3 Integration Tests

Uh oh!

github-actions Bot commented Jun 22, 2026

Gen3 Integration Tests

Uh oh!

github-actions Bot commented Jun 26, 2026

Gen3 Integration Tests

Uh oh!

github-actions Bot commented Jun 26, 2026

Gen3 Integration Tests

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

lbeckman314 commented Jun 22, 2026 •

edited

Loading