refactor: pipeline orchestration + cli seams#354
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
Breaking change: Yes — this PR removes raitap.pipeline.run and renames/removes raitap.pipeline.run_without_tracking (now run_phases) from the importable raitap.pipeline surface.
Warning
The PR contains breaking changes, but the tiles does not contain "!". Please mark it as breaking by adding "!" after the type or scope (e.g. feat!: or feat(api)!:).
This PR refactors pipeline orchestration and CLI argument handling to reduce duplication and improve internal seams, while updating tests/docs to match the new names and extraction boundaries.
Changes:
- Decomposes
pipeline.orchestrator._run_pipelineinto smaller helpers and renamesrun_without_tracking→run_phases. - Consolidates Hydra argv munging into a new stdlib-only
src/raitap/_cli_argv.py, used bycli.py,pipeline/__main__.py, anddeps/bootstrap.py. - Adds debug logging to previously silent best-effort fallbacks in
models/model.pyand updates call sites/tests/docs for the renamed pipeline helpers.
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/raitap/tracking/smoke_test_mlflow.py | Switches smoke test to use public raitap.run. |
| src/raitap/tests/test_run_main.py | Updates tests to call _run_pipeline and patches run_phases instead of run_without_tracking. |
| src/raitap/tests/test_run_labels.py | Updates metrics-label threading test for run_phases rename. |
| src/raitap/tests/test_reproducibility_orchestrator.py | Updates reproducibility tests to patch run_phases. |
| src/raitap/tests/test_memory_leaks.py | Updates memory leak test imports and name references for run_phases. |
| src/raitap/tests/test_e2e_detection.py | Updates E2E detection test to use run_phases. |
| src/raitap/tests/test_cli_argv.py | New unit tests covering shared argv helper behavior. |
| src/raitap/robustness/tests/test_e2e_real_data.py | Updates robustness E2E test naming/imports for run_phases. |
| src/raitap/pipeline/phases/registry.py | Updates doc reference from run_without_tracking to run_phases. |
| src/raitap/pipeline/outputs.py | Updates docstring to reference raitap.run as the primary entrypoint. |
| src/raitap/pipeline/orchestrator.py | Extracts helpers (_build_run_context, _write_run_reproducibility, _generate_report, _log_run_to_tracker) and renames run_without_tracking → run_phases. |
| src/raitap/pipeline/main.py | Uses _cli_argv constants/helpers for Hydra config-path/name and argv rewriting. |
| src/raitap/pipeline/init.py | Replaces run/run_without_tracking wrappers with typed lazy __getattr__ exports for run_phases and other functions. |
| src/raitap/models/model.py | Adds debug logging when best-effort metadata parsing falls back. |
| src/raitap/deps/bootstrap.py | Deduplicates Hydra flag parsing by importing _cli_argv helpers/constants. |
| src/raitap/cli.py | Delegates demo/help-frame logic to _cli_argv. |
| src/raitap/_cli_argv.py | New shared stdlib-only argv parsing/rewriting utilities. |
| docs/contributor/architecture.md | Updates contributor docs for run_phases rename. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Four contained pipeline/CLI-seam cleanups surfaced by the advisor review (issue #1, the config-access campaign, shipped separately as #353). No user-facing config or behaviour change; the public
raitap.runentry is untouched._run_pipeline— the 108-line god function inpipeline/orchestrator.pysplit into four order-preserving helpers (_build_run_context,_write_run_reproducibility,_generate_report,_log_run_to_tracker). Pure extraction.except Exception: return Nonefallbacks inmodels/model.pynow emit araitap_log.debug(..., module=Module.models)line. Fallback behaviour unchanged.cli.py+pipeline/__main__.pyinto a new stdlib-onlyraitap/_cli_argv.py;deps/bootstrap.pyalso now consumes it (killing a third duplicate copy of the flag constants + parser). Newtests/test_cli_argv.py.runnames —orchestrator.run_without_tracking→run_phases(named by what it does); the duplicatepipeline.runre-export deleted (tracking/smoke_test_mlflowmigrated to publicraitap.run);pipeline/__init__now uses a typed lazy__getattr__re-export soraitap.pipeline.run_phasesgets a real typed signature instead of(*args, **kwargs).API surface note (pre-1.0, non-headline):
raitap.pipeline.runis removed andraitap.pipeline.run_without_tracking→run_phases. The documented public entryraitap.runis unchanged. Judged non-breaking for release purposes; flag if you'd prefer a!title.Reviewed by whole-branch correctness (clean) + refactor (bootstrap dedup folded in) passes.
Closes #351
Checklist
!. Assessed: publicraitap.rununtouched; only pre-1.0 internalraitap.pipelinehelpers renamed/removed — not marked breaking.Optional
docs/contributor/architecture.mdupdated for therun_phasesrename.test_cli_argv.py; helper unit tests added for the decomposition.