handles the case of running a model where a provider doesn't have A&E data#649
handles the case of running a model where a provider doesn't have A&E data#649tomjemmett wants to merge 6 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #649 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 20 20
Lines 1129 1149 +20
Branches 58 64 +6
=========================================
+ Hits 1129 1149 +20 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
✅ A new build is available.You can use the following to use pull the image into your local environment: docker pull ghcr.io/the-strategy-unit/nhp_model:pr-649 |
There was a problem hiding this comment.
Pull request overview
This PR addresses issue #648 by making the model runner resilient when a provider lacks data for a given model type (notably A&E). It does so by adding “data presence” checks, skipping downloads/runs where data is missing, and refactoring model/data wiring so model instances receive a Data instance rather than a factory callable.
Changes:
- Add
Data.data_exists_for_model_type(...)and implement it forLocal, then use it to skip model types that have no data. - Refactor model/run entrypoints to pass a
Datainstance through to model classes (and update unit/integration/e2e tests accordingly). - Make SDEC patching and Azure data download logic no-op safely when the expected data/paths are missing.
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/nhp/model/test_run.py | Updates run_all/run_single_model_run tests for the new “data instance + filtering” behavior and adds coverage for skipping missing model types. |
| tests/unit/nhp/model/test_results.py | Adds a regression test ensuring SDEC patching is a no-op when the expected results key is missing. |
| tests/unit/nhp/model/test_model.py | Updates model construction tests for the new “data instance” contract. |
| tests/unit/nhp/model/data/test_local.py | Adds unit tests for Local.data_exists_for_model_type(...) including unknown model types. |
| tests/unit/nhp/model/data/test_data.py | Adds a base-class contract test for Data.data_exists_for_model_type(...). |
| tests/unit/nhp/docker/test_run.py | Adds coverage ensuring Azure downloads are skipped when a directory doesn’t exist. |
| tests/integration/nhp/model/test_single_model_run.py | Updates integration setup to instantiate Local directly with year/dataset. |
| tests/e2e/test_run_model_snapshot.py | Updates E2E setup to pass a Local instance (not a callable factory) into HSA/model execution. |
| src/nhp/model/run.py | Refactors run_all/_run_model to work with a Data instance and to skip model types with missing data. |
| src/nhp/model/results.py | Guards _patch_converted_sdec_activity so it no-ops when the target aggregation is absent. |
| src/nhp/model/outpatients.py | Updates constructor parameter naming/types to accept a Data instance. |
| src/nhp/model/model.py | Refactors base Model to accept a Data instance instead of creating one from a callable. |
| src/nhp/model/inpatients.py | Updates constructor parameter naming/types to accept a Data instance. |
| src/nhp/model/data/local.py | Implements data_exists_for_model_type(...) for the local filesystem layout. |
| src/nhp/model/data/data.py | Adds the new data_exists_for_model_type(...) interface method and fixes an HSA activity table docstring. |
| src/nhp/model/aae.py | Updates constructor parameter naming/types to accept a Data instance. |
| src/nhp/docker/run.py | Skips downloading data subpaths that don’t exist in ADLS to avoid crashes for missing model-type directories. |
| aggregation_columns, | ||
| ) | ||
| for m in model_types | ||
| if nhp_data.data_exists_for_model_type(m) | ||
| ] |
| nhp_data: Data | Callable[[int, str], Data], | ||
| progress_callback: Callable[[Any], Callable[[Any], None]] = noop_progress_callback, | ||
| save_full_model_results: bool = False, |
fixes #648