Skip to content

handles the case of running a model where a provider doesn't have A&E data#649

Open
tomjemmett wants to merge 6 commits into
mainfrom
fix_648
Open

handles the case of running a model where a provider doesn't have A&E data#649
tomjemmett wants to merge 6 commits into
mainfrom
fix_648

Conversation

@tomjemmett

Copy link
Copy Markdown
Member

fixes #648

  • adds step to skip downloading data if it doesn't exist
  • changes type of the data argument to model instances to be Data, not a callable
  • adds method to test that files exist for a specified model type
  • skips running a model type if data doesn't exist
  • handles the edge case in sdec patching for providers without A&E data

Copilot AI review requested due to automatic review settings July 23, 2026 20:50
@tomjemmett
tomjemmett requested a review from a team as a code owner July 23, 2026 20:50
@tomjemmett tomjemmett self-assigned this Jul 23, 2026
@tomjemmett tomjemmett added the priority: must We must implement this feature as soon as is reasonably possible label Jul 23, 2026
@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (0daacc3) to head (a0cc5b8).

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #649   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           20        20           
  Lines         1129      1149   +20     
  Branches        58        64    +6     
=========================================
+ Hits          1129      1149   +20     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

Copy link
Copy Markdown

✅ A new build is available.

You can use the following to use pull the image into your local environment:

docker pull ghcr.io/the-strategy-unit/nhp_model:pr-649

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses issue #648 by making the model runner resilient when a provider lacks data for a given model type (notably A&E). It does so by adding “data presence” checks, skipping downloads/runs where data is missing, and refactoring model/data wiring so model instances receive a Data instance rather than a factory callable.

Changes:

  • Add Data.data_exists_for_model_type(...) and implement it for Local, then use it to skip model types that have no data.
  • Refactor model/run entrypoints to pass a Data instance through to model classes (and update unit/integration/e2e tests accordingly).
  • Make SDEC patching and Azure data download logic no-op safely when the expected data/paths are missing.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/unit/nhp/model/test_run.py Updates run_all/run_single_model_run tests for the new “data instance + filtering” behavior and adds coverage for skipping missing model types.
tests/unit/nhp/model/test_results.py Adds a regression test ensuring SDEC patching is a no-op when the expected results key is missing.
tests/unit/nhp/model/test_model.py Updates model construction tests for the new “data instance” contract.
tests/unit/nhp/model/data/test_local.py Adds unit tests for Local.data_exists_for_model_type(...) including unknown model types.
tests/unit/nhp/model/data/test_data.py Adds a base-class contract test for Data.data_exists_for_model_type(...).
tests/unit/nhp/docker/test_run.py Adds coverage ensuring Azure downloads are skipped when a directory doesn’t exist.
tests/integration/nhp/model/test_single_model_run.py Updates integration setup to instantiate Local directly with year/dataset.
tests/e2e/test_run_model_snapshot.py Updates E2E setup to pass a Local instance (not a callable factory) into HSA/model execution.
src/nhp/model/run.py Refactors run_all/_run_model to work with a Data instance and to skip model types with missing data.
src/nhp/model/results.py Guards _patch_converted_sdec_activity so it no-ops when the target aggregation is absent.
src/nhp/model/outpatients.py Updates constructor parameter naming/types to accept a Data instance.
src/nhp/model/model.py Refactors base Model to accept a Data instance instead of creating one from a callable.
src/nhp/model/inpatients.py Updates constructor parameter naming/types to accept a Data instance.
src/nhp/model/data/local.py Implements data_exists_for_model_type(...) for the local filesystem layout.
src/nhp/model/data/data.py Adds the new data_exists_for_model_type(...) interface method and fixes an HSA activity table docstring.
src/nhp/model/aae.py Updates constructor parameter naming/types to accept a Data instance.
src/nhp/docker/run.py Skips downloading data subpaths that don’t exist in ADLS to avoid crashes for missing model-type directories.

Comment thread src/nhp/model/run.py
Comment on lines 164 to 168
aggregation_columns,
)
for m in model_types
if nhp_data.data_exists_for_model_type(m)
]
Comment thread src/nhp/model/run.py
Comment on lines +119 to 121
nhp_data: Data | Callable[[int, str], Data],
progress_callback: Callable[[Any], Callable[[Any], None]] = noop_progress_callback,
save_full_model_results: bool = False,
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority: must We must implement this feature as soon as is reasonably possible

Projects

None yet

Development

Successfully merging this pull request may close these issues.

model fails to run if a provider doesn't have data for A&E

2 participants