Initial support for aerosols in PROTEUS, improve chemistry plot, and fix broken links in readme

[nichollsh]

Description

• Adds option for enabling aerosols in AGNI radiative transfer. Closes Configure AGNI aerosol/haze parametrisation via PROTEUS #663.
• Downloads aerosol scattering data from Zenodo.
• Reads aerosol MMR profiles from AGNI output NetCDF files.
• Updates chemistry plot to include aerosol and cloud profiles, with T(p) for context
• Fixes broken links in readme. Closes Links in PROTEUS readme are broken #664.

This PR aims to introduce aerosol support via PROTEUS. We will need to re-visit it via future PRs, when some kind of microphysical/chemical scheme is implemented in AGNI to derive aerosol mixing ratio profiles.

Validation of changes

PROTEUS runs fine and makes the new plot (see below). Note that the aerosol profiles are at MMR=0 because AGNI does not yet have a scheme for calculating these profiles. Also added tests for these new features (manually and using Copilot), which pass on my laptop and on the github runners.

Checklist

• I have followed the contributing guidelines
• My code follows the style guidelines of this project
• I have performed a self-review of my code
• My changes generate no new warnings or errors
• I have checked that the tests still pass on my computer
• I have updated the docs, as appropriate
• I have added tests for these changes, as appropriate
• I have checked that all dependencies have been updated, as required

Relevant people

@timlichtenberg

[Copilot]

Pull request overview

This PR adds a configurable switch for AGNI aerosol radiative effects via the PROTEUS config system, updates the AGNI demo configuration accordingly, and fixes broken README documentation links to match the current Diátaxis docs structure.

Changes:

• Add atmos_clim.aerosols_enabled configuration option and wire it into AGNI initialization.
• Detect available aerosol species from the FWL data directory and pass them to AGNI (initially with zero MMR).
• Update README documentation links to the correct /How-to, /Explanations, /Reference, and /Community pages.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
src/proteus/config/_atmos_clim.py	Adds aerosols_enabled to AtmosClim config and reuses existing validators.
src/proteus/atmos_clim/agni.py	Detects aerosol species and passes flag_aerosol/aerosol_species into AGNI.atmosphere.setup_b.
input/demos/agni.toml	Adds aerosol/cloud toggles to the demo configuration (currently placed in the wrong table).
README.md	Fixes documentation links to match the current docs site structure.

[codecov]

Codecov Report

❌ Patch coverage is 84.61538% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.23%. Comparing base (4b47696) to head (2fa6c8c).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/proteus/atmos_clim/agni.py	70.58%	5 Missing ⚠️
src/proteus/atmos_clim/common.py	82.75%	0 Missing and 5 partials ⚠️
src/proteus/utils/data.py	50.00%	2 Missing and 2 partials ⚠️
src/proteus/cli.py	60.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #665      +/-   ##
==========================================
+ Coverage   69.08%   70.23%   +1.15%     
==========================================
  Files          98       98              
  Lines       10123    10207      +84     
  Branches     1401     1421      +20     
==========================================
+ Hits         6993     7169     +176     
+ Misses       2791     2749      -42     
+ Partials      339      289      -50     

Flag	Coverage Δ
unit-tests	57.42% <84.61%> (+1.11%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[nichollsh]

@timlichtenberg I think this is now ready for review. Not sure why the codecov report says some lines are uncovered, because the function is clearly being called and I have confirmed this manually by adding print() calls when running tests locally. You can also see that the "project" coverage is overall increased by this PR.

[timlichtenberg]

Awesome work! When fully integrated this will open a whole new category of applications. A few things to address before merging, most important is a potential crash when plotting output from runs without aerosols.

1. KeyError on atm_profile['aerosols'] in cpl_chem_atmosphere.py: the aerosol loop accesses this key unconditionally, but read_ncdf_profile only populates it when the NetCDF has aerosol data. The cloud block right above correctly guards with if 'cloud_mmr' in atm_profile.keys(), but the aerosol block does not. This will crash on any pre-aerosol run output. Should be atm_profile.get('aerosols', []).

2. num_aerosols starts at 1, not 0. This inflates the legend column count. If there are no aerosols and no cloud, it still counts 1. Should start at 0 and increment separately when the cloud is plotted.

3. fig.text with transform=ax1.transAxes: this works by accident because (0.02, 0.98) in ax1 axes coords happens to land in the right spot. Should be ax1.text(...) instead. If the layout changes this will silently break.

4. aer_mmr reading in common.py: the guard checks 'aer_mmr' in ds.variables but then unconditionally reads ds.variables['aerosols'] for species names. If the file has one but not the other, this crashes. Add and 'aerosols' in ds.variables.keys() to the guard.

5. No OSF fallback for scattering data. All other entries in DATA_SOURCE_MAP have both Zenodo and OSF IDs. I know we move to other sources, but should we add it until then?

[nichollsh]

Thanks, Tim, your comments are very useful. It's exciting to introduce this functionality and I look forward to developing it further with some schemes for calculating MMR profiles from the chemistry.

1. I have now added this guard into the function and tested running PROTEUS with and without aerosols.

2. Following changes for (1) above, I have also reworked the aerosol counting to start at 0.

3. Fixed.

4. This is not an issue because aerosols and aer_mmr are always written together by AGNI. There will never be a case where one is missing while the other is present.

5. I have uploaded these to OSF now. I did not initially do this for a couple of reasons. Firstly, because we are moving away from OSF. Secondly, because there's a good chance the aerosol scattering data will be updated in the near future anyway.

[timlichtenberg]

Nice!

[timlichtenberg]

Another Code review found 3 more issues

1. warn_if_dummy uses attribute.name but existing test passes SimpleNamespace (no .name attribute). The error message was changed from a hardcoded string to f'Dummy atmos_clim module is incompatible with {attribute.name}=True', but tests/config/test_config.py::test_atmos_clim_warn_if_dummy_rayleigh_incompatible still passes a types.SimpleNamespace and matches the old string. CI confirms AttributeError.

PROTEUS/src/proteus/config/_atmos_clim.py

Lines 19 to 21 in 24a9aae

	def warn_if_dummy(instance, attribute, value):
	if (instance.module == 'dummy') and value:
	raise ValueError(f'Dummy atmos_clim module is incompatible with {attribute.name}=True')

2. All 8 new plot tests fail with IndexError due to inconsistent mocks. The tests mock ax1_mock.get_legend_handles_labels.return_value = ([], []) (empty handles), but the mocked profile data contains gases with nonzero VMR. The production code at line 173 does order = np.argsort(vmr_surf)[::-1] and then indexes into the empty handles list, causing IndexError: list index out of range. The mock handles list needs entries matching the number of plotted species.

PROTEUS/tests/plot/test_cpl_chem_atmosphere.py

Lines 78 to 80 in 24a9aae

	# Mock get_legend_handles_labels to return empty lists
	ax1_mock.get_legend_handles_labels.return_value = ([], [])

PROTEUS/src/proteus/plot/cpl_chem_atmosphere.py

Lines 171 to 176 in 24a9aae

	handles, labels = ax1.get_legend_handles_labels()
	order = np.argsort(vmr_surf)[::-1]
	ax1.legend(
	[handles[idx] for idx in order],
	[labels[idx] for idx in order],
	loc='center left',

3. test_plot_chem_atmosphere_time_annotation asserts against wrong mock object. The test checks fig_mock.text.assert_called_once(), but the production code calls ax1.text(...). Should be ax1_mock.text.assert_called_once().

PROTEUS/tests/plot/test_cpl_chem_atmosphere.py

Lines 345 to 348 in 24a9aae

	# Verify time annotation was added to figure
	fig_mock.text.assert_called_once()
	call_args = fig_mock.text.call_args[0]
	# Should contain the year value (5000)

PROTEUS/src/proteus/plot/cpl_chem_atmosphere.py

Lines 251 to 261 in 24a9aae

	ax1.text(
	0.02,
	0.98,
	f't = {year:.2e} yr',
	transform=ax1.transAxes,
	ha='left',
	va='top',
	fontsize=11,
	weight='bold',
	zorder=999,
	)

[timlichtenberg]

See new comment.

[nichollsh]

The above commit should fix the issues with the tests. The main changes are: update the mock-function calls to ensure that the correct number of gas labels/lines are returned, check that fig.savefig is called (which it is) rather than plt.savefig, and corrects issue with the dummy atmos_clim validator.

I have also made sure to run these tests with pytest -k test_cpl_chem_atmosphere locally and they all pass now. Additionally, I've run the dummy atmosphere module with aerosols enabled to check that it does (correctly) fail the config validation.

This should address the issues in your last review @timlichtenberg

One thing it made me realise: there's no functionality to check that all config options are parsed correctly. For example, a user might write rayleigh_enabled=True and believe that they've enabled rayleigh scattering. However, the correct option is rayleigh=True. This could lead to substantial issues when running+interpreting simulation outputs. I'll make an issue.

[timlichtenberg]

Great!

[nichollsh]

Thanks, again, for catching these issues! Merging this now.