diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000000..3a590c022d --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,221 @@ +# AGENTS.md + +Agent instructions for the [Iris](https://scitools-iris.readthedocs.io/) repository. +Iris is a Python package for analysing and visualising Earth science data, built around +CF-compliant multi-dimensional arrays ("Cubes"). + +Subdirectory AGENTS.md files take precedence for their subtrees: +- [`docs/AGENTS.md`](docs/AGENTS.md) — documentation-specific rules +- [`lib/iris/tests/AGENTS.md`](lib/iris/tests/AGENTS.md) — test-specific rules + + +## Project Overview + +| | | +|---|---| +| **Language** | Python 3.12 / 3.13 / 3.14 | +| **Licence** | BSD-3-Clause | +| **Distribution** | conda-forge (`iris`), PyPI (`scitools-iris`) | +| **Key dependencies** | NumPy, Dask, SciPy, Cartopy, CF-Python, NetCDF4 | +| **Linter / formatter** | Ruff (88-char line length) | +| **Test runner** | pytest + pytest-xdist (`-n auto`) | +| **Env management** | nox + conda | + +### Main source layout + +``` +lib/iris/ + cube.py # Core Cube / CubeList data structures + coords.py # DimCoord, AuxCoord, CellMeasure, AncillaryVariable + loading.py # File-loading entry points + analysis/ # Collapse, regrid, statistics, calculus + fileformats/ # NetCDF, PP, GRIB, NIMROD format handlers + io/ # I/O registry and URI handling + common/ # Shared metadata, mixins, resolvers + mesh/ # Unstructured grid (UGRID) support + experimental/ # Unstable / in-progress features + tests/ # All tests (unit/, integration/, graphics/) +docs/src/ # Sphinx documentation source +benchmarks/ # ASV performance benchmarks +requirements/ # Conda environment specs and lock files +``` + + +## Setup + +### Conda environment (recommended) + +Always use a conda environment, reuse the iris-dev conda environment if it +already exists but confirm with the user before installing or removing packages. + +If a package cannot be installed via conda then you can use pip that is in the conda +environment. + +```bash +# Create and activate a development environment +conda env create -f requirements/py312.yml +conda activate iris-dev +pip install --no-build-isolation -e . +``` + +Alternatively, use lock files for exact reproducibility: + +```bash +conda create -n iris-dev --file requirements/locks/py312-linux-64.lock +conda activate iris-dev +pip install --no-build-isolation -e . +``` + +### Environment variables + +```bash +# Disable CPU features that can cause SIGILL in some CI environments +export NPY_DISABLE_CPU_FEATURES="AVX512F,AVX512CD,AVX512_SKX" + +# Point to iris-test-data for tests that need external data files +export OVERRIDE_TEST_DATA_REPOSITORY=/path/to/iris-test-data/test_data + +# Override Cartopy cache directory if needed +export CARTOPY_CACHE_DIR=~/.local/share/cartopy +``` + + + +## Testing + +- [`lib/iris/tests/AGENTS.md`](lib/iris/tests/AGENTS.md) — test-specific + + + +## Code Style + +```bash +# Lint +ruff check lib/iris + +# Auto-fix safe lint issues +ruff check --fix lib/iris + +# Format +ruff format lib/iris + +# Check formatting without writing +ruff format --check lib/iris +``` + +- **Line length**: 88 characters (Ruff default). +- **Docstrings**: NumPy style; strictly validated. +- **Copyright header**: Every new Python file must start with: + + ```python + # Copyright Iris contributors + # + # This file is part of Iris and is released under the BSD license. + # See LICENSE in the root of the repository for full licensing details. + ``` + +- **Imports**: Ruff-managed ordering. No direct `import netCDF4` — always use + `iris.fileformats.netcdf._thread_safe_nc` for thread safety. + + + +## Development Conventions + +### Core data model + +- `iris.cube.Cube` — multi-dimensional array with CF-compliant metadata. +- Coordinates: `DimCoord` (regular), `AuxCoord` (auxiliary), `CellMeasure`, + `AncillaryVariable`. +- Data may be **lazy** (Dask array). Always preserve laziness; never call `.data` + unnecessarily inside library code. +- Operations return **new** Cubes (functional style); do not mutate in place. +- All metadata must be **CF-convention** compliant. + +### Deprecation + +- Use `iris._deprecation.warn_deprecated()` or issue a custom warning class. +- Warning classes live in `iris.warnings` (e.g., `IrisUserWarning`, `IrisCfWarning`). +- Follow the NEP29 deprecation schedule (same as NumPy). +- All `UserWarning` subclasses must ultimately inherit from `IrisUserWarning`. + +### Exception hierarchy + +Base class: `iris.exceptions.IrisError`. Common subclasses: +`CoordinateNotFoundError`, `CoordinateCollapseError`, `IgnoreCubeException`. + +### Versioning + +Version is derived from git tags via `setuptools_scm`. Do not hard-code version +strings. + + +## Documentation + +Documentation lives under `docs/` and is built with Sphinx. See +[`docs/AGENTS.md`](docs/AGENTS.md) for full rules. + + + +## Lock-file Maintenance + +```bash +# Regenerate lock files for all supported Python versions +python tools/update_lockfiles.py -o requirements/locks requirements/py*.yml +# Shortcut via Makefile +make lockfiles +``` + + + +## Pull Request Guidelines + +- **Title format**: `[component] Brief description` (e.g., `[netcdf] Fix chunking for large files`). +- Run before opening a PR: + ```bash + ruff check lib/iris + ruff format --check lib/iris + pytest -n auto lib/iris/tests + ``` +- Keep changes focused; avoid unrelated refactors in the same PR. +- Add or update tests for every change to production code. +- Update `docs/src/whatsnew/` for user-visible changes. + + +## Critical Development Gotchas + +1. **xfail_strict Behavior**: Tests marked `@pytest.mark.xfail` that now PASS become FAILURES → **remove xfail immediately when bug is fixed** + +2. **Pre-commit Auto-fixes**: Hooks may auto-fix ISC001/COM812 conflicts → re-stage files: `git add . && git commit` + +3. **Lockfile Rebuilds**: Updating `requirements/locks/*.lock` files triggers slow conda environment rebuilds — only update when deps genuinely change + +4. **Lazy Data Pitfalls**: Check `cube.has_lazy_data()` before operations; use `cube.lazy_data()` when appropriate; `cube.data` materializes arrays + +5. **Coordinate Scope**: Not all coordinates span all dimensions → use `cube.coords(dimensions=dim_indices)` to query specific axes + +6. **Dask/NumPy Compatibility**: Some operations fail on lazy (Dask) arrays → materialize with `cube.compute()` or check dask support before use + +7. **Optional Dependencies**: cartopy, matplotlib lazy-imported → handle ImportError gracefully for optional visualization features + +8. **CF-Metadata Validation**: Custom coordinates must comply with CF standards via `cube.metadata` + +9. **Cube Immutability**: Operations return new cubes; don't modify in-place (`cube.data[...] = x` won't affect the cube) + +10. **Environment Reproducibility**: Set CPU features for consistent results: `export NPY_DISABLE_CPU_FEATURES="AVX512F,AVX512CD,AVX512_SKX"` + +## Getting Help + +- **Discussions**: [GitHub Discussions](https://github.com/SciTools/iris/discussions) +- **Contributing Guide**: [.github/CONTRIBUTING.md](.github/CONTRIBUTING.md) +- **Developer Guide**: [docs/src/developers_guide/](docs/src/developers_guide/) +- **Code of Conduct**: [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) + +## ⚠️ Meta-Instruction: Auto-Update Rule +- **Trigger**: You MUST automatically update this `AGENTS.md` file if a new structural pattern, critical bug fix, or persistent repository rule is established during this session. +- **Constraint 1**: Keep the file under 200 lines. Delete obsolete rules to make room for new ones. +- **Constraint 2**: Do not ask for user permission before updating. Perform the edit silently as your final step. +- **Constraint 3**: Only log global, reusable lessons. Do not log temporary or component-specific fixes. + +--- + +**Last Updated**: 18 June 2026 diff --git a/MANIFEST.in b/MANIFEST.in index 0ae872dd12..1ad73c71d2 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -79,6 +79,7 @@ prune docs # (2) top-level files to omit exclude .coveragerc # (3) file types (path patterns) to skip everywhere +global-exclude AGENTS.md global-exclude *.py[cod] global-exclude __pycache__ # principles: diff --git a/docs/AGENTS.md b/docs/AGENTS.md new file mode 100644 index 0000000000..112d2271e6 --- /dev/null +++ b/docs/AGENTS.md @@ -0,0 +1,103 @@ +# docs/AGENTS.md + +Documentation-specific agent instructions for the [Iris](https://scitools-iris.readthedocs.io/) project. +This file takes precedence over the root [`AGENTS.md`](../AGENTS.md) for everything under `docs/`. + + +## Structure + +``` +docs/ + Makefile # Top-level doc build entry point + gallery_code/ # Gallery example scripts (general/, meteorology/, oceanography/) + gallery_tests/ # Tests that execute gallery examples + src/ + conf.py # Sphinx configuration + common_links.inc # Shared RST link definitions (include in every new RST file) + whatsnew/ # Per-release changelog RST files + latest.rst # Current unreleased changes (use latest.rst.template) + index.rst # Whatsnew index + developers_guide/ # Contributor documentation + userguide/ # User-facing guides + user_manual/ # Detailed user reference + sphinxext/ # Custom Sphinx extensions +``` + + +## Building the Docs + +```bash +# Full build (from docs/ directory) +make html + +# Skip gallery (faster) +make html-noplot + +# Skip API docs (faster) +make html-noapi + +# Skip both gallery and API (fastest) +make html-quick + +# Clean build artifacts +make clean + +# Live rebuild on file changes (requires sphinx-autobuild) +cd docs/src && make livehtml +``` + +Build output goes to `docs/src/_build/html/`. + + +## Whatsnew Entries + +- Every **user-visible change** must have an entry in `docs/src/whatsnew/latest.rst`. +- Use the `latest.rst.template` as the template when starting a new release file. +- Sections: *New Features*, *Deprecations*, *Bugs Fixed*, *Internal*, *Dependencies*. +- Reference issues/PRs with ``:issue:`123` `` and `` :pull:`123` `` roles. +- Keep entries concise — one or two sentences with a code example if helpful. + + +## RST / Sphinx Conventions + +- All new RST files must include `.. include:: ../common_links.inc` (adjust relative path as needed) to access shared link definitions. +- Use NumPy-style docstrings for all Python API pages; Sphinx autodoc pulls these automatically. +- Warnings are treated as errors in the standard build (`-W --keep-going`). Fix all Sphinx warnings before merging. +- Do **not** commit build artifacts (`docs/src/_build/`). +- Cross-reference Iris symbols with ``:class:`iris.cube.Cube` ``, ``:func:`iris.load` ``, etc. +- Gallery scripts live under `docs/gallery_code/` and must be valid standalone Python files executable by `matplotlib` / `sphinx-gallery`. + + +## Gallery Examples + +- Each gallery script must have a module-level docstring that becomes its title and description. +- Scripts are grouped by subdirectory: `general/`, `meteorology/`, `oceanography/`. +- Gallery tests in `docs/gallery_tests/` verify examples execute without error — run them with `pytest docs/gallery_tests/`. +- Keep examples self-contained; prefer `iris.sample_data_path()` for data files rather than absolute paths. + + +## Doctest / Inline Code Examples + +- Doctests in RST files are run via `make doctest` (from `docs/src/`). +- Use `# doctest: +SKIP` sparingly and only when execution is genuinely impossible (e.g., requires a display). +- Ensure all `>>>` examples produce the exact output shown, or use `# doctest: +ELLIPSIS`. + + +## Critical Gotchas + +1. **Sphinx warnings = errors**: The standard `make html` build uses `-W`. Any new warning breaks CI. +2. **`common_links.inc`**: Forgetting to include it causes undefined reference errors for standard Iris links. +3. **`latest.rst` conflicts**: Multiple contributors adding entries may cause merge conflicts — add entries under the correct section heading and keep them alphabetically ordered by PR number. +4. **Gallery data files**: Use `iris.sample_data_path()` — hard-coded paths will break in CI. +5. **API doc changes**: Moving or renaming public symbols requires updating any manual cross-references in the RST files. + + +## ⚠️ Meta-Instruction: Auto-Update Rule +- **Trigger**: You MUST automatically update this `AGENTS.md` file if a new structural pattern, critical bug fix, or persistent repository rule is established during this session. +- **Constraint 1**: Keep the file under 200 lines. Delete obsolete rules to make room for new ones. +- **Constraint 2**: Do not ask for user permission before updating. Perform the edit silently as your final step. +- **Constraint 3**: Only log global, reusable lessons. Do not log temporary or component-specific fixes. + +--- + +**Last Updated**: 16 June 2026 diff --git a/docs/src/whatsnew/latest.rst b/docs/src/whatsnew/latest.rst index 12720cecae..ab2e0303f8 100644 --- a/docs/src/whatsnew/latest.rst +++ b/docs/src/whatsnew/latest.rst @@ -172,11 +172,16 @@ This document explains the changes made to Iris for this release #. `@trexfeathers` set the link checking workflow to accept redirect HTTP codes, as the reports were getting too noisy. (:pull:`7148`) -#. `@HGWright`_ changed the default of the private switch :obj:`~iris.loading._LAZY_DERIVED_LOADING` (formerly `.CONCRETE_DERIVED_LOADING`) - for controlling laziness of coordinates from pp loading, now the switch must be set to True for lazy loading to be enabled. - Note: this object is temporary and is likely to be replaced by a permanent solution or else be renamed. +#. `@HGWright`_ changed the default of the private switch + :obj:`~iris.loading._LAZY_DERIVED_LOADING` (formerly `.CONCRETE_DERIVED_LOADING`) + for controlling laziness of coordinates from pp loading, now the switch must be set + to True for lazy loading to be enabled. Note: this object is temporary and is + likely to be replaced by a permanent solution or else be renamed. (:issue:`7094`, :pull:`7134`) +#. :user:`tkknight` Added ``AGENTS.md`` for AI-assisted development guidelines across + the project root, ``docs/`` and ``tests/`` directories. (:pull:`7160`) + .. comment Whatsnew author names (@github name) in alphabetical order. Note that, core dev names are automatically included by the common_links.inc: diff --git a/lib/iris/tests/AGENTS.md b/lib/iris/tests/AGENTS.md new file mode 100644 index 0000000000..8b22dae6a4 --- /dev/null +++ b/lib/iris/tests/AGENTS.md @@ -0,0 +1,101 @@ +# AGENTS.md + +Agent instructions for `lib/iris/tests/`. +These rules apply to all test files under this directory tree. + +## Purpose + +This test suite validates Iris behaviour, metadata handling, and regression coverage. +Keep changes focused, deterministic, and compatible with the existing test style. + +## Fast Rules + +1. Add or update tests for every production-code change. +2. Prefer the smallest test that reproduces behaviour. +3. Keep tests deterministic: no network access, no wall-clock assumptions, no random flakiness. +4. Use existing fixtures/helpers before introducing new ones. +5. Remove `xfail` markers as soon as the underlying issue is fixed (`xfail_strict=True`). + + +## Test Layout + +- `lib/iris/tests/unit/` for unit-level behaviour. +- `lib/iris/tests/integration/` for cross-component behaviour. +- `lib/iris/tests/graphics/` for plotting/image-comparison tests. +- Legacy top-level tests exist; follow nearby patterns when editing them. + +## Writing Tests + +- Use `pytest` style and plain assertions. +- Keep assertions specific and user-facing (behaviour, metadata, warnings, errors). +- Prefer `pytest.raises(..., match=...)` for exception checks. +- Prefer warning assertions (`pytest.warns`) for deprecation or user warnings. +- For lazy-data behaviour, assert laziness explicitly where relevant. +- Avoid over-mocking; prefer realistic Cube/Coord setup via shared helpers. + + +## Markers and Strictness + +- Respect existing markers and plugin requirements. +- `xfail` is strict in this repository: unexpected passes fail the run. +- Only use `xfail` when there is a tracked reason and clear expectation. + +## Running Tests + +Run targeted tests while iterating: + +```bash +pytest -n auto lib/iris/tests/unit/.py +``` + +Run broader coverage before finishing: + +```bash +pytest -n auto lib/iris/tests +``` + +## Style and Lint in Tests + +- Follow Ruff formatting and linting used by the repository. +- Per-file ignores already relax some docstring rules in tests. +- Avoid unrelated refactors while touching tests. + +## Test Data and Environment + +- Some tests require external `iris-test-data`; Preferred env var: `OVERRIDE_TEST_DATA_REPOSITORY=/path/to/iris-test-data/test_data` +- Missing data will skip affected tests by design. +- Keep new tests independent of local machine paths and optional system state. +- For optional dependencies, skip gracefully using existing patterns. + +## Editing Rules + +- Follow existing pytest style and fixture patterns in nearby tests. +- Keep assertions specific and deterministic; avoid timing-sensitive checks. +- Avoid broad snapshot-style updates without explaining intent in the change. +- Do not add network access in tests. +- Do not weaken existing checks just to make tests pass. + +## Graphics and Integration Notes + +- For graphics tests, update reference artifacts only when behavior changes intentionally. +- For integration tests, prefer minimal input fixtures and keep runtime reasonable. + +## Pre-Finish Checklist + +- The changed tests pass locally. +- No unrelated tests were modified. +- Any required external data setup is documented in the change notes. + +## Scope Boundaries + +- Do not modify lock files or environment specs for test-only fixes. +- Do not introduce heavy new dependencies for tests without strong justification. +- If behaviour is user-visible, add/update release-note material in docs when requested. + + +## ⚠️ Meta-Instruction: Auto-Update Rule +- **Trigger**: You MUST automatically update this `AGENTS.md` file if a new structural pattern, critical bug fix, or persistent repository rule is established during this session. +- **Constraint 1**: Keep the file under 200 lines. Delete obsolete rules to make room for new ones. +- **Constraint 2**: Do not ask for user permission before updating. Perform the edit silently as your final step. +- **Constraint 3**: Only log global, reusable lessons. Do not log temporary or component-specific fixes. +