Finalize salience-weighted note-density pipeline

.gitignore

@@ -1,3 +1,5 @@
+local_audit_artifacts/
+audit_artifacts/
 # Bytecode / caches
 __pycache__/
 *.py[cod]
@@ -16,20 +18,43 @@ venv/
 
 # Logs and local run artifacts
 *.log
+gui_worker*.log
 segmentation_metadata.csv
 segmentation_metadata.json
 orchestrator.log
 
 # Default / cache output dirs (regenerated by runs)
 main_analysis_results/
+analysis_results/
+*_analysis_results/
 publication_html_package/
 .analysis_cache/
 reference_audio/
 Screenshots/
+local_audit_artifacts/
+local_audit_artifacts/**
+_archive_obsolete_*/
+_archive_obsolete_*/**
+temporary_cursor_outputs/
+
+# Workbook and audit artifacts (local/generated)
+compiled_density_metrics*.xlsx
+compiled_density_metrics_research*.xlsx
+spectral_analysis.xlsx
+audit_*.json
+release_acceptance_bundle_*/
+
+# Audio corpora / media artifacts (local input data)
+*.wav
+*.aif
+*.aiff
+*.mp3
+*.flac
 
 # OS
 .DS_Store
 Thumbs.db
+.ipynb_checkpoints/
 
 # Sample / one-off exports (regenerate from code or keep only Markdown source)
 output_example*/

API_REFERENCE.md

@@ -4,7 +4,7 @@
 **Package version:** 3.7.0 (`soundspectranalyse` in `pyproject.toml`)  
 **Last updated:** May 2026
 
-**Scope:** This page summarises the **classes and modules used by the active pipeline** (`proc_audio.AudioProcessor`, `density.py`, `compile_metrics`, orchestrator). The **primary** public density/fatness scalar for compiled workbooks is **`effective_partial_density`** (participation-ratio on the effective-component power vector — see `docs/DENSITY_EXPORT_SCHEMA.md`). Older metric names (**Density Metric**, **Combined Density Metric**, **Spectral Density Metric**, **Filtered Density Metric**, **Weighted Combined Metric**, **R_norm**, **P_norm**, **D_agn**, **D_harm**) appear on per-note **`Legacy_Density_Metrics`**, **`Legacy_Compatibility`**, and **`Diagnostic_Metrics`**; they are **not** the canonical `Density_Metrics` contract. Research-only **`density_weighted_sum_cdm_mean`** is documented in **`docs/DENSITY_EXPORT_SCHEMA.md`** §R.
+**Scope:** This page summarises the **classes and modules used by the active pipeline** (`proc_audio.AudioProcessor`, `density.py`, `compile_metrics`, orchestrator). The current final note-density architecture uses **`final_note_density_salience_weighted`** (primary final scalar) and **`final_note_density_count_based`** (control scalar). **`effective_partial_density`** remains a secondary/diagnostic participation descriptor, not the final note-density scalar. Older metric names (**Density Metric**, **Combined Density Metric**, **Spectral Density Metric**, **Filtered Density Metric**, **Weighted Combined Metric**, **R_norm**, **P_norm**, **D_agn**, **D_harm**) appear on per-note **`Legacy_Density_Metrics`**, **`Legacy_Compatibility`**, and **`Diagnostic_Metrics`**; they are **not** the canonical final-density contract. Research-only **`density_weighted_sum_cdm_mean`** is legacy opt-in and documented in **`docs/DENSITY_EXPORT_SCHEMA.md`** §R.
 
 ---
 

CHANGELOG.md

@@ -32,7 +32,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Added
 
 - **`Legacy_Density_Metrics` (per-note export, default ON):** every **`spectral_analysis.xlsx`** now includes a dedicated sheet with **`Density Metric`**, **`Spectral Density Metric`**, **`Filtered Density Metric`**, **`Combined Density Metric`**, and **`spectral_masking_enabled`** (`False` — no v5 masking GUI in v6). Stage 2 **`read_excel_metrics`** merges this sheet so **`Weighted Combined Metric`** is recomputed from real SDM/FDM on **`Diagnostic_Metrics`** / **`Legacy_Compatibility`**, not from zero placeholders.
-- **Research workbook (`compiled_density_metrics_research.xlsx`):** **`Spectral_Density_Metrics`** includes **`Combined Density Metric`**, derived **`density_weighted_sum_cdm_mean`** = \((\texttt{density\_weighted\_sum} + \texttt{Combined Density Metric}) / 2\), soft column highlights (blue / yellow / lavender), and merge from compiled **`Legacy_Compatibility`**. Tests: **`tests/test_legacy_density_export.py`**, extended **`tests/test_research_density_export.py`**.
+- **Research workbook (`compiled_density_metrics_research.xlsx`) historical note:** this changelog entry describes the behavior at the time of that release. In the current contract, **`Combined Density Metric`** remains legacy-only (not primary `Spectral_Density_Metrics`), and **`density_weighted_sum_cdm_mean`** is legacy opt-in only.
 
 ### Fixed
 

README.md

@@ -1,6 +1,18 @@
 # SoundSpectrAnalyse
 
-Spectral analysis for acoustic research. **Canonical publication pipeline:** **`proc_audio.AudioProcessor`** (Stage 1) writes per-note **`spectral_analysis.xlsx`** plus standard PNGs (**`spectrogram.png`**, two **semantically distinct** component pies — linear **amplitude-mass** vs **energy-ratio** — and a legacy-alias **`component_energy_pie.png`**; see **`docs/CANONICAL_PIPELINE_AND_EXPORT_SEMANTICS.md`**); **`compile_metrics.compile_density_metrics_with_pca`** (Stage 2) builds **`compiled_density_metrics.xlsx`** with multi-sheet exports (`Density_Metrics`, `Canonical_Metrics`, `Diagnostic_Metrics`, `Debug_Counts`, …). The primary public spectral-fatness scalar on **`Density_Metrics`** is **`effective_partial_density`**.
+Spectral analysis for acoustic research. **Canonical publication pipeline:** **`proc_audio.AudioProcessor`** (Stage 1) writes per-note **`spectral_analysis.xlsx`** plus standard PNGs (**`spectrogram.png`**, two **semantically distinct** component pies — linear **amplitude-mass** vs **energy-ratio** — and a legacy-alias **`component_energy_pie.png`**; see **`docs/CANONICAL_PIPELINE_AND_EXPORT_SEMANTICS.md`**); **`compile_metrics.compile_density_metrics_with_pca`** (Stage 2) builds **`compiled_density_metrics.xlsx`** with multi-sheet exports (`Density_Metrics`, `Canonical_Metrics`, `Diagnostic_Metrics`, `Debug_Counts`, …). For note body/thickness analysis, use **`spectral_body_thickness_index`**; **`effective_partial_density`** remains an effective-component participation descriptor.
+
+Current accepted final-density architecture:
+- primary final metric: `final_note_density_salience_weighted`
+- control metric: `final_note_density_count_based`
+- canonical mode defaults: `his_weighted`, `wH=1.0`, `wI=0.5`, `wS=0.25`, threshold `-45 dB`, ceiling `5000 Hz`
+
+Canonical processing chain:
+`GUI/Orchestrator config -> Stage 1 per-note spectral analysis -> Stage 2 compile -> Stage 3 research export -> Dashboard/Charts/Metadata`.
+
+Legacy warning:
+- `density_metric_raw`, `density_weighted_sum`, `Combined Density Metric`, and related legacy fields are not the final note-density definition.
+- fallback f0 (`nominal_fallback_used_not_acoustically_verified`) is not acoustic verification.
 
 Optional **batch preprocessing** (`batch_audio_analyzer` / `super_audio_analyzer`) may supply **`batch_summary.xlsx`** for empirical **H+I+S** profiles and **H/(H+I)** model coefficients; it is **not** required for the canonical chain above. Legacy Tk / PyQt entry points remain ancillary.
 
@@ -36,6 +48,11 @@ pip install --upgrade --force-reinstall -r requirements-pins.txt
 | Document | Purpose |
 |----------|---------|
 | **[docs/CANONICAL_PIPELINE_AND_EXPORT_SEMANTICS.md](docs/CANONICAL_PIPELINE_AND_EXPORT_SEMANTICS.md)** | **Normative** pipeline, f0, harmonics, nonharmonics, subfundamental, Debug_Counts, missing metrics, audit CLI. |
+| **[docs/TECHNICAL_MANUAL.md](docs/TECHNICAL_MANUAL.md)** | Complete technical manual for the current final-density architecture (formulas, pipeline, GUI options, workbook schema, interpretation, limitations). |
+| **[docs/QUICK_GUIDE.md](docs/QUICK_GUIDE.md)** | User quick-start: what to run, recommended defaults, which metrics to use, common pitfalls. |
+| **[docs/TUTORIAL.md](docs/TUTORIAL.md)** | Step-by-step tutorials for default, harmonic-only, weighted H/I/S, clarinet/cello comparisons, and validity checks. |
+| **[docs/FINAL_ACCEPTANCE_REPORT.md](docs/FINAL_ACCEPTANCE_REPORT.md)** | Final acceptance evidence (population, formula checks, regression gate, release decision). |
+| **[docs/GUI_OPTION_EFFECT_AUDIT.md](docs/GUI_OPTION_EFFECT_AUDIT.md)** | GUI wiring/effect audit for mode, weights, threshold, ceiling, metadata and propagation checks. |
 | **[docs/CURRENT_DOCUMENTATION_INDEX.md](docs/CURRENT_DOCUMENTATION_INDEX.md)** | What is safe to cite vs legacy vs archived. |
 | **[docs/DOCUMENTATION_AUDIT_REPORT.md](docs/DOCUMENTATION_AUDIT_REPORT.md)** | 2026-05-13 documentation audit register. |
 | **[docs/MATHEMATICAL_FORMALISATION_VERIFICATION_REPORT_FIRST_PASS.md](docs/MATHEMATICAL_FORMALISATION_VERIFICATION_REPORT_FIRST_PASS.md)** | LaTeX formalisation of six core `density.py` metrics (read-only vs code). |
@@ -48,7 +65,7 @@ pip install --upgrade --force-reinstall -r requirements-pins.txt
 | **`docs/DENSITY_EXPORT_SCHEMA.md`** | **Authoritative** export schema: `Density_Metrics`, `Per_Note_Processing_Metadata`, dissonance/PCA separation, redaction notes. |
 | **`docs/BATCH_ANALYSIS_AUDIT.md`** | Batch row semantics, H+I+S validation, model weights **H/(H+I)** (optional Phase 1). |
 | **`docs/BATCH_ANALYSIS_FIELD_MAP.md`** | Short field map for `batch_summary.xlsx` and orchestrator handoff. |
-| [TECHNICAL_MANUAL.md](TECHNICAL_MANUAL.md) | Long-form architecture (includes **historical** sections; canonical export semantics in **`docs/`**). |
+| [TECHNICAL_MANUAL.md](TECHNICAL_MANUAL.md) | Legacy root manual retained for historical compatibility; use **`docs/TECHNICAL_MANUAL.md`** as current technical reference. |
 | [TESTING.md](TESTING.md) | Pytest policy, slow-marker contract, pipeline invariants, **formula-validation** command. |
 | [QUICK_START_ORCHESTRATOR.md](QUICK_START_ORCHESTRATOR.md) | CLI examples for **`run_orchestrator.py`**. |
 | [ORCHESTRATOR_INTEGRATION_GUIDE.md](ORCHESTRATOR_INTEGRATION_GUIDE.md) | Optional preprocessing → main analysis integration. |
@@ -105,7 +122,9 @@ The research workbook is written for **Microsoft Excel compatibility**: it does
 
 The full **`compiled_density_metrics.xlsx`** remains the complete technical and audit export; **`compiled_density_metrics_research.xlsx`** is the recommended workbook for analysis, plotting, and thesis-ready tables.
 
-On **`Spectral_Density_Metrics`**, the research export adds **`density_weighted_sum_cdm_mean`** = \((\texttt{density\_weighted\_sum} + \texttt{Combined Density Metric}) / 2\) and applies soft column highlights (blue / yellow / lavender) to **`density_weighted_sum`**, **`Combined Density Metric`**, and that mean — for side-by-side reading only; normative definitions are in **`docs/DENSITY_EXPORT_SCHEMA.md`** §R.
+On **`Spectral_Density_Metrics`**, the research export keeps **`density_metric_raw`** as an explicitly diagnostic, energy-weighted component sum (`D_H*w_H + D_I*w_I + D_S*w_S`) and does **not** export **`density_weighted_sum_cdm_mean`** by default.  
+**`Combined Density Metric`** is legacy-only and exported on **`Legacy_Compatibility`**, not as a primary `Spectral_Density_Metrics` metric.
+If you need the deprecated editorial blend **`density_weighted_sum_cdm_mean`**, pass **`--include-legacy-cdm-mean`** explicitly; it is not dimensionally/acoustically valid as a final scalar.
 
 **Per-note legacy sheet (Stage 1):** each **`spectral_analysis.xlsx`** also writes **`Legacy_Density_Metrics`** (SDM, FDM, CDM, `Density Metric`) so compile can populate **`Weighted Combined Metric`** on diagnostic sheets. v6 does **not** restore the v5 spectral-masking checkbox; masking stays off in the physical workflow.
 

TESTING.md

@@ -134,6 +134,14 @@ pytest tests/test_research_density_export.py tests/test_post_compile_research_ex
 
 The suite includes a **zip** check that the output has **no** `xl/tables/*.xml` parts (Excel repair regression guard), tests for **`density_weighted_sum_cdm_mean`** and column highlights, and **`tests/test_legacy_density_export.py`** for **`Legacy_Density_Metrics`** merge / `_build_legacy_density_metrics_row`.
 
+`density_weighted_sum_cdm_mean` is now legacy opt-in (`--include-legacy-cdm-mean`), and tests cover both default omission and explicit inclusion.
+
+After changes to acoustic density descriptors or f0 provenance, also run:
+
+```bash
+pytest tests/test_acoustic_density_constructs.py tests/test_f0_canonical_density_regression.py -v
+```
+
 After changes to **`proc_audio`** legacy export or **`compile_metrics.read_excel_metrics`**:
 
 ```bash