[codex] Improve report layout and backup coverage

.gitignore

@@ -75,6 +75,9 @@ ipython_config.py
 __pypackages__/
 
 backups/
+reports/
+unifi/backups/
+unifi/reports/
 meraki_backup_*/
 meraki_backup_sample_*/
 */report.pdf

README.md

@@ -11,6 +11,7 @@ A reporting pipeline that collects Meraki org data, generates network health and
 | `ollama_review.py` | Optional local LLM review stage |
 | `python -m reporting` | Direct report generation from existing backup data |
 | `report_generator.py` | Compatibility wrapper for report generation |
+| `report_inventory.py` | Validates the expected latest report deliverables after generation |
 | `run.sh` | Full pipeline orchestrator |
 | `legacy/` | Original MX baseline scripts (reference only) |
 | `docs/cis-meraki-reference.md` | CIS Controls to Meraki reference mapping |
@@ -51,6 +52,7 @@ Generate a demo report from sanitized fixtures without Meraki API access:
 
 ```bash
 ./run.sh --demo-report --no-open
+./run.sh --demo-report --fixed-now 2026-05-02T21:30:00 --no-open
 ```
 
 Optional — specify a local Ollama model for AI-enhanced recommendations:
@@ -68,22 +70,41 @@ ollama pull gemma4:e2b
 
 ## Output
 
-All output is written to `backups/<org>/` (gitignored):
+`./run.sh` keeps raw Meraki backup data in `backups/<org>/` and writes generated
+shareable reports to `reports/` (both gitignored). By default, `./run.sh` runs
+the full pipeline: Meraki query, backup, recommendation merge, optional AI review,
+report generation, and a final deliverable inventory check.
 
 - `recommendations.md` — per-org findings and recommendations
-- `SITE_NAME_Complete_Report_YYYY-MM-DD.html` / `.pdf` — named full report for sharing
-- `SITE_NAME_Executive_Summary_Report_YYYY-MM-DD.html` / `.pdf` — named executive summary
-- `SITE_NAME_Backup_Settings_Report_YYYY-MM-DD.html` / `.pdf` — named backup settings report
-- `report.html` / `report.pdf` — compatibility aliases for older scripts
 - `backups/master_recommendations.md` — combined across all orgs
 - `backups/recommendations_ai_enhanced.md` — LLM-reviewed version
+- `reports/<org>/<timestamp>/SITE_NAME_Complete_Report_YYYY-MM-DD.pdf` — run-specific full report
+- `reports/<org>/<timestamp>/SITE_NAME_Executive_Summary_Report_YYYY-MM-DD.pdf` — run-specific executive summary
+- `reports/<org>/<timestamp>/SITE_NAME_Backup_Settings_Report_YYYY-MM-DD.pdf` — run-specific backup settings report
+- `reports/<org>/<timestamp>/SITE_NAME_Battery_Backup_Pricing_Calculation_Report_YYYY-MM-DD.pdf` — run-specific UPS runtime and pricing report
+- `reports/<org>/<timestamp>/SITE_NAME_AP_Spectrum_Report_YYYY-MM-DD.pdf` — run-specific AP spectrum and interference report
+- `reports/<org>/<timestamp>/SITE_NAME_UPS_Switch_Power_Plan_Report_YYYY-MM-DD.json` — run-specific UPS sizing data
+- `reports/latest/<org>/report.pdf` — compatibility alias for the latest full report
+- `reports/latest/report_inventory.json` — generated manifest of latest report deliverables and file sizes
+- `reports/latest/index.html` — generated report index with links to each latest deliverable
+
+By default `run.sh` passes `--pdf-only`, so generated HTML is removed after PDFs
+are rendered. Use `./run.sh --keep-html` when HTML inspection is useful.
+Direct `python3 -m reporting` remains backward-compatible and writes reports into
+each `backups/<org>/` directory unless `--reports-dir` or `--output-dir` is used.
 
 ## Optional Pricing Input
 
 To enable the Hardware Cost & Refresh Plan section, create a `pricing.json` at the repo root
 or within a specific org backup directory. See `pricing.json.example` for the expected shape.
 Set `unit_cost` and optional `replacement_cycle_years` per model.
 
+The UniFi migration section also reads `reporting/reference/pricing_reference.json`, which
+contains maintained public UniFi planning prices, product source URLs, UI Care add-ons, and
+Meraki-to-UniFi model-family mappings. Use an org-local `pricing.json` whenever reseller,
+E-rate, Meraki, support, optics, or professional-services pricing needs to override the
+public planning reference.
+
 ## Requirements
 
 Install dependencies:
@@ -118,10 +139,21 @@ Run the script entrypoint against existing backups:
 
 ```bash
 python3 -m reporting
+python3 -m reporting --reports-dir reports --pdf-only
 python3 -m reporting --source-dir tests/fixtures --org-name "Fixture Demo Org" --output-dir backups/.demo/Fixture_Demo_Org
 ./run.sh --report-only --no-ai-review --no-open
 ```
 
+Generate deterministic fixture output for regression checks:
+
+```bash
+./run.sh --demo-report --fixed-now 2026-05-02T21:30:00 --no-open
+python3 -m reporting --source-dir tests/fixtures --org-name "Fixture Demo Org" --output-dir backups/.demo/Fixture_Demo_Org --fixed-now 2026-05-02T21:30:00
+```
+
+The same fixed clock can be set for compatible report-generation paths with
+`MERAKI_REPORT_FIXED_NOW=2026-05-02T21:30:00`.
+
 Run tests:
 
 ```bash

ROADMAP.md

@@ -6,14 +6,20 @@ This project is currently functional as a Python reporting pipeline. The immedia
 
 - `./run.sh` is the main pipeline runner.
 - Python dependencies install cleanly into `.venv`.
-- Tests pass locally: `80 passed`.
+- Tests pass locally: `115 passed`.
 - Report-only generation works from existing `backups/`.
+- `run.sh` now separates generated report deliverables into `reports/` while leaving raw backup data in `backups/`.
 - `.env` is gitignored and should remain local because it may contain `MERAKI_API_KEY`.
 - Clean-history repository is published at `https://github.com/techmore/TM-Meraki_Baseline_Reporter.git`.
 - `legacy/` contains historical scripts that should not be run in production.
 - `docs/cis-meraki-reference.md` preserves the useful upstream CIS mapping as reference material.
 - Generated reports now include named aliases like `SITE_NAME_Complete_Report_YYYY-MM-DD.pdf`.
 - Ollama review unloads the active model after each generation pass to reduce idle RAM usage.
+- Deterministic report generation is available with `./run.sh --fixed-now ...`,
+  `python -m reporting --fixed-now ...`, or `MERAKI_REPORT_FIXED_NOW`.
+- `./run.sh` remains the full default pipeline and now validates the generated latest
+  report deliverables after report generation, including a latest report manifest
+  and static HTML index.
 
 ## Phase 1: Stabilize The Existing Python App - Complete
 
@@ -56,25 +62,113 @@ This project is currently functional as a Python reporting pipeline. The immedia
   - full API collection
   - report-only from existing backups
   - fixture/demo report generation
-- Improve AI review controls:
-  - default low-RAM model
-  - explicit model override
-  - no-AI mode for deterministic runs
-- Keep report rendering deterministic enough that tests can catch regressions.
+- ~~Improve AI review controls:~~
+  - ~~default low-RAM model~~
+  - ~~explicit model override~~
+  - ~~no-AI mode for deterministic runs~~
+- ~~Keep report rendering deterministic enough that tests can catch regressions.~~
+- ~~Increase table-of-contents density and make TOC titles link to report sections.~~
+- ~~Add report page furniture:~~
+  - ~~header with `TM Meraki Baseline`~~
+  - ~~page `current / total` footer~~
+  - ~~release number based on the report release date~~
+  - ~~end-of-report page~~
+- ~~Fix switch port issue classification so disconnected/unused ports are not reported as issues.~~
+- ~~Improve switch identification in issue tables by showing switch labels alongside serial numbers.~~
+- ~~Investigate why Client Analysis is blank for current backups and add fallback rendering from `clients_overview.json`.~~
+- ~~Investigate blank Switch Deep Dive sections and improve fallback messaging when port telemetry is missing.~~
+- ~~Increase switch deep-dive table density so the wide port table fits PDF pages.~~
+- ~~Add firmware status/current-vs-available rendering from Meraki firmware upgrade data.~~
+- ~~Highlight EOL/EOS inventory: red when end of support is within 2 years, yellow when announced farther out.~~
+- ~~Further compress switch deep-dive table font, padding, and badge density for PDF fit.~~
+- ~~Replace heuristic UniFi comparison pricing with maintained JSON-backed pricing/equivalent references for Meraki and UniFi.~~
+- ~~Add Meraki hardware capability data, including PoE budgets, from a maintained JSON reference instead of estimates.~~
+- ~~Review the proposed K-12 VLAN structure and add it as a supplemental/reference section if it fits the report audience.~~
+- ~~Clean up completed-report quality issues: suppress benign mesh 404s, collapse disabled default SSIDs, remove empty AP model cells, fix 100 Gbps speed labeling, filter disconnected deep-dive port badges, and avoid false "no significant issues" messages.~~
+- ~~Replace unreliable wireless-only client collection with network-wide client collection and report wired/wireless client detail coverage.~~
+- ~~Separate generated report deliverables into `reports/` and keep `backups/` focused on raw collection data.~~
+- ~~Add PDF-only output mode so routine runs do not retain generated HTML unless requested.~~
+- ~~Add a final report inventory check so missing generated deliverables fail the run visibly.~~
+- ~~Write `reports/latest/report_inventory.json` so the generated report set can be audited without browsing folders.~~
+- ~~Write `reports/latest/index.html` as a static report index with links to each latest deliverable.~~
 
 ## Phase 5: Optional Interfaces
 
 - Do not rewrite to npm unless there is a concrete need for a web UI or Node deployment.
 - If desired later, add a minimal `package.json` as a command wrapper only.
 - Keep Python as the source of truth for Meraki collection, report generation, and tests.
 
+## Phase 6: UniFi / Ubiquiti Reporting - Started
+
+- ~~Add a separate `./unifi/run.sh` runner so UniFi work does not regress the
+  Meraki pipeline.~~
+- ~~Support both official Site Manager API collection and local UniFi Network
+  Application Integration API collection.~~
+- ~~Save raw UniFi JSON backups separately under `unifi/backups/`.~~
+- ~~Generate a first-pass UniFi baseline report under `unifi/reports/`.~~
+- ~~Treat local Network Application endpoint gaps as reportable coverage
+  findings while we learn the exact controller version and API surface.~~
+- ~~Add saved site profiles in `unifi/.env` and `./unifi/run.sh --all-sites`
+  for multi-site runs.~~
+- ~~Write a top-level UniFi multi-site report index for saved profile runs.~~
+- ~~Add per-profile network size and coverage metrics to the UniFi multi-site
+  manifest/index.~~
+- ~~Write UniFi report inventory data and a static `index.html` for generated
+  outputs.~~
+- ~~Improve UniFi executive summary language once more live sites are captured.~~
+- ~~Document UniFi interface telemetry coverage so reports distinguish advertised
+  port/radio capability flags from detailed per-port/per-radio metrics.~~
+- ~~Probe likely UniFi port/radio telemetry endpoints during collection and save
+  structured coverage evidence in the backup/report.~~
+- ~~Add a UniFi configuration backup completeness matrix showing captured,
+  captured-empty, and unsupported endpoint coverage.~~
+- ~~Split UniFi per-device telemetry probes by sampled AP, switch, and gateway
+  roles so future exposed endpoints can be attributed to the right hardware.~~
+- ~~Clarify UniFi hardware planning so retained active gear is not counted as
+  unpriced refresh scope, and summarize refresh/retain/excluded actions.~~
+- ~~Promote high client concentration on one AP/switch into UniFi executive
+  risks, priorities, and implementation planning.~~
+- ~~Promote flat DEFAULT client access policy usage into UniFi executive,
+  security baseline, and implementation planning sections.~~
+- ~~Promote missing UniFi subnet/gateway/DHCP fields into executive,
+  confidence, security baseline, and implementation planning sections.~~
+- ~~Add a UniFi backup completion action plan that ranks missing telemetry,
+  address-plan, WAN, DNS, firewall, and optional endpoint evidence.~~
+- Add deeper UniFi switch/AP port and radio telemetry when the controller API
+  exposes it.
+
+## Phase 7: Codebase Audit and Lean Enhancements - Planned
+
+Goal: audit the working Meraki and UniFi reporting codebase for improvements
+that reduce maintenance burden, improve report reliability, and make future
+enhancements safer without disrupting the default `./run.sh` and
+`./unifi/run.sh` workflows.
+
+- Map the current pipeline modules, generated artifacts, raw backup locations,
+  and test coverage so cleanup work does not regress report generation.
+- Review `run.sh`, `unifi/run.sh`, `reporting/`, `unifi/`, reference JSON, and
+  tests for duplicated logic, overly large functions, weak boundaries, stale
+  compatibility paths, and low-risk extraction opportunities.
+- Identify report-generation quality risks, especially PDF layout pressure,
+  overly wide tables, brittle HTML string assembly, missing fixture coverage,
+  and places where unavailable API fields could be mistaken for network issues.
+- Audit API collection and backup handling for clear separation between
+  customer-specific data, generated reports, reusable references, and source
+  code.
+- Produce a prioritized audit summary with `do now`, `defer`, and `do not
+  change` categories before broad refactors.
+- Implement only surgical cleanup after the audit: small extractions, stronger
+  tests, clearer names, dead-code removal, and documentation updates that keep
+  Meraki and UniFi report output behavior stable.
+
 ## Release Checklist
 
 - Run `./install.sh`.
 - Run `.venv/bin/python -m pytest -q`.
 - Run `./run.sh --report-only --no-ai-review --no-open`.
 - Check `git status --short`.
 - Confirm `.env` and `backups/` are not staged.
+- Confirm `reports/` is not staged unless a sanitized sample is intentionally added.
 - Confirm generated or customer-specific report files are not staged unless sanitized.
 - Commit the surgical changes.
 - Push to `https://github.com/techmore/TM-Meraki_Baseline_Reporter.git` after verification.