Improve architecture, code quality, and test coverage for article.md generation pipeline (aggregate-analysis → article.md)

[pethers]

📋 Issue Type

Architecture / Code Quality / Testing

🎯 Objective

Refactor the article.md generation pipeline into a well-architected bounded-context system with strict typing, clear interfaces, comprehensive tests, and proper separation of concerns. This covers everything from analysis artifacts to the canonical article.md output.

📊 Current State

The article.md generation pipeline involves:

• scripts/aggregate-analysis.ts — main orchestrator (turns analysis folder into article.md)
• scripts/render-lib/aggregator/ — 9 sub-modules (aggregate, cleaning/, frontmatter, order, per-document, reader-guide, reader-guide-i18n, seo/, sources-appendix)
• scripts/analysis-reader.ts — reads analysis artifacts
• scripts/analysis-references.ts — cross-reference utilities
• scripts/validate-article.ts — post-generation validation
• scripts/validate-methodology-reflection.ts — methodology validation
• scripts/populate-analysis-data.ts — pre-population of analysis data
• scripts/statistical-claims-detector.ts — detects statistical claims for verification

Issues:

• Bounded context boundaries unclear between aggregator sub-modules
• Some modules may have overlapping responsibilities (e.g., cleaning/ vs frontmatter.ts)
• Type definitions spread across files rather than centralized interfaces
• Test coverage for aggregator sub-modules (render-lib-architecture.test.ts, render-lib-leaf-modules.test.ts) may not cover all edge cases
• Error handling patterns inconsistent across pipeline stages
• No clear pipeline abstraction connecting stages (reader → validator → aggregator → writer)

🚀 Desired State

Architecture (Bounded Contexts)

scripts/article-pipeline/
├── interfaces.ts          # All shared types/interfaces for pipeline
├── pipeline.ts            # Pipeline orchestrator (compose stages)
├── stages/
│   ├── read/              # Read analysis artifacts from filesystem
│   │   ├── artifact-reader.ts
│   │   ├── artifact-inventory.ts
│   │   └── index.ts
│   ├── validate/          # Validate completeness and quality
│   │   ├── gate-checker.ts
│   │   ├── methodology-validator.ts
│   │   ├── statistical-claims.ts
│   │   └── index.ts
│   ├── aggregate/         # Transform artifacts into article sections
│   │   ├── section-ordering.ts
│   │   ├── frontmatter-generator.ts
│   │   ├── content-cleaner.ts
│   │   ├── per-document-synthesis.ts
│   │   ├── reader-guide.ts
│   │   ├── sources-appendix.ts
│   │   └── index.ts
│   ├── enrich/            # Add cross-references, SEO, metadata
│   │   ├── cross-references.ts
│   │   ├── seo-metadata.ts
│   │   └── index.ts
│   └── write/             # Write final article.md
│       ├── markdown-writer.ts
│       └── index.ts
└── index.ts               # Public API

Code Quality

• Each stage has a clear input/output interface (typed)
• Pipeline is composable — stages can be tested independently
• Error propagation uses Result types or typed exceptions
• No any types — full strict TypeScript
• Maximum 200 lines per module (single responsibility)

Test Quality

• Each stage has dedicated test file with:
  • Happy path tests with real fixture data
  • Edge cases (empty analysis folders, missing artifacts, malformed files)
  • Error handling tests (filesystem errors, validation failures)
  • Integration tests for full pipeline
• Test fixtures in tests/fixtures/ for reproducible article generation
• Snapshot tests for generated article.md output stability

🔧 Implementation Approach

1. Map current architecture: Document current data flow through aggregate-analysis.ts → render-lib/aggregator/
2. Define interfaces: Create shared type definitions for pipeline stages
3. Extract and refactor: Move aggregator logic into bounded-context stages
4. Add pipeline orchestrator: Composable pipeline with typed stage inputs/outputs
5. Write tests: Comprehensive unit tests for each stage + integration tests
6. Validate: Ensure generated article.md output is identical before/after refactoring
7. Update docs: Update Article-Generation.md §"How article.md Is Generated"

📚 Key Files

File	Purpose
scripts/aggregate-analysis.ts	Main aggregation orchestrator
scripts/render-lib/aggregator/ (9 files)	Aggregator sub-modules
scripts/analysis-reader.ts	Analysis artifact reader
scripts/analysis-references.ts	Cross-reference utilities
scripts/validate-article.ts	Post-generation validation
scripts/validate-methodology-reflection.ts	Methodology validation
scripts/populate-analysis-data.ts	Analysis data pre-population
scripts/statistical-claims-detector.ts	Statistical claims detection
tests/render-lib-architecture.test.ts	Existing architecture tests
tests/render-lib-leaf-modules.test.ts	Existing leaf module tests
tests/validate-article.test.ts	Existing validation tests
tests/validate-methodology-reflection.test.ts	Existing methodology tests
Article-Generation.md	System documentation
.github/prompts/04-analysis-pipeline.md	Analysis pipeline contract
.github/prompts/05-analysis-gate.md	Gate validation rules

✅ Acceptance Criteria

• Pipeline stages have explicit TypeScript interfaces for inputs/outputs
• Each bounded context (read/validate/aggregate/enrich/write) is independently testable
• No module exceeds 200 lines (split large modules)
• Zero any types in pipeline code
• Each stage has >90% line coverage in unit tests
• Integration test validates full pipeline produces correct article.md
• Snapshot tests ensure output stability across refactoring
• Article-Generation.md updated to reflect new architecture
• All existing tests pass (npm test)
• Generated article.md output is byte-identical before/after refactoring (regression-free)

🤖 Recommended Agent

code-quality-engineer — Architecture refactoring, bounded contexts, TypeScript interfaces, test coverage

🏷️ Labels

enhancement, refactor, testing, component:content-generation, news-generation, priority-high, size-xl

[pethers]

🔗 Related Issues (Architecture & Quality Improvement Series)

This issue is part of a coordinated effort to improve architecture, code quality, and test coverage across the riksdagsmonitor platform:

1. Improve architecture and quality of agentic workflows: extract embedded code, enforce bounded contexts, and add strict-typed TypeScript #2279 — Agentic workflows: extract embedded code, enforce bounded contexts
2. Resolve all ESLint warnings, improve strict typing, fix knip issues, and enhance TypeDoc/reporting configuration #2276 — ESLint warnings, strict typing, knip, TypeDoc/reporting
3. Improve architecture, code quality, and test coverage for article.md generation pipeline (aggregate-analysis → article.md) #2277 — article.md generation pipeline (aggregate-analysis) (this issue)
4. Improve architecture, code quality, and test coverage for HTML/XML/RSS/sitemap generation from article.md (render-articles pipeline) #2278 — HTML/XML/RSS/sitemap rendering pipeline (render-articles)

Dependency order: #2276 (tooling) first, then this issue (#2277) can proceed. #2278 builds on the interfaces defined here since the rendering pipeline consumes article.md output.