Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
137 changes: 133 additions & 4 deletions FLOWCHART.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,22 @@

<p align="center">
<img src="https://img.shields.io/badge/Owner-CEO-0A66C2?style=for-the-badge" alt="Owner"/>
<img src="https://img.shields.io/badge/Version-1.2-555?style=for-the-badge" alt="Version"/>
<img src="https://img.shields.io/badge/Effective-2026--04--20-success?style=for-the-badge" alt="Effective Date"/>
<img src="https://img.shields.io/badge/Version-1.3-555?style=for-the-badge" alt="Version"/>
<img src="https://img.shields.io/badge/Effective-2026--05--06-success?style=for-the-badge" alt="Effective Date"/>
<img src="https://img.shields.io/badge/Review-Quarterly-orange?style=for-the-badge" alt="Review Cycle"/>
</p>

**📋 Document Owner:** CEO | **📄 Version:** 1.2 | **📅 Last Updated:** 2026-04-20 (UTC)
**🔄 Review Cycle:** Quarterly | **⏰ Next Review:** 2026-07-20
**📋 Document Owner:** CEO | **📄 Version:** 1.3 | **📅 Last Updated:** 2026-05-06 (UTC)
**🔄 Review Cycle:** Quarterly | **⏰ Next Review:** 2026-08-06
**🏢 Owner:** Hack23 AB (Org.nr 5595347807) | **🏷️ Classification:** Public

> **🆕 What changed since last review (v1.2 → v1.3, 2026-05-06):**
> - Added **Political Intelligence Generation Flow** (§16): documents `generate-political-intelligence.ts` pipeline with catalog, daily-streams, i18n, and render stages.
> - Added **Analysis Gate Validation Flow** (§17): documents the 9-check (1–9b) analysis gate in `scripts/agentic/analysis-gate.ts` validating 23 artifacts across Families A–D.
> - Added **Parliamentary Data Download Flow** (§18): documents `download-parliamentary-data.ts`, `fetch-voting-records.ts`, and `fetch-calendar.ts` pipelines.
> - Updated Process Inventory tables with political intelligence, analysis gate, and parliamentary data processes.
> - Package version aligned to 0.8.76; 51 workflow files (22 standard + 14 agentic .md + 14 compiled .lock.yml + 1 README).
>
> **🆕 What changed since last review (v1.1 → v1.2, 2026-04-20):**
> - 📈 Added **IMF** to the agentic news-pipeline fan-out as a third primary economic data source alongside SCB and World Bank, per [ADR 0001](docs/adr/0001-adopt-imf-data-alongside-world-bank.md). IMF is reached via the **IMF TypeScript client `scripts/imf-client.ts` invoked through the bash tool** — *pure-TS, no MCP* — so the MCP server count is unchanged.
>
Expand Down Expand Up @@ -345,6 +352,9 @@ flowchart TD
| 4 | User Journey | Page visit | < 3s | On demand |
| 5 | Security Scanning | Code change | 5-10 min | Per commit |
| 6 | Multi-Language | Content creation | 15-30 min | Per article |
| 7 | Political Intelligence | Prebuild chain | 2-4 min | Per build |
| 8 | Analysis Gate | Pre-article | 1-2 min | Per article |
| 9 | Parliamentary Data | Cron daily | 5-10 min | Daily |

---

Expand Down Expand Up @@ -881,6 +891,122 @@ flowchart TD

---

## 16. 🧠 Political Intelligence Generation Flow

```mermaid
flowchart TD
TRIGGER[generate-political-intelligence.ts] --> CATALOG[catalog.ts<br/>Load article-types.json registry]
CATALOG --> STREAMS[daily-streams.ts<br/>Identify active streams per horizon]
STREAMS --> HORIZON{Horizon Stratification}
HORIZON --> H72[T+72h Short-term]
HORIZON --> H7D[T+7d Weekly]
HORIZON --> H30D[T+30d Monthly]
HORIZON --> H90D[T+90d Quarterly]
HORIZON --> H365D[T+365d Annual]
HORIZON --> H1460D[T+1460d Election cycle]

H72 --> I18N[i18n/<br/>14-language translation]
H7D --> I18N
H30D --> I18N
H90D --> I18N
H365D --> I18N
H1460D --> I18N

I18N --> RENDER[render/<br/>Template-based HTML rendering]
RENDER --> OUTPUT[Output: political-intelligence*.html<br/>14 language variants]

style TRIGGER fill:#2196f3,color:#ffffff
style OUTPUT fill:#4caf50,color:#000000
style HORIZON fill:#ff9800,color:#000000
style CATALOG fill:#9c27b0,color:#ffffff
style STREAMS fill:#9c27b0,color:#ffffff
style RENDER fill:#9c27b0,color:#ffffff
```

---

## 17. ✅ Analysis Gate Validation Flow

```mermaid
flowchart TD
INPUT[analysis/daily/DATE/ folder] --> CHECK1{Check 1:<br/>Artifact existence<br/>23 files across Families A-D}
CHECK1 -->|Pass| CHECK2{Check 2:<br/>No stub content}
CHECK1 -->|Fail| FAILED[❌ Gate FAILED<br/>Missing artifacts]

CHECK2 -->|Pass| CHECK3{Check 3:<br/>Minimum word count}
CHECK2 -->|Fail| FAILED

CHECK3 -->|Pass| CHECK4{Check 4:<br/>Evidence citations}
CHECK3 -->|Fail| FAILED

CHECK4 -->|Pass| CHECK5{Check 5:<br/>Mermaid diagrams present}
CHECK4 -->|Fail| FAILED

CHECK5 -->|Pass| CHECK6{Check 6:<br/>Pass-2 iteration evidence}
CHECK5 -->|Fail| FAILED

CHECK6 -->|Pass| CHECK7{Check 7:<br/>Cross-references valid}
CHECK6 -->|Fail| FAILED

CHECK7 -->|Pass| CHECK8{Check 8:<br/>Data-source audit trail}
CHECK7 -->|Fail| FAILED

CHECK8 -->|Pass| CHECK9A{Check 9a:<br/>Political classification}
CHECK8 -->|Fail| FAILED

CHECK9A -->|Pass| CHECK9B{Check 9b:<br/>Agency evidence}
CHECK9A -->|Fail| FAILED

CHECK9B -->|Pass| PASSED[✅ Gate PASSED<br/>Article generation proceeds]
CHECK9B -->|Fail| FAILED

FAILED --> BLOCK[Block article pipeline<br/>Return error details]

style INPUT fill:#2196f3,color:#ffffff
style PASSED fill:#4caf50,color:#000000
style FAILED fill:#f44336,color:#ffffff
style BLOCK fill:#f44336,color:#ffffff
style CHECK1 fill:#ff9800,color:#000000
style CHECK9B fill:#ff9800,color:#000000
```

---

## 18. 📥 Parliamentary Data Download Flow

```mermaid
flowchart TD
TRIGGER[download-parliamentary-data.ts] --> PROPS[Fetch Propositions<br/>from data.riksdagen.se]
TRIGGER --> MOTIONS[Fetch Motions<br/>from data.riksdagen.se]
TRIGGER --> BET[Fetch Betänkanden<br/>from data.riksdagen.se]

PROPS --> VALIDATE_P[Validate & transform]
MOTIONS --> VALIDATE_M[Validate & transform]
BET --> VALIDATE_B[Validate & transform]

VALIDATE_P --> DATA_DIR[Output: data/ directory]
VALIDATE_M --> DATA_DIR
VALIDATE_B --> DATA_DIR

TRIGGER --> VOTING[fetch-voting-records.ts<br/>Download voting records]
VOTING --> VOTE_DATA[Voting data validated]
VOTE_DATA --> DATA_DIR

TRIGGER --> CALENDAR[fetch-calendar.ts<br/>Download calendar events]
CALENDAR --> CAL_DATA[Calendar data validated]
CAL_DATA --> DATA_DIR

DATA_DIR --> READY[Parliamentary data ready<br/>for analysis pipeline]

style TRIGGER fill:#2196f3,color:#ffffff
style READY fill:#4caf50,color:#000000
style DATA_DIR fill:#ff9800,color:#000000
style VOTING fill:#9c27b0,color:#ffffff
style CALENDAR fill:#9c27b0,color:#ffffff
```

---

## Updated Process Inventory

| # | Process | Trigger | Duration | Frequency | Security Controls |
Expand All @@ -897,6 +1023,9 @@ flowchart TD
| 10 | Data Validation | Per data fetch | 1-2 min | Per fetch | 9-stage validation pipeline |
| 11 | Content Integrity | Per content | < 1 min | Per article | Git signatures, Sigstore (build artifacts) |
| 12 | Runner Hardening | Per job | Continuous | Per job | iptables, egress audit |
| 13 | Political Intelligence | Prebuild chain | 2-4 min | Per build | HTMLHint, schema validation |
| 14 | Analysis Gate | Pre-article | 1-2 min | Per article | 9-check validation (23 artifacts) |
| 15 | Parliamentary Data | Cron daily | 5-10 min | Daily | Data validation, freshness check |

---

Expand Down
166 changes: 161 additions & 5 deletions STATEDIAGRAM.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,20 +11,27 @@

<p align="center">
<img src="https://img.shields.io/badge/Owner-CEO-0A66C2?style=for-the-badge" alt="Owner"/>
<img src="https://img.shields.io/badge/Version-1.1-555?style=for-the-badge" alt="Version"/>
<img src="https://img.shields.io/badge/Effective-2026--04--20-success?style=for-the-badge" alt="Effective Date"/>
<img src="https://img.shields.io/badge/Version-1.2-555?style=for-the-badge" alt="Version"/>
<img src="https://img.shields.io/badge/Effective-2026--05--06-success?style=for-the-badge" alt="Effective Date"/>
<img src="https://img.shields.io/badge/Review-Quarterly-orange?style=for-the-badge" alt="Review Cycle"/>
</p>

**📋 Document Owner:** CEO | **📄 Version:** 1.1 | **📅 Last Updated:** 2026-04-20 (UTC)
**🔄 Review Cycle:** Quarterly | **⏰ Next Review:** 2026-07-20
**📋 Document Owner:** CEO | **📄 Version:** 1.2 | **📅 Last Updated:** 2026-05-06 (UTC)
**🔄 Review Cycle:** Quarterly | **⏰ Next Review:** 2026-08-06
**🏢 Owner:** Hack23 AB (Org.nr 5595347807) | **🏷️ Classification:** Public

> **🆕 What changed since last review (v1.1 → v1.2, 2026-05-06):**
> - Added **analysis artifact lifecycle** state diagram (empty → populated → gate-checked → passed → consumed) documenting the 23-artifact analysis folder lifecycle.
> - Added **analysis gate check** state machine (check-1 through check-9b) per `scripts/agentic/analysis-gate.ts`.
> - Added **political intelligence states** (collection → processing → analysis → dissemination → feedback).
> - Anchored to current toolchain: Node.js ≥26, TypeScript 6.0.3, Vite 8.0.10, Vitest 4.1.5, Cypress 15.14.2.
> - Verified all existing state diagrams: news article lifecycle, agentic workflow states, CIA data refresh, release pipeline, incident state machine, npm package lifecycle.
>
> **🆕 What changed since last review (v1.0 → v1.1, 2026-04-20):**
> - Refreshed lifecycle state machines for: (a) **news articles** (draft → safe-output validation → translator fan-out across 14 languages → merged → published → superseded/retired); (b) **agentic workflow runs** (queued → running → safe-output-validation → reviewer-PR → merged/rolled-back); (c) **CIA data refresh** (scheduled → download → schema-validate → diff → auto-PR → merge → deploy); (d) **release pipeline** (tag → build → provenance-attest → npm publish → S3 sync → CloudFront invalidation); (e) **translation states** across 14 locales including RTL (HE, AR).
> - Added state diagrams for **npm package lifecycle** (unpublished → published-with-provenance → deprecated → security-advisory → superseded) aligned with `End-of-Life-Strategy.md`.
> - Added **incident state machine**: detected → triaged → contained → eradicated → recovered → post-mortem, mapped to [Incident_Response_Plan](https://github.com/Hack23/ISMS-PUBLIC/blob/main/Incident_Response_Plan.md).
> - Anchored to current toolchain: Node.js ≥25, TypeScript 6.0.3, Vite 8.0.9, Vitest 4.1.4, Cypress 15.14.0 (optional), Playwright 1.59.1.
> - Anchored to current toolchain: Node.js ≥26, TypeScript 6.0.3, Vite 8.0.10, Vitest 4.1.5, Cypress 15.14.2, Playwright 1.52.0.
> - Compliance mapping: ISO 27001:2022 A.5.24/A.5.26, NIST CSF 2.0 RS.AN/RS.MI/RC.RP, CIS Controls v8.1 #17, NIS2 Art. 21, EU CRA Annex I §2.

---
Expand Down Expand Up @@ -930,6 +937,155 @@ Action Required:

---

## 6A. 📦 Analysis Artifact Lifecycle States

**📊 Data Focus:** 23 analysis artifacts (Families A-D) produced by every agentic news workflow.

**🔄 Process Focus:** Shows artifact folder lifecycle from creation through consumption by article rendering.

**🤖 AI Integration:** Artifacts are produced by Copilot agents and validated by `scripts/agentic/analysis-gate.ts`.

```mermaid
stateDiagram-v2
[*] --> Empty

Empty --> Populating: Agentic workflow starts writing artifacts

Populating --> Populated: All 23 artifacts written
Populating --> Incomplete: Timeout or agent error

Populated --> GateChecking: analysis-gate.ts invoked

GateChecking --> GatePassed: All checks 1-9b pass
GateChecking --> GateFailed: One or more checks fail

GateFailed --> Populating: Agent retries failed artifacts

GatePassed --> Consumed: render-articles.ts reads artifacts

Consumed --> Archived: Article published, artifacts retained
Archived --> [*]

Incomplete --> [*]: Workflow terminates with error

note right of Empty
analysis/daily/YYYY-MM-DD/type/ folder
23 expected files (Families A-D)
end note

note right of GateChecking
Check 1: Artifact existence (23 files)
Check 2: No stub placeholders
Check 3: Minimum word count
Check 4: Evidence citations
Check 5: Mermaid diagrams with colour
Check 6: Pass-2 evidence (revision proof)
Check 7: Cross-references
Check 8: Data-source connectivity audit
Check 9a: Political classification
Check 9b: Agency evidence (Statskontoret)
end note

note right of Consumed
aggregate-analysis.ts aggregates
render-articles.ts renders HTML
Artifacts become read-only
end note
```

### 6A.1 State Definitions

| State | Description | Entry Conditions | Exit Conditions | Typical Duration |
|-------|-------------|------------------|-----------------|------------------|
| **EMPTY** | Analysis folder created but no artifacts written | Workflow triggered | First artifact written | Seconds |
| **POPULATING** | Agent writing artifacts to folder | First artifact write | All 23 present or timeout | 5-15 minutes |
| **INCOMPLETE** | Timeout or error, fewer than 23 artifacts | Agent timeout/error | Workflow terminates | N/A |
| **POPULATED** | All 23 artifacts present in folder | 23 files written | Gate check invoked | Seconds |
| **GATE_CHECKING** | analysis-gate.ts running checks 1-9b | Populated state | Pass or fail | 5-30 seconds |
| **GATE_PASSED** | All gate checks passed | All checks pass | Render starts | Seconds |
| **GATE_FAILED** | One or more checks failed | Any check fails | Retry or terminate | Variable |
| **CONSUMED** | Artifacts read by render pipeline | render-articles.ts invoked | Article published | 1-5 minutes |
| **ARCHIVED** | Artifacts retained for audit trail | Article published | Permanent | Permanent |

---

## 6B. 🕵️ Political Intelligence Assessment States

**📊 Data Focus:** Intelligence assessment lifecycle for political analysis products.

**🔄 Process Focus:** Shows the intelligence cycle from collection through dissemination and feedback.

**🤖 AI Integration:** Horizon stratification (T+72h → T+1460d) drives assessment scope and confidence levels.

```mermaid
stateDiagram-v2
[*] --> Collection

Collection --> Processing: Raw data gathered

Processing --> Analysis: Data cleaned and structured

Analysis --> Assessment: Analytical products created

Assessment --> Dissemination: Confidence labels assigned

Dissemination --> Feedback: Published to audience

Feedback --> Collection: New requirements identified
Feedback --> [*]: Cycle complete

Collection --> Collection: Continuous monitoring

note right of Collection
Sources: Riksdag MCP (32 tools)
SCB statistics, IMF economic data
Government documents, voting records
end note

note right of Analysis
20 methodologies applied
39 analysis templates used
Horizon-specific framing:
T+72h, T+7d, T+30d, T+90d, T+365d, T+1460d
end note

note right of Assessment
Confidence levels: High/Medium/Low
WEP language ladder per horizon band
Scenario analysis (≥3 scenarios)
end note

note right of Dissemination
14 languages, RTL support
Article rendering pipeline
RSS, sitemap, indexes generated
end note
```

### 6B.1 State Definitions

| State | Description | Entry Conditions | Exit Conditions | Typical Duration |
|-------|-------------|------------------|-----------------|------------------|
| **COLLECTION** | Gathering raw political data from sources | Workflow trigger or schedule | Sufficient data gathered | 2-5 minutes |
| **PROCESSING** | Cleaning, structuring, deduplication | Raw data available | Structured data ready | 1-3 minutes |
| **ANALYSIS** | Applying methodologies and templates | Structured data ready | Analysis artifacts produced | 5-15 minutes |
| **ASSESSMENT** | Assigning confidence, creating judgments | Analysis complete | Assessment products ready | 2-5 minutes |
| **DISSEMINATION** | Publishing to audience in multiple languages | Assessment approved | Content published | 3-10 minutes |
| **FEEDBACK** | Receiving feedback, identifying new requirements | Content published | New PIRs identified | Continuous |

### 6B.2 Horizon Stratification

| Horizon | Timeframe | Scope | Confidence Band | Scenario Depth |
|---------|-----------|-------|-----------------|----------------|
| **T+72h** | 3 days | Tactical | High | 2-3 scenarios |
| **T+7d** | 1 week | Tactical/Operational | High-Medium | 3 scenarios |
| **T+30d** | 1 month | Operational | Medium | 3-4 scenarios |
| **T+90d** | Quarter | Strategic | Medium-Low | 4 scenarios |
| **T+365d** | 1 year | Strategic | Low-Medium | 4+ scenarios + wildcards |
| **T+1460d** | Election cycle | Grand strategic | Low | 4×3 coalition branches + wildcards |

---

## 7. 📊 State Metrics and Monitoring

### 7.1 Time in State (Average Duration)
Expand Down
Loading
Loading