docs: define D1.4 two-book product architecture

.github/workflows/validate.yml

@@ -37,6 +37,9 @@ jobs:
       - name: Validate D1.3 deterministic data-quality reports
         run: node scripts/audit-data-quality.mjs --check
 
+      - name: Validate D1.4 two-book architecture
+        run: node scripts/validate-two-book-architecture.mjs
+
       - name: Require a new uNews patchnote for Pull Request changes
         if: ${{ github.event_name == 'pull_request' }}
         env:
@@ -69,6 +72,7 @@ jobs:
           node --check scripts/validate-data-provenance.mjs
           node --check scripts/validate-dataset-registry.mjs
           node --check scripts/audit-data-quality.mjs
+          node --check scripts/validate-two-book-architecture.mjs
           node --check scripts/validate-patchnote-diff.mjs
           node --check scripts/validate-screenshot-tooling.mjs
           node --check tools/screenshots/playwright.config.mjs

ROADMAP.md

@@ -31,7 +31,7 @@ See `docs/MODULARIZATION_PLAN.md`.
 - [x] Rank exact and primary symbol matches before redirect and distant matches.
 - [x] Resolve Russian aliases to their owning primary cards.
 - [x] Make all five search filters strict to their declared fields.
-- [x] Keep search filters above the expanding autocomplete panel.
+- [x] Keep search filters above the expanding autocomplete list.
 - [x] Add release-specific screenshots for the uNews publication.
 
 ## Completed: PWA update and installation v23.8.0
@@ -67,7 +67,6 @@ See `docs/MODULARIZATION_PLAN.md`.
 - [x] Require planned work, actual work, branch, commit, checks, blockers and exact next action.
 - [x] Require a pushed handoff before switching devices or pausing unfinished work.
 - [x] Make every future agent read and update the handoff at task start, pause and completion.
-- [x] Record the current signal as `READY`: no unfinished task or open Pull Request.
 
 ## Completed: unified AI/GitHub workflow v23.8.6
 
@@ -79,87 +78,81 @@ See `docs/MODULARIZATION_PLAN.md`.
 - [x] Require new screenshot provenance metadata in patchnotes.
 - [x] Require a newly added PNG/JPEG in the same Pull Request instead of reusing an older image.
 - [x] Add structural validation for `WORK_STATUS.md`.
-- [x] Synchronize stale cross-device documentation references.
 
 See `docs/AI_GITHUB_WORKFLOW.md`.
 
 ## Completed: Playwright screenshots v23.8.7
 
 - [x] Add isolated Playwright tooling outside the public runtime.
 - [x] Pin Playwright package and lockfile versions.
-- [x] Start the exact branch checkout through a local HTTP server in GitHub Actions.
 - [x] Capture desktop and mobile screenshots from real Chromium.
-- [x] Define reusable JSON scenarios for page startup, search ranking and Russian alias behavior.
-- [x] Allow only explicit safe actions instead of arbitrary scenario JavaScript.
-- [x] Require visible assertions before every capture.
-- [x] Upload screenshots, per-scenario provenance, manifest and test results as workflow artifacts.
-- [x] Keep the permanent workflow read-only and prohibit automatic commits.
-- [x] Preserve successful manifest entries across Playwright retries.
-- [x] Validate package/lock, workflow permissions, scenario coverage, artifact cleanup and runtime isolation.
-- [x] Complete a four-of-four successful Chromium run.
-- [x] Visually inspect all generated PNGs before selecting the uNews image.
-- [x] Document local and GitHub Actions invocation.
+- [x] Define reusable JSON scenarios and visible assertions.
+- [x] Upload screenshots and provenance as workflow artifacts.
+- [x] Keep the permanent workflow read-only.
 
 See `docs/SCREENSHOT_AUTOMATION.md` and `tools/screenshots/README.md`.
 
 ## Completed: D1.1 data provenance v23.8.8
 
-- [x] Recover the retained Git history of `data/divinity_code_ru.json`, `data/bd2.json`, `data/db.json` and `data/report.txt`.
-- [x] Record exact bytes, raw SHA-256, canonical JSON SHA-256, schema and ordered ID coverage.
-- [x] Separate proven facts, reasonable inferences and unknowns in `docs/DATA_PROVENANCE.md`.
-- [x] Correct the earlier byte-duplicate claim for `bd2.json` and `db.json`.
-- [x] Prove that `bd2.json` and `db.json` are byte-different but parsed/canonical-JSON equal.
-- [x] Record that the active dataset preserves IDs and core source fields while changing `aliases`, `notes` and `tags`.
-- [x] Invalidate the earlier intermediate metric that inspected nonexistent field `note` instead of `notes`.
-- [x] Add `scripts/validate-data-provenance.mjs` and integrate it into project validation.
-- [x] Keep all current data files, runtime, PWA, saved versions and archives unchanged.
+- [x] Recover retained Git history of maintained data files.
+- [x] Record bytes, raw/canonical hashes, schema and ordered IDs.
+- [x] Separate proven facts, inferences and unknowns.
+- [x] Prove that `bd2.json` and `db.json` are byte-different but canonical-JSON equal.
+- [x] Add permanent provenance validation.
+- [x] Keep data, runtime, PWA, saved versions and archives unchanged.
 
 See `docs/DATA_PROVENANCE.md`.
 
 ## Completed: D1.2 dataset registry v23.8.9
 
-- [x] Add machine-readable `data/datasets.json` with schema version 1.
-- [x] Define stable logical dataset IDs `source-divinity-code-en` and `ru-current-v1`.
-- [x] Define stable physical file IDs for both English serializations and the active runtime file.
-- [x] Select `data/bd2.json` as the canonical maintained physical serialization through a project-governance decision.
-- [x] State explicitly that the canonical choice does not prove the historical original or authoritative source edition.
-- [x] Retain `data/db.json` as an equivalent compatibility serialization.
-- [x] Record bytes, raw hashes, canonical hashes, record counts, roles, statuses and source relationships.
-- [x] Add `docs/DATASET_REGISTRY.md` with identity, reference-audit, migration and rollback rules.
-- [x] Add `scripts/validate-dataset-registry.mjs` and require it in GitHub Actions.
-- [x] Preserve all existing data files, runtime, PWA, package metadata, saved versions and archives unchanged.
-- [x] Keep the physical migration in status `planned-not-executed`.
+- [x] Add machine-readable `data/datasets.json`.
+- [x] Define stable logical and physical dataset IDs.
+- [x] Select `data/bd2.json` as canonical maintained serialization by governance decision.
+- [x] Retain `data/db.json` as equivalent compatibility serialization.
+- [x] Add registry documentation and validator.
+- [x] Keep physical migration `planned-not-executed`.
 
 See `data/datasets.json` and `docs/DATASET_REGISTRY.md`.
 
 ## Completed: D1.3 data-quality audit v23.8.10
 
-- [x] Define a four-level severity model: error, warning, review and info.
-- [x] Add structural checks for arrays, required fields, types, dates, unique/ordered IDs and registered record counts.
-- [x] Add data-hygiene and ambiguity checks for whitespace, empty values, duplicate aliases/tags, control characters, HTML-like content and normalized collisions.
-- [x] Compare `source-divinity-code-en` with `ru-current-v1` without changing either dataset.
-- [x] Confirm 4,086 unique ordered IDs and exact source/current alignment.
-- [x] Confirm zero differences in preserved fields `id`, `symbol`, `description`, `source` and `date_added`.
-- [x] Record expected changed-field counts: aliases 4,083; notes 4,086; tags 4,086.
-- [x] Generate deterministic machine-readable and human-readable reports.
-- [x] Add permanent `scripts/audit-data-quality.mjs` with write and `--check` modes.
-- [x] Require report freshness and a passing structural gate in GitHub Actions.
-- [x] Record 0 structural errors, 0 warnings and 5,022 human-review instances in five aggregated groups.
-- [x] State explicitly that heuristic review counts are not proven content errors and are not auto-fixed.
-- [x] Preserve all existing data files, runtime, PWA, package metadata, saved versions and archives unchanged.
+- [x] Define error, warning, review and info severities.
+- [x] Add deterministic structural and hygiene checks.
+- [x] Confirm 4,086 unique ordered and aligned IDs.
+- [x] Confirm zero differences in preserved fields.
+- [x] Generate deterministic machine and human reports.
+- [x] Record 0 structural errors, 0 warnings and 5,022 overlapping human-review instances.
+- [x] Keep the audit read-only and preserve all runtime/data files.
 
 See `docs/DATA_QUALITY_AUDIT.md` and `reports/data-quality-audit.md`.
 
-## Next approved work: D1.4 — two-book product architecture
+## Completed: D1.4 two-book product architecture v23.8.11
 
-- [ ] Identify and register the second source book/dataset as far as retained evidence allows.
-- [ ] Define the intended relationship between the two source books.
-- [ ] Compare separate-database switching, combined search and side-by-side comparison.
-- [ ] Define user-visible provenance for every result in every future mode.
-- [ ] Define validation, reload, cache-clearing and automatic fallback contracts before any selector implementation.
-- [ ] Decide how history, sharing and deep links preserve dataset identity.
-- [ ] Approve a migration and rollback plan before implementing a user-facing selector or merged index.
-- [ ] Keep D1.4 architecture-only unless a separate functional implementation phase is approved.
+- [x] Register stable source-work and retained-document identities as architecture metadata.
+- [x] Keep the second PDF classified as retained evidence, not a ready dataset.
+- [x] Define global record identity as `(dataset_id, record_id)`.
+- [x] Compare separate switching, federated combined search and side-by-side comparison.
+- [x] Recommend staged implementation: separate mode first, combined mode second, reviewed comparison third.
+- [x] Require visible source-work, dataset, translation/variant and source-reference provenance.
+- [x] Define dataset-aware URL, history, sharing and legacy-link migration contracts.
+- [x] Define validation, atomic activation, cache isolation and automatic fallback to `ru-current-v1`.
+- [x] Define an explicit reviewed relation-map contract without numeric-ID assumptions.
+- [x] Define migration and rollback before any functional implementation.
+- [x] Add machine-readable `docs/two-book-architecture.json` and permanent validator.
+- [x] Preserve existing data files, runtime, PWA, package metadata, saved versions and archive unchanged.
+
+See `docs/TWO_BOOK_ARCHITECTURE.md`.
+
+## Next approved work: second-book evidence and dataset preparation
+
+Before any selector or combined-search implementation:
+
+- [ ] archive exact source/edition evidence and known distribution statements for the second retained work;
+- [ ] define a separate extraction task and immutable raw-output policy;
+- [ ] create a new logical dataset ID only after actual extraction exists;
+- [ ] validate schema, required fields, local IDs, record-count policy, hashes and source references;
+- [ ] register the second dataset without replacing `ru-current-v1`;
+- [ ] approve a separate functional phase for dataset switching only after data readiness.
 
 ## D1.5 — AI-assisted translation experiment
 
@@ -172,24 +165,19 @@ See `docs/DATA_QUALITY_AUDIT.md` and `reports/data-quality-audit.md`.
 
 See `docs/TRANSLATION_WORKFLOW.md`.
 
-## Rights and provenance completion
-
-- [ ] Archive the original source URLs, access dates and exact open-distribution statements for bundled PDFs.
-- [ ] Review the retained PDF set against `docs/CONTENT_AND_RIGHTS.md` and `THIRD_PARTY_NOTICES.md`.
-
 ## Product completion after D1
 
 - [ ] Restore or redesign the administration workflow outside the public runtime.
-- [ ] Add broader automated browser smoke checks for asset paths, startup and critical flows beyond the current screenshot scenarios.
-- [ ] Review accessibility: keyboard navigation, contrast, focus, screen readers and text selection.
+- [ ] Add broader automated browser smoke checks.
+- [ ] Review accessibility.
 - [ ] Review privacy, local history and sharing behavior.
 - [x] Confirm automatic uDream publications and operational recovery in `@uNewsLog`.
 
 ## Later options
 
-- [ ] Consider moving inline CSS from `index.html` only if it materially improves maintainability.
-- [ ] Consider a build pipeline only when static-file maintenance becomes a real blocker.
-- [ ] Consider additional languages only after the source and translation workflow is documented.
+- [ ] Consider moving inline CSS only if maintainability requires it.
+- [ ] Consider a build pipeline only when static-file maintenance becomes a blocker.
+- [ ] Consider additional languages only after source and translation workflow is documented.
 - [ ] Consider TypeScript only when JSDoc and `@ts-check` no longer provide sufficient protection.
 
-The roadmap is intentionally conservative: preserve the working static site, current data and restoration points while advancing one clearly bounded series at a time.
+The roadmap remains conservative: preserve the working static site, current data and restoration points while advancing one bounded series at a time.

VERSION.md

@@ -8,61 +8,43 @@
 - Published: 2026-07-20
 - Purpose: preserve the verified PWA update, cache migration and installation flow release
 - Release: https://github.com/sunpole/udream/releases/tag/v23.8.0
-- Verification: the immutable tag resolves to the exact functional merge commit; the release workflow re-runs tests, project validation, syntax checks and version consistency before publication
 - Runtime version sources: `package.json`, `src/version.js` and `version.json` all contain `23.8.0`
-- Database: the active 4,086-record JSON database is unchanged
+- Active database: 4,086 records, unchanged
 - Previous checkpoint: `v23.7.0`
 
-## Documentation, provenance, registry and audit baseline
-
-`v23.8.10`
-
-- Status: D1.3 non-destructive data-quality audit milestone; it does not create a new application build or move the `v23.8.0` tag
-- Purpose: define deterministic audit rules, inspect all registered source/current records and preserve the results without modifying data
-- Audit specification: `docs/DATA_QUALITY_AUDIT.md`
-- Audit script: `scripts/audit-data-quality.mjs`
-- Machine report: `reports/data-quality-audit.json`
-- Human report: `reports/data-quality-audit.md`
-- CI command: `node scripts/audit-data-quality.mjs --check`
-- Logical datasets audited: `source-divinity-code-en`, `ru-current-v1`
-- Records audited: 4,086 in each logical dataset
-- ID result: unique, ordered and aligned from 1 through 4,086
-- Preserved-field differences: 0 for `id`, `symbol`, `description`, `source` and `date_added`
-- Expected changed-field differences: aliases 4,083; notes 4,086; tags 4,086
-- Structural gate: pass
-- Structural error instances: 0
-- Warning instances: 0
-- Human-review instances: 5,022 in five aggregated finding groups
-- Important limitation: review counts are heuristic candidates, not 5,022 proven content errors; one record may appear in multiple groups
-- Data mutation policy: read-only; no finding is corrected by the audit
-- Registry baseline: `v23.8.9`, D1.2 completed by PR #27
-- Source logical dataset ID: `source-divinity-code-en`
-- Current localized dataset ID: `ru-current-v1`
-- Canonical physical serialization: `source-divinity-code-en-bd2` at `data/bd2.json`
-- Retained compatibility serialization: `source-divinity-code-en-db` at `data/db.json`
-- Active runtime physical file: `ru-current-v1-runtime` at `data/divinity_code_ru.json`
-- Migration status: `planned-not-executed`; no file removal, rename or runtime switch is approved
-- Active dataset SHA-256: `1def80216e238b0c2a8640aaf1b4e95dd0669d5944a67f4e7c4421fad55a6e64`
-- Shared canonical English JSON SHA-256: `5ebe0d973f9cfd1c9db65a9d5abebe0ca16788261219299a710ed9fe78bb25d1`
+## Documentation, data and architecture baseline
+
+`v23.8.11`
+
+- Status: D1.4 two-book architecture milestone; no new application build and no movement of the `v23.8.0` tag
+- Purpose: define safe multi-book identity, provenance, modes, routing, validation, caching, fallback and rollback before implementation
+- Human architecture: `docs/TWO_BOOK_ARCHITECTURE.md`
+- Machine-readable architecture: `docs/two-book-architecture.json`
+- Validator: `scripts/validate-two-book-architecture.mjs`
+- CI command: `node scripts/validate-two-book-architecture.mjs`
+- Current default dataset: `ru-current-v1`
+- Current source dataset: `source-divinity-code-en`
+- Second source work status: retained PDF evidence only; no registered second-book dataset exists
+- Global record identity: `(dataset_id, record_id)`
+- Recommended order: separate mode, then federated combined search, then reviewed side-by-side comparison
+- Combined search rule: independent indexes and provenance-preserving result federation; no destructive JSON merge
+- Comparison rule: explicit reviewed relation map; no numeric-ID equivalence across books
+- Stable fallback: `ru-current-v1`
+- Runtime/data policy: existing data files, runtime, PWA, package metadata, saved versions and archive remain unchanged
+- Previous baselines: D1.1 `23.8.8`, D1.2 `23.8.9`, D1.3 `23.8.10`
 - Live handoff source: `WORK_STATUS.md`
-- Next approved task after D1.3 merge: D1.4 two-book product architecture
 - Repository download: https://github.com/sunpole/udream/archive/refs/tags/v23.8.0.zip
 
 ## Current development line
 
 No functional application version is assigned after `v23.8.0`.
 
-Documentation, provenance and automation updates `23.8.1` through `23.8.10` finalize the immutable release record, product/data baseline, translation safety, cross-device workflow, screenshot provenance, verified data provenance, dataset-registry governance and deterministic data-quality reporting. They do not change application runtime, PWA behavior or existing database records.
+Documentation, provenance, registry, audit and architecture updates `23.8.1` through `23.8.11` do not change application runtime, PWA behavior or existing database records.
+
+The next implementation phase must be approved separately. A second-book dataset must first be extracted, validated, registered and reviewed before selector, combined search or comparison UI work begins.
 
 ## Unified version line
 
 Version `v23.7.0` began the unified maintained-product line; `v23.8.0` is its next feature release.
 
-The repository previously used two separate counters:
-
-- Git releases: `v1.0.0`, `v2.0.0`, `v3.0.0`, `v3.5.0`, `v3.6.0`;
-- legacy UI iterations: numbered folders through `019` and the visible interface label `v19`.
-
-The product-version decision advanced the legacy visible line from `19` to `20` and combined it with the `3.7.0` development line, producing the unified version `23.7.0`. New maintained releases continue from that unified semantic version.
-
-Historical tags, releases, archived folders and runnable snapshots keep their original numbers.
+Historical tags, releases, archived folders and runnable snapshots keep their original numbers. Existing release tags are immutable.