Export output shape: sections.tsv/pages.tsv split + human-readable keys (#106)

[mobileskyfi]

Post-#103 per-TSV review of rosetta export. Pure output shape — no schema change, no re-extraction, no MCP/TUI change. The export still reads the DB alone and rebuilds byte-identically. Closes #106; part of #95 / B-0022.

What changed

• Filenames now match their source table (like every other dataset): the per-section file pages.tsv → sections.tsv (source sections, includes the B-0023 _lead fragment), and the per-page rollup pages_summary.tsv → pages.tsv (source pages). This is the "two views of page content" split — pages.tsv is one whole-page row, sections.tsv is the pivot-able per-section detail. Grouping sections.tsv by page_id rolls up to (near-)the pages.tsv word count, so no section_word_count rollup column is needed — the pivot is the rollup. The residual (~2%, heading-text lines in no fragment) is disclosed in the manifest.
• callouts.tsv gains rosetta_id + resolved section_anchor so a reader identifies the owning page/section by name without joining back to the DB — the same pairing properties.tsv already emits. Long content field moved last.
• changelog.tsv moves sort_order before description (long free-text column last).
• Re-pointed the stale prerelease-ordering disclosure from Export dataset chunks: E4 per-fragment doc tables (blocked on #108), E3 products #104 → Shared version comparator ignores prerelease number (7.24beta1 == beta2) #107 (where that finding now lives).

Verification

bun test full suite 877 pass / 0 fail; typecheck, biome, markdownlint, cspell all clean. Live export against the local schema-v10 corpus (365 pages) confirms:

• headers correct on all four touched files;
• callout page/section names resolve (e.g. page …/certificates + section certificate-properties);
• the sections.tsv → pages.tsv pivot reconciles at 98.2% of page words, 365 _lead fragments (one per page), worst per-page residual 251 words — exactly the disclosed heading-line gap.

Notes

Export is [Unreleased] (only in rc builds), so the renames carry no back-compat cost. MANUAL.md and CHANGELOG.md updated. The deferred #104 chunks (E3/E4) and the #108 safe-replacement scheme are untouched — this is only the output-shape cleanup.

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Added sections.tsv to exports, providing one row per page section for easier grouping and analysis.
  • Updated pages.tsv to provide authoritative whole-page records, including rollup counts.
  • Enhanced callouts.tsv with page identifiers and resolved section context.

• Updates

  • Removed pages_summary.tsv from the exported dataset.
  • Refined changelog column ordering and export documentation.
  • Improved section coverage for content appearing before the first heading.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: eeb0867b-dd5d-45c3-b76a-e4a05d6b62dd

📥 Commits

Reviewing files that changed from the base of the PR and between 047c9ce and 15f15c4.

📒 Files selected for processing (4)

• CHANGELOG.md
• MANUAL.md
• src/export.test.ts
• src/export.ts

📜 Recent review details

⏰ Context from checks skipped due to timeout. (2)

• GitHub Check: copilot-pull-request-reviewer
• GitHub Check: Analyze (javascript-typescript)

🧰 Additional context used

📓 Path-based instructions (6)

*

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Keep this Copilot instructions file short; put substantive rules in narrow instruction files under .github/instructions/*.instructions.md.

Files:

• CHANGELOG.md
• MANUAL.md

CHANGELOG.md

📄 CodeRabbit inference engine (CLAUDE.md)

Record user-visible shipped changes in CHANGELOG.md under [Unreleased] or release sections.

Files:

• CHANGELOG.md

**/*.md

📄 CodeRabbit inference engine (CLAUDE.md)

Prefer each document's canonical home and do not duplicate operational detail, schema blocks, or long rule lists in the routing index.

Files:

• CHANGELOG.md
• MANUAL.md

{README.md,src/setup.ts,MANUAL.md}

📄 CodeRabbit inference engine (AGENTS.md)

Keep public client setup snippets aligned across README.md, src/setup.ts, and MANUAL.md, including the Codex command codex mcp add rosetta -- bunx @tikoci/rosetta``.

Files:

• MANUAL.md

MANUAL.md

📄 CodeRabbit inference engine (CLAUDE.md)

Put installation, operations, release/re-extraction procedures, and schema reference in MANUAL.md.

Files:

• MANUAL.md

**/*.{ts,tsx}

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Use Bun and TypeScript, including Bun-native runtime APIs such as bun:sqlite, Bun.serve, and bunx.

Rosetta uses Bun and TypeScript; prefer bun and bun test rather than Node/npm-oriented substitutes.

Files:

• src/export.ts
• src/export.test.ts

🔇 Additional comments (4)

src/export.ts (1)

197-198: LGTM!

Also applies to: 210-226, 320-328, 350-360, 386-386, 414-415, 435-439

src/export.test.ts (1)

67-68: LGTM!

Also applies to: 123-123, 203-208, 210-220, 273-284, 286-291

CHANGELOG.md (1)

27-27: LGTM!

Also applies to: 48-48, 49-49

MANUAL.md (1)

298-305: LGTM!

Also applies to: 306-307

---

📝 Walkthrough

Walkthrough

The export contract now emits revised changelog and callout columns, adds sections.tsv, redefines pages.tsv as the whole-page dataset, removes pages_summary.tsv, and updates tests and documentation for the new rollup semantics.

Changes

Export dataset contract

Layer / File(s)	Summary
Tabular export contracts src/export.ts, src/export.test.ts, CHANGELOG.md	changelog.tsv reorders sort_order, while callouts.tsv adds rosetta_id and section_anchor through page and section joins; tests validate the revised schemas and values.
Page and section dataset views src/export.ts, src/export.test.ts	sections.tsv becomes the per-section dataset, pages.tsv becomes the whole-page view, and pages_summary.tsv is removed from the dataset manifest and test allowlist.
Export documentation and disclosures CHANGELOG.md, MANUAL.md, src/export.ts	Documentation describes the revised files, rollup relationships, residual heading text, and synthetic H0 lead-fragment coverage.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related issues

• Umbrella: schema/ETL findings from the B-0022 DB-only export audit #95 — Covers the H0 section-coverage correction and related export rollup changes.

Possibly related PRs

• tikoci/rosetta#13 — Introduces the stable pages.rosetta_id data used by the revised callout export.
• tikoci/rosetta#103 — Provides the initial export implementation whose TSV contracts are revised here.
• tikoci/rosetta#105 — Introduces the synthetic H0 fragment reflected in the updated export coverage semantics.

Suggested labels: area:docusaurus

Suggested reviewers: copilot

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	src/export.ts and tests align on callouts and changelog, but #106 required a pages_summary.tsv reconciliation column that this PR removes.	Add the section_word_count reconciliation in pages_summary.tsv, or update the issue/acceptance to match the new sections.tsv to pages.tsv pivot design.
Docstring Coverage	⚠️ Warning	Docstring coverage is 71.43% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	Title matches the main export-shape change: splitting sections/pages and adding human-readable keys.
Out of Scope Changes check	✅ Passed	Changes stay within export output shape, docs, tests, and exporter code; no unrelated MCP/TUI or schema work is evident.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch export-output-shape

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Copilot]

Pull request overview

Refines the rosetta export TSV output shape (no schema/ETL changes) to make datasets more readable and table-aligned: renames the page/section exports so filenames match their source tables, reorders long-text columns to the end, and adds human-readable page/section keys to callouts.tsv. Updates the export contract tests and operator docs accordingly.

Changes:

• Renamed pages.tsv → sections.tsv (per-section rows) and pages_summary.tsv → pages.tsv (per-page rows).
• Updated callouts.tsv to include rosetta_id and resolved section_anchor, with content last; reordered changelog.tsv to put sort_order before description.
• Updated export tests and documentation to reflect the new filenames and column layouts.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
src/export.ts	Adjusts dataset readers: column order changes, callouts joins for human-readable keys, and renames pages/sections export outputs + new disclosure.
src/export.test.ts	Updates fixtures and assertions for new column orders and renamed TSV files.
MANUAL.md	Updates the export directory layout and describes the new pages.tsv vs sections.tsv split.
CHANGELOG.md	Adds an [Unreleased] bullet documenting the user-visible export output-shape changes and updates prior wording to match the new filenames.