Export tables.tsv: flat inventory of captured doc tables (#111)#112
Conversation
Add a top-level `tables.tsv` dataset to `rosetta export` — one row per `page_tables` record (855 on the rc.99 corpus) with page/section provenance, table shape (column/data-row counts, ragged flag), source size (`raw_bytes`, UTF-8 length of the stored Markdown), and a `table_url` deep-linking to the containing section (`url#anchor`; bare page URL when no section resolves, since Docusaurus has no per-table anchor; NULL when the page has no URL). This is the "what tables exist and where" list — it answers most table-audit questions that today only `table_count`/`table_row_count` on pages.tsv/sections.tsv hint at, without exporting the cell data itself (the per-fragment cell files remain E4, blocked on #108). Flat top-level dataset: no variable file set, no #108 dependency, reusing the settled TSV/manifest contract and manifest-owned prune. Adds a disclosure for table_url granularity + raw_bytes, and an anchor test asserting shape/size/deep-link for a seeded table. Closes #111. Intermediate step under #104 (E4/E3 unchanged). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthrough
ChangesTables export
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related issues
Possibly related PRs
Suggested labels: Suggested reviewers: Sequence Diagram(s)sequenceDiagram
participant runExport
participant readTables
participant SQLite
participant manifest
runExport->>readTables: request tables.tsv rows
readTables->>SQLite: query page_tables with pages and sections
SQLite-->>readTables: table metadata and provenance
readTables-->>runExport: tables.tsv dataset
runExport->>manifest: serialize dataset metadata
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
Adds a new top-level export dataset (tables.tsv) to rosetta export, providing a flat inventory of captured Markdown tables (page_tables) with page/section provenance, deep-link URL, shape metrics, and source-size (raw_bytes) for table-audit workflows.
Changes:
- Add
readTables()dataset reader to emittables.tsvand register it inDATASET_READERS, plus a new disclosure describingtable_url+raw_bytes. - Extend
src/export.test.tsto expecttables.tsvand assert deep-link/shape/size for a seeded table row. - Add a
[Unreleased]changelog entry describing the new dataset.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
src/export.ts |
Adds tables.tsv dataset reader, computes table_url and raw_bytes, registers dataset, and adds disclosure. |
src/export.test.ts |
Updates expected export file set and adds a focused tables.tsv test for link/shape/bytes. |
CHANGELOG.md |
Documents the new tables.tsv dataset under [Unreleased]. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/export.test.ts`:
- Line 312: Update the assertion in the relevant export test to use the exported
utf8Bytes helper from export.ts instead of calling Buffer.byteLength directly,
while preserving the existing input string and expected comparison.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: a794dcb5-d009-47ec-906d-64f162b2954a
📒 Files selected for processing (3)
CHANGELOG.mdsrc/export.test.tssrc/export.ts
📜 Review details
⏰ Context from checks skipped due to timeout. (1)
- GitHub Check: copilot-pull-request-reviewer
🧰 Additional context used
📓 Path-based instructions (4)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use Bun and TypeScript, including Bun-native runtime APIs such as
bun:sqlite,Bun.serve, andbunx.Rosetta uses Bun and TypeScript; prefer
bunandbun testrather than Node/npm-oriented substitutes.
Files:
src/export.tssrc/export.test.ts
*
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Keep this Copilot instructions file short; put substantive rules in narrow instruction files under
.github/instructions/*.instructions.md.
Files:
CHANGELOG.md
CHANGELOG.md
📄 CodeRabbit inference engine (CLAUDE.md)
Record user-visible shipped changes in
CHANGELOG.mdunder[Unreleased]or release sections.
Files:
CHANGELOG.md
**/*.md
📄 CodeRabbit inference engine (CLAUDE.md)
Prefer each document's canonical home and do not duplicate operational detail, schema blocks, or long rule lists in the routing index.
Files:
CHANGELOG.md
🔇 Additional comments (4)
src/export.ts (2)
353-394: LGTM!
532-532: LGTM!Also applies to: 558-561
src/export.test.ts (1)
123-123: LGTM!CHANGELOG.md (1)
29-30: LGTM!
- MANUAL.md: add tables.tsv to the export output tree + prose, noting cell data is not exported here (Copilot). - CHANGELOG.md: cite the tracking issue #111, not the #104 umbrella (Copilot). - export.test.ts: use the exported utf8Bytes helper instead of a raw Buffer.byteLength call (CodeRabbit). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a top-level
tables.tsvdataset torosetta export— the "what tables exist and where" list, an intermediate step under #104 taken before E4.What
One flat row per
page_tablesrecord (855 on the rc.99 corpus), ordered by theUNIQUE(page_id, sort_order)key:page_id, rosetta_id, slug, title, urlsection_id, section_anchor, source_headingsort_order, table_urlcolumn_count, data_row_count, is_ragged, raw_bytestable_urldeep-links to the section containing the table (url#anchor). Docusaurus has no per-table anchor, so a table with no resolvable section (16/855 on rc.99) links to the bare page URL; a page with no URL yields NULL.raw_bytesis the UTF-8 size of the storedraw_markdown(source, not rendered). The cell data itself is not exported here — that stays E4.Why
Today tables surface only as counts (
table_count/table_row_countonpages.tsv/sections.tsv). This lists the actual tables with a per-row manual.mikrotik.com deep link, covering most of the need to export table data without E4's variable per-fragment file set (blocked on #108).Contract fit
Flat top-level dataset: no variable file set, no #108 dependency. Reuses the settled TSV serialization +
manifest.tomlprovenance + manifest-owned prune/ownership. Adds one disclosure (table_url granularity + raw_bytes) and an anchor test asserting shape/size/deep-link for a seeded table.Verification
bun test src/export.test.ts— 29 pass (newtables.tsvtest + updatedALL_FILES; determinism/boundary tests unchanged).tables.tsv= 855 rows; spot-checked section-linked and NULL-section (\N) rows and the manifest[[files]]entry.biome checkclean on both changed source files.Scope
Does not close #104 — E4 (per-fragment cell files, blocked on #108) and E3 (products) remain. Closes #111.
Summary by CodeRabbit
tables.tsvexport containing an inventory of captured document tables.tables.tsvto the exported dataset set and documented its contents.