feat(ogar-from-docv1): W2 — the pixel retina (doc.v1 → ogar-doc-ir)#202
Conversation
The consumer-side transcode that closes the doc-IR convergence's second retina: parse tesseract-rs's doc.v1 recognition JSON into the source-agnostic ogar-doc-ir perceptual IR, so a scanned document and a web page (spider_doc_ir, W3) produce ONE shape. Honors the tesseract-rs operator boundary (its CLAUDE.md): tesseract-rs = faithful recognition -> rich doc.v1; store/graph/OGAR are NOT its concerns. So the doc.v1 -> ogar-doc-ir transcode lives HERE on the OGAR side, exactly as ogar-from-ruff lifts ruff's output. tesseract-rs is untouched; its structured.rs is the schema spec this crate targets. Mapping: region type text/table/figure/header/footer -> the closed ogar_doc_ir::RegionKind (already 1:1 — live evidence the IR is source- agnostic); pixel bboxes quantized to the u8:u8 unit-square rail (Rendered geometry, vs the DOM retina's DomOrder); a table's rows/cols/cells emitted FLAT on the region object (per emit_table_json) -> ogar TableCells; harvested fields -> document-level TypedFields (OCR conf on its 0..100 scale, kept distinct from DOM trust by the Provenance lane); content_sha256 caller- supplied (per-acquisition dedup, NOT the cross-retina key — #199). 4 tests: transcode incl. quantization + table cells + field harvest; the SAME ogar_doc_ir::from_json load gate a DOM producer passes (the convergence proof); wrong-schema refused; unknown region kind fails loud. clippy -D warnings + fmt clean. serde-only, canon-free (the 0x080A/0x080B mints stay in ogar-vocab/W4, gated on the doc-layer council). With W2 + W3 both live, P-XRETINA can run for real: the same invoice via scan and via HTML must converge on facts (converges_on_facts, on main via #199). Co-Authored-By: Claude <noreply@anthropic.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_c6c7e058-0143-4330-b6d9-a22447bf2979) |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7b4d9787de
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "figure" => RegionKind::Figure, | ||
| "header" => RegionKind::Header, | ||
| "footer" => RegionKind::Footer, | ||
| other => return Err(FromDocV1Error::UnknownRegionKind(other.to_string())), |
There was a problem hiding this comment.
When this is fed JSON from tesseract-rs structured::render_json (the default non-classified doc.v1 renderer), regions are emitted with type: "paragraph"; that valid output currently falls into this arm and from_doc_v1 returns UnknownRegionKind, so consumers that do not use render_json_with_regions cannot transcode tesseract's doc.v1 at all. Please map paragraph to RegionKind::Text (or otherwise handle it explicitly) before treating other values as producer drift.
Useful? React with 👍 / 👎.
| row: c.row.min(255) as u8, | ||
| col: c.col.min(255) as u8, |
There was a problem hiding this comment.
For OCR tables with more than 256 rows or columns, doc.v1 can carry row/col values >=256, but these casts collapse all of those cells onto index 255. That silently corrupts downstream table facts by making distinct cells appear to occupy the same coordinate; since the IR cannot represent those indexes as u8, the transcode should fail loud or preserve overflow explicitly instead of clamping.
Useful? React with 👍 / 👎.
2 participants
What
W2 of the doc-IR × spider convergence — the pixel retina. Parses tesseract-rs's
doc.v1recognition JSON into the source-agnosticogar-doc-irperceptual IR, so a scanned document and a web page (spider_doc_ir, W3) produce one shape.Honors the tesseract-rs boundary
Per the tesseract-rs operator doctrine (its
CLAUDE.md): tesseract-rs = faithful recognition → richdoc.v1; store/graph/OGAR are NOT its concerns. So thedoc.v1 → ogar-doc-irtranscode lives here on the OGAR side, exactly asogar-from-rufflifts ruff's output. tesseract-rs is untouched — itsstructured.rsis the schema spec this crate targets. (This corrected my own plan, which had said "tesseract emits ogar-doc-ir".)Mapping
typetext/table/figure/header/footer→ the closedogar_doc_ir::RegionKind— already 1:1, live evidence the IR is genuinely source-agnostic (the DOM retina'sheader/main/footer/nav/table/figureare the same vocabulary).u8:u8unit-square rail (Geometry::Rendered, vs the DOM retina'sDomOrder).rows/cols/cellsare emitted flat on the region object (peremit_table_json) →ogarTableCells. (This was the one bug — the initial struct expected them nested under a"table"key; caught by the test.)TypedFields (OCRconfon its 0..100 scale, kept distinct from DOM trust by theProvenancelane).content_sha256caller-supplied (per-acquisition dedup, not the cross-retina key — per feat(ogar-doc-ir): P-XRETINA convergence probe + the content_sha256 identity finding #199).Tests (4, all green)
Transcode incl. quantization + table cells + field harvest; the output passes the same
ogar_doc_ir::from_jsonload gate a DOM producer passes (the convergence proof); wrong-schema refused; unknown region kind fails loud.clippy -D warnings+fmtclean. serde-only, canon-free (the0x080A/0x080Bmints stay in ogar-vocab/W4, gated on the doc-layer council).Unblocks P-XRETINA
With W2 + W3 both live, the killer probe can run for real: the same invoice via scan and via HTML must converge on facts (
converges_on_facts, on main via #199).Generated with Claude Code
Generated by Claude Code