test(docx-core): wire a libreoffice accept/reject oracle voter into the differential harness#345
Conversation
…he differential harness The Lean↔TS helper differential proved the model and the engine AGREE, but had no independent ground truth. This adds LibreOffice — the native engine for the .uno:Accept/RejectAllTrackedChanges dispatches — as a third voter, so the paragraph-collapse claims (G3/G4/G5 + the mark-based drop rule) are oracle-backed. New committed helper packages/docx-core/src/integration/libreoffice-oracle.ts: resolveSoffice, packMinimalDocx/extractDocumentXml (reusing primitives/zip), runLibreOfficeOracle (drives LibreOffice headless via an injected Basic macro — pyuno is blocked on macOS — batching all jobs in one launch), and paragraphShape. Gated voter [LEAN-HELP-09..11] in lean-differential-helpers.test.ts asserts LibreOffice agrees with the TS engine on paragraph STRUCTURE (count + which paragraphs collapsed to empty), not the full token projection: - [09] kept-not-dropped on G3/G4/G5; the contrived nested ins[del] G3 content divergence (LibreOffice keeps the inserted-then-deleted text on accept, where Word/Lean/TS collapse to empty — a likely LibreOffice nested-redline limitation) is pinned, not hidden. - [10] full structural agreement on the clean single-level fixtures (G4/G5). - [11] a PPR-INS-reject / PPR-DEL-accept drop control (the other direction). Local-only and best-effort: resolveSoffice only checks a binary EXISTS, not that it can LAUNCH. CI installs no LibreOffice (skips), and LibreOffice aborts (SIGABRT) under a sandboxed shell — so beforeAll catches a launch failure, logs why, and the assertions no-op rather than fail. A real terminal with a working LibreOffice runs it fully. The macro driver waits out the single-instance lock + retries once; SAFE_DOCX_ORACLE_DEBUG=1 keeps the temp profile for debugging. No production-engine change. Full docx-core suite: 1350 passed / 3 skipped.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
LLM-Based Quality GateOverall: ✅ PASS (3 pass · 0 warn · 11 skipped · 14 total)
Full checklist questions
Estimated cost (this run): $0.0084 — 26,942 input + 176 output tokens (≈4 chars/token) on |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…m coverage The oracle driver (src/integration/libreoffice-oracle.ts) drives headless LibreOffice via an injected macro; its core cannot run in CI, which installs no LibreOffice, so its lines are uncovered there and sink package coverage below the ratchet (workspace-test failed: 'Coverage ratchet failed'). Exclude it from the v8 coverage 'include', matching how the tool/environment-dependent src/benchmark/** is handled. The gated voter still exercises it locally with a real LibreOffice.
✅ Post-merge smoke passedMerged: Steps
This is a test/tooling change (no production-engine code), so no real-DOCX smoke was required. The oracle voter running end-to-end against a live LibreOffice on merged main is the relevant validation. Cleanup
Log: |
1 participant
Summary
Adds LibreOffice as an independent third voter to the Lean↔TS differential harness (PR-B of the G-case workstream). The harness proved the Lean model and TS engine agree, but had no external ground truth — both could be wrong the same way. LibreOffice is the native engine for the
.uno:AcceptAllTrackedChanges/.uno:RejectAllTrackedChangesdispatches, so wiring it in makes the paragraph-collapse claims (G3/G4/G5 + the mark-based drop rule) oracle-backed, and turns the throwaway.tmporacle script into a committed, reproducible check.What's here
packages/docx-core/src/integration/libreoffice-oracle.ts—resolveSoffice,packMinimalDocx/extractDocumentXml(reusingprimitives/zip),runLibreOfficeOracle(drives LibreOffice headless via an injected Basic macro — pyuno is blocked on macOS by Launch Constraints — batching all jobs in one launch), andparagraphShape(structural projection).[LEAN-HELP-09..11]inlean-differential-helpers.test.ts, asserting LibreOffice agrees with the TS engine on paragraph structure (count + which paragraphs collapsed to empty), not the full token projection:[09]kept-not-dropped on G3/G4/G5. The contrived nestedins[del]G3 content divergence — LibreOffice keeps the inserted-then-deleted text on accept, where Word/Lean/TS collapse to empty (a likely LibreOffice nested-redline limitation) — is pinned, not hidden.[10]full structural agreement on the clean single-level fixtures (G4 reject, G5 accept).[11]aPPR-INS-reject /PPR-DEL-accept drop control (the other direction of the mark-based rule).add-libreoffice-accept-reject-oracle([LEAN-HELP-09..11]);verification/ROADMAP.mdrecords the voter landed.Why the comparison is structural (not token-level)
LibreOffice rewrites styles/run-props (a token projection would throw on
w:pStyleetc.), and it interprets the nestedins[del]G3 case differently from Lean/TS. The paragraph count (kept-not-dropped) is what the oracle is authoritative for; the nested-revision content difference is recorded as a characterized divergence so a change in LibreOffice's behavior is noticed.Local-only and best-effort (no CI impact)
resolveSoffice()only checks a binary exists, not that it can launch. Two realities:describe.skips (like odf-core's LibreOffice test).beforeAllcatches the launch failure, logs why, and the assertions no-op rather than fail.So sandboxed/restricted environments stay green; a real terminal with a working LibreOffice runs the oracle fully. The driver waits out LibreOffice's single-instance lock + retries once;
SAFE_DOCX_ORACLE_DEBUG=1keeps the temp profile.Verification
@usejunior/docx-coresuite: 1350 passed / 3 skipped (+3 oracle tests).tsc --noEmitclean;openspec validate --strict✅; spec-coverage + allure-labels + allure-quality 0 errors.No production-engine change. This closes the deferred PR-B follow-up; the G-case workstream (G1–G5 + oracle voter) is now complete.