Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .claude/board/EPIPHANIES.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,14 @@
## 2026-07-08 — E-OCR-H2H-1 — the defensible head-to-head: recognition byte-parity confirmed on real photographed lines, the quality gap is INPUT CONDITIONING not the core, the matched speed gap is 7.3× (single-thread forward) with 4.6× leaner RAM
**Status:** FINDING (measured over 2286 HierText line crops + 300-crop matched speed; harnesses committed tesseract-rs `a02cb96`, report `.claude/harvest/h2h-report.md`)

**Why it's defensible where P6/perf weren't:** matched inputs (both engines fed the IDENTICAL GT-boxed crop → recognizer ISOLATED from the line finder), matched threading (single-thread both), matched load amortization (our in-process vs the CLI's image-list BATCH mode — the P6 per-page-subprocess CLI number was ~92% process-reload and is NOT a valid ratio). Corpus: HierText legible-horizontal-Latin subset (2286 lines / 60 images; the generator prints drop tallies — rotated/illegible dropped because they measure the finder, not recognition, and defeat both engines). Crops gitignored (reported comparison, not a hermetic gate); generator + `h2h_compare` + `h2h_speed` + `run_cli_speed.py` committed.

**Quality (recognition-only, 2286 lines):** mean CER ours 0.461 vs CLI 0.366 — both POOR because this is SCENE text (HierText exists to motivate detection pipelines; classic OCR fails for both). The **attribution** is the real finding and it reconciles with the byte-parity claim: both-correct 30.4%, both-wrong 56.8% (of which 25.4% IDENTICALLY wrong = the eng.lstm model's OWN ceiling, same model both sides, not a transcode gap), ours-only-wrong **10.1%**, cli-only-wrong 2.7% (we beat it). Where the CLI succeeds we match it on **75%** of lines. **The 10.1% ours-only gap is largely INPUT CONDITIONING, not the recognizer core** — the CLI's `RecognizeLine` auto-invert retry (`lstmrecognizer.cpp:349-380`: light-on-dark lines re-run inverted, keep the better) + DPI/contrast prep, which we deliberately never transcoded. On scene signage (frequent light-on-dark) that is exactly where the CLI pulls ahead. **Auto-invert is a bounded, byte-parity-able next leaf** that would close most of the 10.1%; the recognizer math stays byte-identical (P6 8/8 model-height lines through the same proven pixScale→FromPix→forward→beam chain).

**Speed (matched, 300 crops, single-thread both, load amortized once):** ours **12.88** lines/s vs CLI **93.97** (CLI **7.3× faster**) — BUT peak RSS **8.4 MB vs 38.3 MB** (ours **4.6× leaner**) and cold load **0.5 ms vs 98 ms** (ours **196× faster to load**). Per-stage (our side): `network.forward` is **99.4%** of the time — the ENTIRE gap is the single-threaded int8 LSTM GEMM, which ALREADY runs ndarray's AVX-512 VNNI path (byte-parity proven, E-OCR-MATDOTVEC-1). So it is an ENGINEERING gap, not a compute-primitive gap: single-thread + per-line under-filled VNNI tiles (variable-width lines) + per-line allocs. Closable by (1) Rayon-over-lines (lines independent, `&self` compute, per-line seeded RNG → near-linear with cores, exactly how the CLI OpenMP-threads), (2) width-bucketed batch-forward to fill the tiles, (3) stage pipelining — NONE changes a single output byte (separate mul+add, no FMA).

**The one-line verdict:** as an OCR *engine* (line → text) it IS the original (byte-parity where transcoded); the quality gap on hard scene text is a missing preprocessing branch + the model's own ceiling, not the recognizer; the matched speed gap is 7.3× (unoptimized single-thread forward) traded for 4.6× leaner RAM + 196× faster load — the edge/SoC-favorable side of the tradeoff. Two-tier reading: lean recognizer+ndarray at the edge (leaner+instant-load wins), the throughput gap is where a parallelization wave or the server tier earns its keep. Branch `claude/happy-hamilton-0azlw4`.

## 2026-07-08 — E-OCR-MINSIZE-GUARD-1 — HierText first-contact: the untranscoded PrepareLSTMInputs min-size guard was a live crash — fixed, and the noise-fixture blind-spot class claims its third scalp
**Status:** FINDING (crash root-caused + fixed faithfully; verhaltensneutral on the golden path; tesseract-rs PR #21)

Expand Down