feat(validate): W-INVISIBLE-CHAR author lint for invisible characters (Closes #75)

[astrapi69]

Closes #75.

Why

Text pasted from a PDF or a web page carries characters that render as nothing: zero-width spaces, byte-order marks, directional marks, soft hyphens. They are legal JSON string content, so every structural check passes them, and nobody spots them by reading the file. The reference app's book-text wizard asks the author to paste a textbook section, so this is a live content path.

The rule that prompted it was our own. The blog style guide audited invisible characters with:

grep -oP '[^\x00-\x7F]' <file> | sort -u   # review every non-ASCII char

That lists every umlaut and accent and asks a human to notice an invisible character among them. An invisible character is invisible in that output too. Naming the codepoint is strictly better, and lesson content is where the paste actually lands.

What it does

W-INVISIBLE-CHAR, warning tier, never blocks. Example output:

[W-INVISIBLE-CHAR]  lesson text contains invisible characters (U+200B ZERO WIDTH SPACE):
1 occurrence at /title. These usually arrive by pasting from a PDF or a web page;
they are invisible when reading the file.

Detection sits in its own module and walks every string rather than a field list. That is what covers ext_payload without the engine knowing its shape, and covers future schema fields for free. Aggregated once per lesson, following the W-CARD-UNUSED precedent from #49: pasted prose can carry dozens, and per-occurrence emission is exactly the alert fatigue that aggregation solved.

Deliberately not flagged: U+00A0 NO-BREAK SPACE and U+202F NARROW NO-BREAK SPACE. Both render as whitespace and are legitimate typography, notably in the French content this ecosystem carries ("Comment ca va ?" sets one before the question mark). Flagging them would make the lint usually-wrong, which is the over-sensitive third-party behaviour it exists to avoid. Two boundary tests pin the exclusion.

Verification

TDD RED-first, 15 tests (10 red before the implementation): one per codepoint class, ext_payload reach, per-lesson aggregation, and the silence cases (clean content, umlauts and accents, both no-break spaces, lesson stays valid).

Measured against real content, not assumed. 528 lessons across eight content repositories:

Lessons scanned	528
Findings	4
False positives	0

All four are genuine soft hyphens sitting mid-word in pasted prose:

"...er drückt die Irrealit<U+00AD>ät aus."
"...kultur<U+00AD>spezi<U+00AD>fische Befunde..."

Typesetting hyphenation points that survived a copy, invisible to a reader, sitting in the official content repos until now. Those are content fixes in other repos, so they are reported here rather than silently changed.

Reaches content repos without their cooperation. With this build overlaid as the engine dependency, the unmodified adaptive-learner-content-test gate emitted the warning through its existing make lint-warnings, and make lint still exited 0. No content repo has to change anything.

Release

0.13.2, patch. Additive: warning tier only, no schema change, no schema_version bump. Version verified free on npm before bumping.

make release-check green, 494 tests across 29 files.

🤖 Generated with Claude Code

[astrapi69]

Content findings filed in the affected repositories, as agreed:

• adaptive-learner-content#155 (2 lessons in en-b1: Irrealität, Festgehalt)
• alc-psychology#3 (2 lessons in psych-intro: kulturspezifische carrying two, Ertrinkungsunfällen)

Five U+00AD characters across four German words in total. Each issue names the lesson, the JSON path, the affected word and its cleaned form, so the fix is mechanical.

Nothing was changed in those repositories from here: the lint is the engine's job, the content is theirs.