feat(YAML) + fix(filter): vendor real YAML.pm 1.31 and scope source filters per compilation unit

[fglock]

Summary

A single PR with two related fixes that together unblock the bulk of the upstream YAML-1.31 test suite (and, more broadly, every CPAN module that uses Spiffy / Test::Base / Filter::Simple / any Filter::Util::Call-based source filter).

Unified with #604 — adopts that PR's cleaner design (wire-up at ModuleOperators.do_file rather than PerlLanguageProvider.executePerlCode, plus the FilterStateSnapshot / saveAndResetFilterState / restoreFilterState naming). See the discussion at the bottom of this PR.

Fix 1 — vendor real YAML.pm 1.31 instead of the YAML::PP shim

src/main/perl/lib/YAML.pm was a 12-line wrapper around YAML::PP that pinned $YAML::VERSION = '1.31' but produced YAML 1.2 output (null instead of ~, JSON-style booleans, snakeyaml error messages). This caused 5/12 sub-test failures in the YAML-1.31 distribution's own t/2-scalars.t and made YAML.pm-1.x-specific assertions impossible to satisfy.

Vendor the actual pure-Perl YAML.pm 1.31 into src/main/perl/lib/:

• YAML.pm (replaced) + YAML/{Any,Dumper,Error,Loader,Marshall,Mo,Node,Tag,Types}.pm + YAML/{Loader,Dumper}/Base.pm.

YAML::PP (Java-backed via YAMLPP.java + snakeyaml-engine) is kept unchanged — use YAML::PP; is its own bundled module with its own unit test (yaml_pp.t), docs, and CPAN consumers. Storable.java already uses snakeyaml-engine directly and is unaffected.

Fix 2 — scope Filter::Util::Call state per compilation unit

Real Perl scopes source filters per compilation unit (PL_compiling / PL_rsfp_filters): each require / do FILE starts with its own initially-empty filter chain, and the outer chain is restored when the nested compilation finishes.

PerlOnJava kept the active filter stack and the one-shot filterInstalledDuringUse flag in plain ThreadLocals — global per thread. A filter installed by an outer use Foo (whose import() runs while the parent file is still being parsed) leaked into whatever module Foo::import() happened to require next.

Most visible victim — Spiffy:

use Spiffy -Base;
field _filters => [qw(norm trim)];

Before the fix, Spiffy::import installed its filter, then called Exporter::export which required Exporter::Heavy. The filter leaked into Exporter::Heavy.pm (visible as "my" variable $self masks earlier declaration … at Exporter/Heavy.pm line 237), and the parent's filterInstalledDuringUse flag was consumed by the nested file's own use statements. The parent was then parsed unfiltered, and field FOO => [qw(...)] died with syntax error … near "=> [qw". This bricked Test::Base.pm and ~35 of the 54 YAML-1.31 test files at compile time.

Fix: introduce FilterUtilCall.saveAndResetFilterState() / restoreFilterState(FilterStateSnapshot) and wrap the require/do compile in ModuleOperators.do_file with try { … } finally { restoreFilterState(snapshot); }. Mirrors real Perl's per-compilation-unit scoping for the require/do path, and naturally leaves preprocessWithBeginFilters' recursive executePerlCode call alone (so op/incfilter.t does not regress).

Fix 3 — do CODEREF $_ localization + __FILE__ for filehandle/coderef do

Two related issues uncovered while bringing perl5_t/t/op/incfilter.t past the Spiffy regression (it gained +5 tests as a side effect):

• do CODEREF and tied $_: the generator loop did getGlobalVariable("main::_").set("") on each iteration to clear $_. When a previous iteration's coderef tied $_ to a class with only TIESCALAR/FETCH (the pattern in incfilter.t lines 261-268), the next iteration's set("") invoked the missing STORE and died. Fixed by replacing the $_ slot with a fresh untied scalar via aliasGlobalVariable (matches Perl's local $_ semantics in pp_require).
• __FILE__ NPE inside do FILEHANDLE / do CODEREF: actualFileName was only set in the scalar-ref branch. __FILE__ produced a StringNode with null value and crashed downstream. Fixed by setting actualFileName = fileName (the stringified GLOB(0x…) / CODE(0x…)). Matches incfilter.t's assertion qr/(?:GLOB|CODE)\(0x[0-9a-f]+\)/.

What changed

Java

• runtime/perlmodule/FilterUtilCall.java — new FilterStateSnapshot, saveAndResetFilterState(), restoreFilterState().
• runtime/operators/ModuleOperators.java — wrap require/do compile in try/finally; localize $_ in do CODEREF generator loop; set actualFileName for filehandle / coderef do.

Perl (vendored)

• src/main/perl/lib/YAML.pm (replaced) + YAML/{Any,Dumper,Error,Loader,Marshall,Mo,Node,Tag,Types}.pm + YAML/{Loader,Dumper}/Base.pm.

Tests

• src/test/resources/unit/source_filter_scope.t — Spiffy-style minimal repro (without depending on Spiffy itself: an inline InlineFilter package whose import() does filter_add then requires Cwd). Confirmed to catch the bug — fails 1/4 with the fix neutralised, passes 4/4 with the fix.
• src/test/resources/module/YAML/t/ — 34 upstream YAML-1.31 tests added to the bundled-module suite.

Docs

• dev/design/source_filters.md — full investigation, design rationale, results table, and explanation of why do_file is the right wire-up location.

What did not change

• src/main/java/.../YAMLPP.java — kept (separate bundled module).
• src/main/perl/lib/YAML/PP.pm — kept.
• src/main/perl/lib/Storable.pm — unaffected.
• src/main/java/.../PerlLanguageProvider.java — no longer touched. Earlier revisions of this PR wrapped executePerlCode and needed a skipSaveRestore carve-out; the unified design does neither.

Test plan

• make — green.
• ./gradlew test — green (the same task CI runs via make ci).
• make test-bundled-modules — 317 tests, only the 2 pre-existing unrelated failures (Net-SSLeay/t/local/33_x509_create_cert.t, Text-CSV/t/55_combi.t — present on master, no diff in those module dirs). All 34 YAML tests green.
• ./jperl -e 'use Test::Base; print "ok\n"' works (was: syntax error at .../Test/Base.pm line 53).
• t/2-scalars.t against the upstream YAML-1.31 distribution: 12/12 (was 7/12).
• perl5_t/t/op/incfilter.t: 148/153 (was 143/153 on master, +5 from the do CODEREF/__FILE__ fixes).
• Regression test confirmed to actually catch the Spiffy bug pattern: with saveAndResetFilterState/restoreFilterState neutralised to no-ops, test 2 fails with got: 'REPLACEME', expected: 'ok_marker'.

Commits

docs(source-filters): document Phase 6 + 7 fixes and incfilter.t residue
fix(do): localize $_ as fresh slot + set __FILE__ for filehandle/coderef do
fix(filter): scope Filter::Util::Call state per compilation unit
test(YAML): add YAML-1.31 distribution tests to bundled module suite
feat(YAML): vendor real YAML.pm 1.31 instead of YAML::PP shim

Generated with Devin