Skip to content

feat(moose): Phase 2 stubs — metaclass / Test::Moose / Moose::Util / skeletons#572

Open
fglock wants to merge 18 commits intomasterfrom
feature/moose-stubs-round2
Open

feat(moose): Phase 2 stubs — metaclass / Test::Moose / Moose::Util / skeletons#572
fglock wants to merge 18 commits intomasterfrom
feature/moose-stubs-round2

Conversation

@fglock
Copy link
Copy Markdown
Owner

@fglock fglock commented Apr 27, 2026

Summary

This stacks on top of #570. After that PR fixed the Class::MOP::class_of
hard-die at compile time, this PR fills in the next batch of compile-time
and runtime blockers and one TAP-bailout fix.

What's new

  • Moose.pm and Moose::Role.pm now use Class::MOP () at top so Moo's
    runtime calls into Class::MOP::class_of (made whenever
    $INC{"Moose.pm"} is set) are always defined. Was the cause of ~50+
    "Undefined subroutine &Class::MOP::class_of" runtime errors on the
    previous baseline.

  • metaclass.pm stub — installs a no-op meta method on the caller.
    ~34 upstream test files use use metaclass; directly.

  • Test::Moose.pm — covers meta_ok, does_ok, has_attribute_ok,
    with_immutable. has_attribute_ok falls back to
    $class->can($attr) when no real metaclass is available. ~30 test
    files use use Test::Moose.

  • Moose::Util.pm — covers find_meta, is_role, does_role,
    apply_all_roles, english_list, throw_exception, plus
    trait/metaclass alias passes-through. ~13 test files use it.

  • Skeleton stubs that let require X + X->new(...) succeed:

    • Class::MOP::Class
    • Class::MOP::Attribute
    • Moose::Meta::Class (inherits from Class::MOP::Class)
    • Moose::Meta::TypeConstraint::Parameterized
    • Moose::Meta::Role::Application::RoleSummation
    • Moose::Exporter (forwards setup_import_methods to
      Moose->import on the consumer)
  • Moose::Util::TypeConstraints now pre-populates standard-type stubs
    (Any, Item, Defined, Bool, Str, Num, Int, ArrayRef,
    HashRef, Object, ClassName, ...) as small blessed objects with
    .name, .has_parent, .check, .can_be_inlined, .is_a_type_of,
    etc. Required to prevent
    t/type_constraints/util_std_type_constraints.t from calling
    BAIL_OUT("No such type ...") — which would have killed prove and
    lost ~7 trailing test files / 50+ assertions.

  • dev/modules/moose_support.md updated: new column in the baseline
    table, Phase 2 stubs marked done in the progress tracker.

Effect on ./jcpan -t Moose (Moose 2.4000)

Metric Before (#570) After
Files executed 478 478
Assertions executed 667 1419
Fully passing files 36 56
Partially passing files 98 184
Compile/load fail (no tests) 344 238
Assertions ok 419 953
Assertions fail 248 466

Net: +752 assertions executed, +534 newly pass, +20 fully-green
files, -106 files compile that previously didn't.
The +218
newly failing assertions are mostly tests that hadn't reached their
assertion phase before (so "fail" is the honest answer); the rest
reflect real shortcomings of stubs that would only be fixed by Phase D
(real Class::MOP / Moose port).

Test plan

  • make — full unit suite passes (no regressions on either backend).
  • ./jperl -e 'use metaclass; print "ok"'ok.
  • ./jperl -e 'use Moose::Util qw(english_list); print english_list("a","b","c")'a, b, and c.
  • ./jperl -e 'use Test::Moose; print "ok"'ok.
  • ./jperl -e 'require Class::MOP::Class; my $m = Class::MOP::Class->initialize("Foo"); print $m->name'Foo.
  • ./jperl -e 'use Moose::Util::TypeConstraints; my $t = find_type_constraint("Int"); print $t->name, " ", $t->check(42)'Int 1.
  • ./jcpan -t Moose — numbers above; saved in /tmp/moose_test4.txt.
  • No BAIL_OUT events in the test run.

What's intentionally not in this PR

  • Phase B (strip XS in WriteMakefile). Not yet needed.
  • Phase C-full — real Class::MOP::Class instances backed by Java
    helpers. Would unlock the next big chunk of failing tests.
  • Phase D — bundle pure-Perl Class::MOP::* and Moose::*. Largest
    payoff but largest effort; tracked in the design doc.

Generated with Devin

@fglock fglock changed the base branch from feature/moose-phase-abc to master April 27, 2026 12:17
fglock added a commit that referenced this pull request Apr 27, 2026
After two iterative shim-widening PRs (#570, #572), the original
phase plan ("ship Quick path, then do A→B→D") needs revision. The
shim approach has paid out much faster than a full pure-Perl port
would have, so the doc now:

1. Records concrete lessons learned (compile-time stubs are
   high-leverage; pre-loading matters as much as having stubs;
   BAIL_OUT is a hidden multiplier; the gap is method surface, not
   metaclass semantics; stubs need correct @isa).

2. Replaces the stale "Decision needed" section with a concrete,
   data-driven Phase 3+ plan, sized against the actual remaining
   failure counts in the latest run:

   - Phase 3 (rich Moose::_FakeMeta + next batch of stubs +
     TypeConstraint isa fix) — ~1 day; expected payoff +15–25 green
     files / +200–500 newly passing assertions.
   - Phase 4 (hook into Moo's attribute store from FakeMeta) — ~2
     days; gives Test::Moose::has_attribute_ok real semantics.
   - Phase 5 (Moose::Util::MetaRole::apply_metaroles) — ~1 day.
   - Phase 6 (full Moose::Exporter sugar installation) — ~2–3 days.
   - Phase B / D moved to "deferred / last resort" status with
     explicit re-trigger conditions.

3. Refreshes the open work items list with phase-tagged TODOs.

No code changes, just doc.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
fglock and others added 6 commits April 27, 2026 16:07
- Add deterministic ExtUtils::HasCompiler stub
  (src/main/perl/lib/ExtUtils/HasCompiler.pm). Always answers "no" to
  can_compile_loadable_object / can_compile_static_library /
  can_compile_extension. Replaces reliance on $Config{usedl} happening
  to be empty.

- Add Class::MOP shim (src/main/perl/lib/Class/MOP.pm) providing
  class_of, get_metaclass_by_name, store_metaclass_by_name,
  remove_metaclass_by_name, does_metaclass_exist,
  get_all_metaclasses (and friends), get_code_info (via B),
  is_class_loaded, load_class, load_first_existing_class. Returns
  "no metaclass" everywhere — the correct answer under the
  Moose-as-Moo shim. Previously Moo's _Utils::_load_module would
  hard-die with "Undefined subroutine &Class::MOP::class_of" the
  moment $INC{"Moose.pm"} was set, which our shim does at startup.

- Update dev/modules/moose_support.md with the new baseline column
  and mark Phase A / Phase C-mini done in the progress tracker.

Effect on `./jcpan -t Moose` (Moose 2.4000 upstream test suite vs.
the shim):

| Metric                        | Before | After |
|-------------------------------|-------:|------:|
| Files executed                |    478 |   478 |
| Assertions executed           |    616 |   667 |
| Fully passing files           |     35 |    36 |
| Partially passing files       |     94 |    98 |
| Compile/load fail (no tests)  |    349 |   344 |
| Assertions ok                 |    372 |   419 |
| Assertions fail               |    244 |   248 |

Net: +51 assertions executed, +47 newly pass, +1 fully-green file,
no regressions in `make` (full unit test suite passes).

See dev/modules/moose_support.md for the broader phase plan.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…skeletons

Adds the next batch of compile-time and runtime stubs that the
Moose-as-Moo shim was missing. Together they unblock a large slice of
the upstream Moose 2.4000 test suite.

Changes:

- Moose.pm / Moose/Role.pm: add `use Class::MOP ()` at top so Moo's
  runtime calls into Class::MOP::class_of (made whenever
  $INC{"Moose.pm"} is set) are always defined. Was the cause of
  ~50+ "Undefined subroutine &Class::MOP::class_of" runtime errors.

- New: src/main/perl/lib/metaclass.pm — `metaclass` pragma stub.

- New: src/main/perl/lib/Test/Moose.pm — meta_ok / does_ok /
  has_attribute_ok / with_immutable. has_attribute_ok falls back to
  $class->can($attr) when no real metaclass is available.

- New: src/main/perl/lib/Moose/Util.pm — find_meta, is_role,
  does_role, search_class_by_role, ensure_all_roles, apply_all_roles,
  with_traits, get_all_attribute_values, get_all_init_args,
  resolve_metatrait_alias, resolve_metaclass_alias,
  add_method_modifier, english_list, throw_exception, plus
  Moose::Exception-style wrappers.

- New: skeleton stubs that let `require X` + `X->new(...)` succeed:
    src/main/perl/lib/Class/MOP/Class.pm
    src/main/perl/lib/Class/MOP/Attribute.pm
    src/main/perl/lib/Moose/Meta/Class.pm
    src/main/perl/lib/Moose/Meta/TypeConstraint/Parameterized.pm
    src/main/perl/lib/Moose/Meta/Role/Application/RoleSummation.pm
    src/main/perl/lib/Moose/Exporter.pm

- Moose/Util/TypeConstraints.pm: pre-populate standard-type stubs
  (Any, Item, Defined, Bool, Str, Num, Int, ArrayRef, HashRef,
  Object, ClassName, ...) as small blessed objects with .name /
  .has_parent / .check / .can_be_inlined / etc. Required to prevent
  Moose's t/type_constraints/util_std_type_constraints.t from calling
  BAIL_OUT("No such type ...") when find_type_constraint returned
  undef — which would have killed prove and lost ~7 trailing test
  files.

- dev/modules/moose_support.md: new column in the baseline table,
  Phase 2 stubs marked done in the progress tracker.

Effect on `./jcpan -t Moose` (Moose 2.4000 upstream test suite):

| Metric                        | Before | After |
|-------------------------------|-------:|------:|
| Files executed                |    478 |   478 |
| Assertions executed           |    667 |  1419 |
| Fully passing files           |     36 |    56 |
| Partially passing files       |     98 |   184 |
| Compile/load fail (no tests)  |    344 |   238 |
| Assertions ok                 |    419 |   953 |
| Assertions fail               |    248 |   466 |

Net: +752 assertions executed, +534 newly pass, +20 fully-green
files, -106 files that previously failed at compile time. The +218
new failing assertions are mostly tests that hadn't reached their
assertion phase before (so "fail" is the honest answer); these would
need real Class::MOP / Moose internals (Phase D) to pass.

`make` still clean on both backends.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
After two iterative shim-widening PRs (#570, #572), the original
phase plan ("ship Quick path, then do A→B→D") needs revision. The
shim approach has paid out much faster than a full pure-Perl port
would have, so the doc now:

1. Records concrete lessons learned (compile-time stubs are
   high-leverage; pre-loading matters as much as having stubs;
   BAIL_OUT is a hidden multiplier; the gap is method surface, not
   metaclass semantics; stubs need correct @isa).

2. Replaces the stale "Decision needed" section with a concrete,
   data-driven Phase 3+ plan, sized against the actual remaining
   failure counts in the latest run:

   - Phase 3 (rich Moose::_FakeMeta + next batch of stubs +
     TypeConstraint isa fix) — ~1 day; expected payoff +15–25 green
     files / +200–500 newly passing assertions.
   - Phase 4 (hook into Moo's attribute store from FakeMeta) — ~2
     days; gives Test::Moose::has_attribute_ok real semantics.
   - Phase 5 (Moose::Util::MetaRole::apply_metaroles) — ~1 day.
   - Phase 6 (full Moose::Exporter sugar installation) — ~2–3 days.
   - Phase B / D moved to "deferred / last resort" status with
     explicit re-trigger conditions.

3. Refreshes the open work items list with phase-tagged TODOs.

No code changes, just doc.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Be explicit that the shim-based Phases 3 → 6 will NOT pass all Moose
self-tests. Project the ceiling at ~150 / 478 fully-green files
(~30%), and call out the test areas that categorically cannot pass
without a real Class::MOP / Moose port:

- make_immutable inlining (t/immutable/)
- MOP introspection symmetry (t/cmop/method.t et al)
- Role composition conflict messages (t/roles/role_conflict_*)
- Native attribute traits (t/native_traits/)
- Type-constraint coercion graphs and _inline_check
- Class::MOP self-bootstrap

If "pass all Moose tests" is the hard goal, only Phase D (bundle
pure-Perl Moose) is credible. If "unblock ordinary Moose consumers"
is the goal, Phases 3-6 are the right move.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The previous revision of this document hedged on whether the shim
ceiling would top out around 30%. With weaken/DESTROY now landed in
core PerlOnJava and a closer look at Moose 2.4000's actual XS surface
(710 lines total, mostly generic hashref accessors), the picture
changes:

- Goal becomes: pass 477/478 Moose tests. The single excluded file
  is t/todo_tests/moose_and_threads.t — already TODO upstream and
  PerlOnJava doesn't implement threads. Zero Moose tests use fork.

- Strategy is two-stage:
  1. Phases 3 → 6 (incremental shim widening, ~1 week) take us from
     56 to ~110–130 fully-green files. Ships value to real-world
     Moose-using CPAN modules immediately.
  2. Phase D (bundle pure-Perl Moose + a single ~500-line
     Class::MOP::PurePerl, ~5 days) takes us to 477/478. The XS
     surface is small enough that this is now a tractable port,
     not the multi-week effort earlier revisions suggested.

- Phase D is broken down into D1-D6 with explicit per-step efforts
  and a per-.xs-file breakdown of what Class::MOP::PurePerl needs
  to provide. Reference: pre-XS Moose commit bf38c2e9 has the
  pure-Perl version that was replaced.

- "Realistic ceiling ~30%" framing removed — was based on assuming
  Phase D was prohibitively large, which it isn't.

- Out-of-scope section trimmed: weaken / DESTROY / B introspection
  are no longer blockers; only `threads` (1 file) and `fork` (0
  files) remain genuinely out of scope.

- Open work items list re-ordered to phase-tagged TODOs ending in
  Phase D6 (snapshot tests under module/Moose/) — the regression
  net that locks the win in via make test-bundled-modules.

No code changes.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Implements Phases 3a-3e from dev/modules/moose_support.md.

Highlights:

- Moose::_FakeMeta gets a real method surface and proper @isa:
  inherits from Moose::Meta::Class and Class::MOP::Class so
  isa_ok($meta, ...) checks pass. Implements add_attribute,
  get_attribute, find_attribute_by_name (walks @isa), has_attribute,
  remove_attribute, get_attribute_list, get_all_attributes,
  get_method (returns a Class::MOP::Method), has_method,
  get_method_list, new_object, superclasses, linearized_isa,
  is_immutable, is_mutable, roles, does_role.

- Per-class meta cache so $class->meta returns the same object
  each call (required for tests that compare metaclass identity).

- Moose.pm and Moose/Role.pm record each `has` declaration on the
  target's _FakeMeta, so $meta->get_attribute_list and
  find_attribute_by_name actually return useful data.

- New compile-time stubs (skeleton .pm files):
    Class/MOP/Method.pm
    Class/MOP/Instance.pm
    Class/MOP/Method/Accessor.pm
    Class/MOP/Package.pm
    Moose/Meta/Method.pm
    Moose/Meta/Attribute.pm
    Moose/Meta/Role.pm                  (with create_anon_role)
    Moose/Meta/Role/Composite.pm
    Moose/Meta/TypeConstraint.pm
    Moose/Meta/TypeConstraint/Enum.pm
    Moose/Util/MetaRole.pm              (apply_metaroles no-op)
    Moose/Exception.pm                  (overload "" + throw)

- Moose::Util::TypeConstraints::_Stub now @isa Moose::Meta::TypeConstraint.

- Moose::Util::TypeConstraints::_store now blesses results into
  _Stub. Was returning unblessed hashrefs, causing "Can't call
  method 'check' on unblessed reference" errors.

- New: find_or_parse_type_constraint (handles Maybe[Foo], Foo|Bar,
  ArrayRef[Foo], HashRef[Foo], ScalarRef[Foo]).

- New: export_type_constraints_as_functions.

- Moose.pm pre-loads Moose::Util::MetaRole so MooseX::* extensions
  that call apply_metaroles without a `use` line don't error out.

- dev/modules/moose_support.md: new column in baseline table,
  Phase 3 sub-phases marked done.

Effect on `./jcpan -t Moose` (Moose 2.4000 upstream):

| Metric                        | Before | After |
|-------------------------------|-------:|------:|
| Files executed                |    478 |   478 |
| Assertions executed           |   1419 |  2226 |
| Fully passing files           |     56 |    65 |
| Partially passing files       |    184 |   240 |
| Compile/load fail (no tests)  |    238 |   173 |
| Assertions ok                 |    953 |  1423 |
| Assertions fail               |    466 |   803 |

Net Phase 3: +807 assertions executed, +470 newly pass, +9
fully-green files, -65 files compile that previously didn't.

Cumulative across this PR (master baseline → end of Phase 3):
+30 fully-green files (35 → 65), +1610 assertions executed
(616 → 2226), +1051 newly passing (372 → 1423).

`make` clean on both backends.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock fglock force-pushed the feature/moose-stubs-round2 branch from fb05abc to c7c271b Compare April 27, 2026 14:35
fglock and others added 12 commits April 27, 2026 18:10
Follow-up to the initial Phase 3 commit. Adds:

- Class::MOP.pm pre-loads Class::MOP::{Class,Attribute,Method,
  Method::Accessor,Instance,Package} so `use Class::MOP;` is
  enough to call Class::MOP::Class->initialize, ::Attribute->new,
  etc. Without these requires, the package exists in @inc but
  isn't loaded, and tests die with "Can't locate object method ...
  via package Class::MOP::Class".

- Moose.pm pre-loads Moose::Meta::{Class,Role,Attribute,Method,
  Method::Delegation,TypeConstraint}, Moose::Exporter,
  Moose::Exception, Moose::Util, and Moose::Util::TypeConstraints.

- New skeleton stubs:
    Moose::Meta::Method::Constructor
    Moose::Meta::Method::Destructor
    Moose::Meta::Method::Accessor
    Moose::Meta::Method::Delegation

- Class::MOP::Method gets ->execute (calls $self->{body}->(@_)).

- Class::MOP::Class gets ->meta (returns a _FakeMeta for itself).

- Moose::_FakeMeta gets attribute-method introspection helpers
  (find_method_by_name alias for get_method, get_method_map,
  attribute_metaclass / method_metaclass / instance_metaclass /
  constructor_class / destructor_class), rebless_instance /
  rebless_instance_back, get_package_symbol /
  list_all_package_symbols, is_pristine /
  _check_metaclass_compatibility, immutable_options,
  add_before_method_modifier / add_after_method_modifier /
  add_around_method_modifier (delegating to
  Class::Method::Modifiers).

- Moose::Util::TypeConstraints gets get_type_constraint_registry
  (returns a Registry façade) and _parse_parameterized_type_constraint.

Effect on `./jcpan -t Moose` (Moose 2.4000):

| Metric                        | Phase 3 (initial) | After follow-ups |
|-------------------------------|------------------:|-----------------:|
| Files executed                |               478 |              478 |
| Assertions executed           |              2226 |             2450 |
| Fully passing files           |                65 |               71 |
| Partially passing files       |               240 |              259 |
| Compile/load fail (no tests)  |               173 |              148 |
| Assertions ok                 |              1423 |             1562 |
| Assertions fail               |               803 |              888 |

Cumulative across this PR (master baseline → end of Phase 3):
+36 fully-green files (35 → 71), +1834 assertions executed
(616 → 2450), +1190 newly passing (372 → 1562).

The last iteration added only +2 fully-green files (69 → 71) while
~25 more files now compile that previously didn't, confirming
diminishing returns. dev/modules/moose_support.md updated to note
that Phase D (bundle pure-Perl Moose) is the next move that
meaningfully advances the pass count.

`make` clean on both backends.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Surfaced during a Phase D plan-review (bundle pure-Perl Moose). Was
silently breaking Class::Load::PP and Package::Stash::PP version-string
detection, which both do:

  my $version = ${ $stash->get_symbol('$VERSION') };

Get_symbol returns *Pkg::VERSION{SCALAR}; on PerlOnJava that yielded
the scalar's *value* (e.g. "1.54") rather than a SCALAR reference.
Real Perl returns a SCALAR ref. Dereferencing the value with `${ ... }`
under strict refs threw "Can't use string as a SCALAR ref".

The ARRAY / HASH / GLOB cases all already used createReference();
the SCALAR case was the outlier. Fixed by mirroring those:

  yield GlobalVariable.getGlobalVariable(this.globName).createReference();
  // anonymous globs: yield this.scalarSlot.createReference();

Verification:

  ./jperl -e 'our $x = "hello"; print ref(*x{SCALAR})'
  # before: "" (the value)
  # after:  "SCALAR"

  ./jperl -e 'use Class::Load qw(load_class); load_class("Carp"); print "ok\n"'
  # before: Can't use string ("1.54") as a SCALAR ref ...
  # after:  ok

Regression test added to src/test/resources/unit/typeglob.t covering
named globs (read + write through ref) and anonymous globs.

Also updates dev/modules/moose_support.md with:

- Phase D plan-review findings (this fix + the prove --not workaround)
- Active Phase-D blocker: a separate refcount bug in
  Scalar::Util::weaken when called on a hash slot inside a sub. Minimal
  reproduction documented along with the suspected root cause
  (refCountOwned flag mismatch in WeakRefRegistry.java) and a checklist
  for resuming Phase D once that's fixed.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
… > 0

Phase D pre-work for the Moose port (see dev/modules/moose_support.md
"Plan: fix the weaken bug").

Bug: when weaken() was called on a hash slot inside a sub, with the
target also held by other strong refs in the caller's scope, all
weak refs to that target became undef immediately. Minimal repro:

  my $m = bless {}, "M";
  my %REG = (x => $m);
  sub attach {
      my ($attr, $class) = @_;
      $attr->{ac} = $class;
      Scalar::Util::weaken($attr->{ac});
  }
  attach($_, $REG{x}) for ({}, {}, {});
  # Real Perl: all three $arr[i]->{ac} stay defined.
  # PerlOnJava (before): all three became undef.

This is exactly the pattern Class::MOP::Attribute::attach_to_class
uses pervasively (`weaken($self->{associated_class} = $class)`),
called for every attribute during Class::MOP.pm's self-bootstrap.
Without this fix, `use Class::MOP;` died in the bootstrap, blocking
Phase D of the Moose port.

Root cause:
  MortalList.flush() runs maybeAutoSweep() on every flush.
  ReachabilityWalker.sweepWeakRefs(true) walks reachable roots
  (globals + ScalarRefRegistry) and clears weak refs to anything the
  walker doesn't reach. The walker does not seed from `my` lexical
  hashes / arrays, so a blessed object held only by `my %REG` in
  the caller's scope is invisible — "unreachable" — and got its
  weak refs cleared even though %REG still held a strong reference
  via its hash slot.

Fix: in quiet (auto-sweep) mode, skip clearing weak refs to any
referent whose cooperative refCount is still positive. Rationale:
PerlOnJava's refCount can drift due to JVM temporaries, but a
positive refCount means at least one tracked container thinks
it's holding a strong ref. Auto-sweep is heuristic — when the walker
disagrees with refCount, prefer the conservative "keep weak refs"
answer. Explicit `Internals::jperl_gc()` (non-quiet) still proceeds
because the user opted in to aggressive cleanup.

Verification:

- src/test/resources/unit/weaken_via_sub.t — new regression test
  with 20 assertions covering: 3-iteration loop, single attach,
  three separate calls, no-weaken sanity, no-other-strong-ref
  cleanup case, and the literal Class::MOP attach_to_class shape.
  Avoids `use Test::More;` masking effects by structuring assertions
  carefully.
- `make` — full unit suite green.
- `./jcpan -t DBIx::Class` — DBIC is the most refcount-heavy
  CPAN dist we test. Identical numbers before/after:
    Files=314, Tests=878, Failed=303, Assertions failing=2
  Zero regressions.
- `./jperl -e 'use Class::MOP; print "ok\n"'` → ok (was: died in
  bootstrap).
- `./jperl -e 'use Moose; print "ok\n"'` → ok.

Documentation: dev/modules/moose_support.md updated to mark the
weaken blocker resolved and re-enable the Phase D resumption
checklist.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Phase D D1-D3 (bundle upstream Moose, patch Class::MOP.pm, write
Class::MOP::PurePerl) was attempted on a feature/moose-phase-d
branch (now deleted; preserved as findings). The bundle and PurePerl
worked, but `use Class::MOP;` still dies in the self-bootstrap.

Root cause traced: PerlOnJava's MortalList.flush() (called from
RuntimeScalar.setLargeRefCounted line 1236, after every reference
assignment) decrements the bootstrap metaclass's refCount past 0
during ordinary Sub::Install method installations, triggering
DESTROY mid-bootstrap. Subsequent attach_to_class calls see
refCount=Integer.MIN_VALUE and the weaken immediately UNDEFs the
slot.

This is a SEPARATE bug from the auto-sweep weaken issue (which is
fixed in commit ca3af1a). It needs its own investigation:

- Hypothesis: setLargeRefCounted is double-counting a tracked-store.
- Suspects: MortalList.deferDecrementIfTracked over-adds to pending,
  or Sub::Install closure captures over-count ownership transitions.

dev/modules/moose_support.md updated with:
- The captured PJ_WEAKEN_TRACE refcount trace showing the metaclass
  bouncing 6→7→5→6→...→MIN_VALUE across the 9 attach_to_class calls.
- Investigation checklist for resuming.
- "Phase D resumption requires fixing both blockers" framing.

No code changes besides docs. The previously-shipped weaken fix
remains in place; DBIC + unit suite still green.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…mpt reverted

Followed Step W2-W6 plan in moose_support.md to investigate the
"MortalList.flush destroys metaclass during Class::MOP bootstrap"
blocker.

Findings:

1. The metaclass itself is NOT being destroyed — its refCount
   oscillates 0↔7 but never reaches Integer.MIN_VALUE (a
   localBindingExists=true guard correctly skips destroy each time).

2. The actual destroy that triggers the failure is on a DIFFERENT
   blessed object — likely an interim object held briefly by
   Sub::Install during method installation. Captured stack trace
   isolates the trigger to MortalList.flush() line 566 →
   DestroyDispatch.callDestroy → WeakRefRegistry.clearWeakRefsTo
   (clearing 4 weak refs).

3. Per-event refcount accounting for the failing object shows real
   asymmetry: 55 increments vs 87 effective decrements (45 immediate
   + 42 deferred). Pinpointing WHICH assignment is asymmetric requires
   deeper instrumentation than fits in this round.

4. A surgical "skip destroy when weak refs exist" guard was tried
   in MortalList.flush() but BROKE 5+ existing weaken/destroy unit
   tests (unit/refcount/weaken_destroy.t, weaken_edge_cases.t,
   weaken_basic.t, destroy_anon_containers.t). Reverted.

5. Doc updated with:
   - Captured PJ_RC=1 trace excerpt isolating the destroy trigger.
   - Stack trace pinning the bug to MortalList.flush via Sub::Install.
   - Concrete starting points for a future deep refcount audit
     (4 specific code sites: @_ aliasing, $h->{key} overwrite path,
     list-assignment from @_, Sub::Install closure captures).
   - Test gate for any future fix: weaken_via_sub.t + zero DBIC
     regressions.

DBIx::Class verification (regression check):
  Files=314 / Tests=878 / 303 failed files / 2 failing assertions
  IDENTICAL to baseline.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Per the user's directive to attempt a more refined fix this round,
two more variants of the "skip destroy when weak refs exist" guard
were tried and both reverted:

Attempt 1: skip destroy when weak refs exist (any object).
  Result: broke unit/weaken_basic.t pattern
    `my $strong = {data=>"hi"}; my $weak=$strong; weaken($weak);
     # inner block exit → $strong scope exits → $weak should clear`
  because skipping destroy here keeps $weak defined incorrectly.

Attempt 2: skip destroy when weak refs exist AND object is blessed.
  Applied at MortalList.flush() AND setLargeRefCounted's
  overwrite-decrement path (line 1192).
  Result: still broke cycle-breaking-via-weaken tests
  (weaken_destroy.t, weaken_edge_cases.t,
  destroy_anon_containers.t) which use blessed objects in cycles.
  Skipping destroy keeps the cycle alive forever.

Lesson: there's no simple predicate at the destroy gate that
distinguishes "transient refCount drift during heavy reference
shuffling" from "genuine end-of-life with weak refs about to clear".
The fix has to be in the refcount accounting itself, not at the
destroy gate.

dev/modules/moose_support.md updated with:
- Both attempt summaries and their failure modes.
- Concrete next-step direction: option (a) walker awareness of
  `my %hash` lexical containers, OR (b) refcount accounting symmetry
  audit on the four candidate sites (@_ aliasing, hash overwrite,
  list-assignment, Sub::Install closure captures).
- Explicit test gate for any future attempt.

DBIx::Class regression check (post-revert): identical to baseline
(Files=314 / Tests=878 / 303 failed files / 2 failing assertions).
make stays green.

The auto-sweep weaken fix (commit ca3af1a) and the *GLOB{SCALAR}
fix (commit 880bf65) are unaffected.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The previous doc revision said the refcount fix has to be in the
accounting itself, not at the destroy gate, but didn't say HOW.
This revision makes that concrete with three priority-ordered paths:

Path 1 (recommended start, ~1 day): walker awareness of hash-element
seeds. The walker currently filters scalars whose declaration scope
has exited — a check that's correct for `my $x` lexicals but wrong
for hash/array element scalars (which have no declaration scope of
their own). Fix: skip the scope-exit filter for scalars registered
via `incrementRefCountForContainerStore`. Use the enclosing
container's `localBindingExists` as the liveness signal instead.
With this, $METAS{HasMethods} becomes a walker root, so the
metaclass it points at is found as reachable.

Path 2 (~2 days): gate `MortalList.flush()` destroy on a per-object
reachability check. When the flush gate would fire DESTROY on a
blessed object with `refCount==0`, do a lightweight "is this single
object reachable from roots" walker query first. If yes, treat as
transient drift; if no, fire DESTROY (preserves cycle-break
semantics — isolated cycles correctly walk to "unreachable").

Path 3 (only if Paths 1+2 don't close the gap, ~3-4 days): refcount
accounting symmetry audit on the four candidate sites: @_ aliasing
on sub call/return, list-assignment from @_, hash-overwrite path,
Sub::Install closure captures. Includes a methodology for unit-
testing each site (per-operation refCount asserts, optionally via a
new SvREFCNT helper or via post-processed PJ_RC=1 trace).

Why this order:
- Path 1 alone might solve the bootstrap (walker correction is
  enough).
- Path 2 closes the gap if the walker is right but flush-destroy
  fires before the next sweep cycle.
- Path 3 only needed if real accounting asymmetry remains beyond
  walker-blindness.

Test gate (unchanged):
- src/test/resources/unit/weaken_via_sub.t                 (20/20)
- src/test/resources/unit/refcount/weaken_basic.t          (all ok)
- src/test/resources/unit/refcount/weaken_destroy.t        (all ok, cycle break)
- src/test/resources/unit/refcount/weaken_edge_cases.t     (all ok)
- src/test/resources/unit/refcount/destroy_anon_containers.t  (all ok)
- ./jperl -e 'use Class::MOP; print "ok\n"'                (ok)
- make                                                      (green)
- ./jcpan -t DBIx::Class                                    (11 green / 876 ok / 2 fail; matches baseline)

Doc-only commit. The auto-sweep weaken fix (commit ca3af1a) and
the *GLOB{SCALAR} fix (commit 880bf65) remain in place.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…cker

Implements Path 2 from dev/modules/moose_support.md (Step W3): a
walker-confirmed reachability check before MortalList.flush() and
setLargeRefCounted()'s overwrite-decrement path fire DESTROY on a
blessed object whose cooperative refCount dipped to 0.

Root problem: PerlOnJava's cooperative refCount drifts under heavy
reference shuffling (Class::MOP self-bootstrap weakens ~10 attribute
back-references to a single metaclass). Without this fix, transient
refCount==0 events fire DESTROY on objects that are still very much
held by `our %METAS` and other still-live containers, breaking
Class::MOP's load entirely.

Fix:

1. New `ReachabilityWalker.isReachableFromRoots(RuntimeBase)` — a
   bounded BFS that returns true as soon as `target` is found from
   any live root, with a hard 50K-visit cap. Cheap enough to call
   from the destroy gate per-event.

2. Roots are seeded from:
   - Package globals (globalCodeRefs, globalVariables, globalArrays,
     globalHashes).
   - ScalarRefRegistry-tracked scalars whose declaration scope is
     still live per `MyVarCleanupStack.isLive(sc)` AND `!sc.scopeExited`.
   - `MyVarCleanupStack.snapshotLiveVars()` — new helper that returns
     the currently-active my-var instances. THIS is what makes
     `$METAS{HasMethods}` reachable (its enclosing my %METAS is on
     the live-vars stack while Class::MOP.pm loads).
   - Rescued objects from DestroyDispatch.

3. Two gate sites:
   - `MortalList.flush()`: when refCount drops to 0 on a blessed
     object with weak refs registered, consult the walker. If still
     reachable, leave refCount at 0 (the next assignment bumps it
     back); don't fire DESTROY.
   - `RuntimeScalar.setLargeRefCounted()` (overwrite-decrement
     path): mirror gate.

Both gates are scoped on `base.blessId != 0 && hasWeakRefsTo(base)`,
so the walker call is only made for the rare case of a blessed
object with weak refs hitting refCount=0 — keeping the cost of the
common path unchanged.

Why this distinguishes the Moose case from cycle-break correctly:

- Moose case: `our %METAS` is in MyVarCleanupStack, walker finds
  the metaclass through it, returns true → skip DESTROY.
- Cycle-break case: cycle's lexicals exited their scope, so they
  are NOT in MyVarCleanupStack. The cycle has no path to roots,
  walker returns false → fire DESTROY normally → cycle freed.

Files changed:
- src/main/java/org/perlonjava/runtime/runtimetypes/ReachabilityWalker.java
- src/main/java/org/perlonjava/runtime/runtimetypes/MyVarCleanupStack.java
- src/main/java/org/perlonjava/runtime/runtimetypes/MortalList.java
- src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeScalar.java

Verification (all gates pass):

- src/test/resources/unit/weaken_via_sub.t                 (20/20 ok)
- src/test/resources/unit/refcount/weaken_basic.t          (34/34 ok)
- src/test/resources/unit/refcount/weaken_destroy.t        (24/24 ok, cycle break)
- src/test/resources/unit/refcount/weaken_edge_cases.t     (42/42 ok)
- src/test/resources/unit/refcount/destroy_anon_containers.t  (21/21 ok)
- `make`                                                    (full unit suite green)
- `./jcpan -t DBIx::Class`                                  (314 files / 878 tests / 303 failed files / 2 failing assertions — IDENTICAL to baseline; zero regressions)

The Class::MOP bootstrap blocker is RESOLVED. The bundled Moose
attempt now reaches the next downstream layer (an unrelated
issue at Class/MOP/Class/Immutable/Trait.pm line 59), which Phase
D's continuation can tackle separately.

dev/modules/moose_support.md updated to mark Path 2 done.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Both core-runtime fixes attempted during the Phase 3 → Phase D push
have been reverted:
- *GLOB{SCALAR} fix (broke Path::Class / DBIC overload setup)
- auto-sweep weaken + walker-gated destroy (broke DBIC t/52leaks.t)

DBIC is back at master parity (314 files / 13851 assertions / 0 failed
assertions). Documented the failure modes and the measurement
methodology mistake that allowed both regressions to be missed.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant