From 220a1578b9f6044fe94c8340362fe66746c3117d Mon Sep 17 00:00:00 2001 From: "Flavio S. Glock" Date: Thu, 30 Apr 2026 10:28:36 +0200 Subject: [PATCH] =?UTF-8?q?docs(refcount):=20rename=20"cooperative"=20?= =?UTF-8?q?=E2=86=92=20"selective";=20new=20memory-management.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The phrase "cooperative reference-counting overlay" was project-internal vocabulary with no presence in the GC literature. Renamed to "selective reference-counting overlay" throughout — "selective" accurately captures the design's defining property (refcount tracking is opt-in per object class, only for those needing DESTROY / weaken semantics) and is descriptive in standard GC terminology. Project-wide rename (~86 occurrences across 32 files): - All user-facing docs (docs/, AGENTS.md, jperl scripts) - All dev/ design and module notes - Source comments in src/main/java/.../runtimetypes/* and operators/* - B.pm and Class/MOP/Method/Accessor.pm Perl modules - Skipped: bundled CPAN/Perl docs (perlthrtut.pod, Moose cookbook), TLD list (PublicSuffix.pm "KPMG International Cooperative"), build/target/cpan_build_dir, "cooperatively" (different word). New docs/reference/memory-management.md: - User-facing summary of the selective refcount overlay (no project history — that lives in dev/architecture/weaken-destroy.md and dev/modules/moose_support.md). - "How the overlay works" mechanism table. - "Where this fits in the GC literature" section with verified links: - The Garbage Collection Handbook (Jones, Hosking, Moss) - Bacon, Cheng, Rajan 2004 "A Unified Theory of Garbage Collection" - Blackburn & McKinley 2003 "Ulterior Reference Counting" - Wikipedia: Reference counting / Tracing GC / Garbage collection / Finalizer - Java java.lang.ref package summary (JDK 21) - perldoc.perl.org perlobj#Destructors - Explains why "selective" rather than "deferred" or "ulterior" — the partition criterion is per-class behavioural (does this need finalization?), not generational or write-barrier-based. - Comparison table: Perl 5 vs JVM finalization vs PerlOnJava. docs/reference/architecture.md: - Replaced inline 14-line DESTROY section with a 4-line pointer to the new memory-management.md. - Added memory-management.md to "Related Documentation" list. dev/architecture/weaken-destroy.md (separate prior work in this branch): - Updated to reflect property-based walker gate (D-W6.18) replacing the class-name heuristic (Class::MOP/Moose/Moo allowlist). - Documented MortalList.flush()'s narrow auto-trigger (storedInPackageGlobal + hasWeakRefsTo) and its per-flush reachable-set cache. - Added section 10b on RuntimeBase.activeOwners / reachableOwnerCount infrastructure (D-W6.14 / D-W6.16). - Documented PJ_REFCOUNT_TRACE / PJ_WEAKCLEAR_TRACE / PJ_DESTROY_TRACE. Verification: - `make` passes (build + all unit tests). - `make check-links` passes (267/267 internal links OK). - All 8 external literature URLs verified with curl (HTTP 200 / valid 302 redirects to ACM for DOIs). Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- AGENTS.md | 2 +- dev/architecture/README.md | 2 +- dev/architecture/weaken-destroy.md | 164 +++++++++++++++--- dev/cpan-reports/Scalar-Util.md | 2 +- dev/design/refcount_alignment_plan.md | 10 +- dev/modules/anyevent_fixes.md | 6 +- dev/modules/dbix_class.md | 6 +- dev/modules/list_moreutils.md | 2 +- dev/modules/moose_support.md | 62 +++---- dev/modules/ppi.md | 6 +- .../DBIx-Class-0.082844/LeakTracer-README.md | 2 +- docs/about/changelog.md | 2 +- docs/about/relation-perlito.md | 2 +- docs/about/roadmap.md | 2 +- docs/reference/architecture.md | 14 +- docs/reference/feature-matrix.md | 4 +- docs/reference/memory-management.md | 135 ++++++++++++++ jperl | 2 +- jperl.bat | 2 +- .../org/perlonjava/core/Configuration.java | 4 +- .../runtime/operators/ReferenceOperators.java | 2 +- .../runtime/runtimetypes/DestroyDispatch.java | 4 +- .../runtime/runtimetypes/MortalList.java | 6 +- .../runtimetypes/ReachabilityWalker.java | 4 +- .../runtime/runtimetypes/RuntimeBase.java | 2 +- .../runtime/runtimetypes/RuntimeCode.java | 6 +- .../runtime/runtimetypes/RuntimeGlob.java | 2 +- .../runtime/runtimetypes/RuntimeHash.java | 2 +- .../runtime/runtimetypes/RuntimeScalar.java | 10 +- .../runtime/runtimetypes/WeakRefRegistry.java | 2 +- src/main/perl/lib/B.pm | 16 +- .../perl/lib/Class/MOP/Method/Accessor.pm | 2 +- .../unit/refcount/drift/our_metas_underflow.t | 4 +- 33 files changed, 375 insertions(+), 118 deletions(-) create mode 100644 docs/reference/memory-management.md diff --git a/AGENTS.md b/AGENTS.md index 1efbd758e..9cc1abedd 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -197,7 +197,7 @@ Example format at the end of a design doc: | Feature | Status | |---------|--------| -| `weaken` / `isweak` | Implemented. Uses cooperative reference counting on top of JVM GC. See `dev/architecture/weaken-destroy.md` for details. | +| `weaken` / `isweak` | Implemented. Uses selective reference counting on top of JVM GC. See `dev/architecture/weaken-destroy.md` for details. | | `DESTROY` | Implemented. Fires deterministically for tracked objects (blessed into a class with DESTROY). See `dev/architecture/weaken-destroy.md`. | | `Scalar::Util::readonly` | Works for compile-time constants (`RuntimeScalarReadOnly` instances). Does not yet detect variables made readonly at runtime via `Internals::SvREADONLY` (those copy type/value into a plain `RuntimeScalar` without replacing the object). | diff --git a/dev/architecture/README.md b/dev/architecture/README.md index ba9f0db65..6db64cb82 100644 --- a/dev/architecture/README.md +++ b/dev/architecture/README.md @@ -29,7 +29,7 @@ PerlOnJava is a Perl 5 implementation that compiles Perl source code to JVM byte | Document | Description | |----------|-------------| | [dynamic-scope.md](dynamic-scope.md) | Dynamic scoping via `local` and DynamicVariableManager | -| [weaken-destroy.md](weaken-destroy.md) | Cooperative reference counting, DESTROY, and weak references | +| [weaken-destroy.md](weaken-destroy.md) | Selective reference counting, DESTROY, and weak references | | [lexical-pragmas.md](lexical-pragmas.md) | Lexical warnings, strict, and features | | [control-flow.md](control-flow.md) | Control flow implementation (die/eval, last/next/redo, block dispatchers) | | [block-dispatcher-optimization.md](block-dispatcher-optimization.md) | Block-level shared dispatchers for control flow | diff --git a/dev/architecture/weaken-destroy.md b/dev/architecture/weaken-destroy.md index 818e43302..a81c6065b 100644 --- a/dev/architecture/weaken-destroy.md +++ b/dev/architecture/weaken-destroy.md @@ -1,6 +1,6 @@ # Weaken & DESTROY - Architecture Guide -**Last Updated:** 2026-04-24 +**Last Updated:** 2026-04-30 **Status:** PRODUCTION READY - 841/841 Moo subtests (100%) - 13858/13858 DBIx::Class subtests across 314 test files (100%, 0 Dubious) — measured on branch `perf/dbic-safe-port` at `2ef41907d` @@ -10,8 +10,9 @@ See also [dev/design/refcount_alignment_plan.md](../design/refcount_alignment_plan.md), [dev/design/refcount_alignment_progress.md](../design/refcount_alignment_progress.md), [dev/design/refcount_alignment_52leaks_plan.md](../design/refcount_alignment_52leaks_plan.md), -and [dev/design/perf-dbic-safe-port.md](../design/perf-dbic-safe-port.md) -for the 2026-04 alignment work that closes the remaining Perl-parity gaps. +[dev/design/perf-dbic-safe-port.md](../design/perf-dbic-safe-port.md), and +[dev/modules/moose_support.md](../modules/moose_support.md) (D-W6.7 → D-W6.18 walker-gate +investigation log) for the 2026-04 alignment work that closes the remaining Perl-parity gaps. --- @@ -41,7 +42,7 @@ The system is designed around three principles: reference-counting burden. Weak references are registered externally and cleared as a side-effect of DESTROY. -3. **Perl-semantics first.** When cooperative refcount drifts from Perl's +3. **Perl-semantics first.** When selective refcount drifts from Perl's accurate refcount (due to JVM temporaries, call-stack lexicals the walker can't see, etc.), the reachability walker (`ReachabilityWalker` + opt-in `Internals::jperl_gc()`) fills the gap, matching what Perl's refcount @@ -171,15 +172,16 @@ variable to trigger destruction. | File | Role | |------|------| -| `RuntimeBase.java` | Defines `refCount`, `blessId`, `destroyFired`, `currentlyDestroying`, `needsReDestroy`, `localBindingExists` fields on all referent types | +| `RuntimeBase.java` | Defines `refCount`, `blessId`, `destroyFired`, `currentlyDestroying`, `needsReDestroy`, `localBindingExists`, `storedInPackageGlobal`, `activeOwners` fields on all referent types; `recordActiveOwner` / `releaseActiveOwner` / `reachableOwnerCount` | | `RuntimeScalar.java` | `setLarge()` (increment/decrement), `scopeExitCleanup()`, `undefine()`, `incrementRefCountForContainerStore()` | | `RuntimeList.java` | `setFromList()` -- list destructuring with materialized copy refcount undo | -| `RuntimeHash.java` | `createReferenceWithTrackedElements()` (birth-tracking for anonymous hashes), `delete()` with deferred decrement | +| `RuntimeHash.java` | `createReferenceWithTrackedElements()` (birth-tracking for anonymous hashes), `delete()` with deferred decrement, `isGlobalPackageHash` flag | | `RuntimeArray.java` | `createReferenceWithTrackedElements()` (element tracking), `setFromListAliased()` (Phase 2: `@DB::args` population without refCount inflation) | | `WeakRefRegistry.java` | Weak reference tracking: forward set + reverse map; `snapshotWeakRefReferents()` for Phase 4 walker | | `DestroyDispatch.java` | DESTROY method resolution, caching, invocation; Phase 3 state machine (`currentlyDestroying` / `needsReDestroy`); `snapshotRescuedForWalk()` for Phase 4 walker | -| `MortalList.java` | Deferred decrements (FREETMPS equivalent); Phase 3 `pendingSize()` / `drainPendingSince()` for DESTROY body's deferred decrements | -| `ReachabilityWalker.java` | Phase 4 mark-and-sweep from Perl roots; `sweepWeakRefs()` clears weak refs for unreachable objects; `findPathTo()` diagnostic | +| `MortalList.java` | Deferred decrements (FREETMPS equivalent); Phase 3 `pendingSize()` / `drainPendingSince()` for DESTROY body's deferred decrements; property-based walker gate (D-W6.18) at `flush()` with per-flush reachable-set cache | +| `ReachabilityWalker.java` | Phase 4 mark-and-sweep from Perl roots; `sweepWeakRefs()` clears weak refs for unreachable objects; `findPathTo()` diagnostic; `isScalarReachable()` for owner-scalar liveness checks (D-W6.16) | +| `RuntimeBaseProxy.java` / `RuntimeHashProxyEntry.java` | Mark stored values as `storedInPackageGlobal` when the parent hash is a package-global (D-W6.18) | | `GlobalDestruction.java` | End-of-program stash walking | | `ReferenceOperators.java` | `bless()` -- activates tracking | | `RuntimeGlob.java` | CODE slot replacement -- optree reaping emulation | @@ -568,7 +570,7 @@ and is later released. **Path:** `org.perlonjava.runtime.runtimetypes.ReachabilityWalker` -Mark-and-sweep reachability walker for when cooperative refcount has +Mark-and-sweep reachability walker for when selective refcount has drifted beyond what `callDestroy` alone can reconcile. Walks the Perl- visible object graph from roots and clears weak refs for referents that no path reaches. @@ -600,16 +602,60 @@ no path reaches. | `walk()` | BFS from roots; returns set of reachable `RuntimeBase` instances. | | `sweepWeakRefs()` | Forces `System.gc()` via `ScalarRefRegistry.forceGcAndSnapshot()` (3 passes with WeakReference sentinels), drains `rescuedObjects`, runs `walk()`, clears weak refs for unreachable referents, fires DESTROY on blessed ones. | | `sweepWeakRefs(true)` | Quiet mode: clears weak refs but does NOT fire DESTROY — for use from safe-to-interrupt callers that must not run Perl code mid-operation. Currently only used by future Phase B2 work (none active). | +| `isScalarReachable(target)` | (D-W6.16) Walks roots looking for a specific `RuntimeScalar` identity (rather than a `RuntimeBase` referent). Skips weak refs and follows closure captures. Used by `RuntimeBase.reachableOwnerCount()` to filter the `activeOwners` set down to scalars that are actually live (not phantom entries left over after their JVM frame slot was nulled). | | `findPathTo(target)` | Diagnostic: returns first path string (e.g. `"%DBIx::Class::Schema::{accessors}{schema}"` or `""`) found to the target, or null. | -**When not to run automatically:** Phase B2 (auto-trigger from hot -paths) was attempted and reverted — see comment in -`Scalar::Util::isweak()` and `MortalList.flush()`. Even Phase B1's -lexical-aware sweep can't safely run inside module-init chains -(e.g. DBICTest::BaseResult's use-chain relies on weak-refed state -remaining defined). Auto-triggering requires a compiler-emitted -"outside-of-module-init" marker which jperl doesn't yet have. -Use `Internals::jperl_gc()` explicitly for leak-tracer integration. +**Auto-trigger from hot paths (D-W6.18 — property-based walker gate).** +A previous attempt at "auto-trigger from hot paths" (Phase B2) was reverted +because seeding the walker from globals + `rescuedObjects` alone misclassified +live-via-lexical objects as unreachable. That ground has now been recovered +with a far narrower trigger: + +`MortalList.flush()` consults the walker before letting `refCount → 0` +fire DESTROY, **but only when** the referent satisfies all of: + +1. `base.blessId != 0` — blessed object; +2. `base.storedInPackageGlobal` — currently held as a value of a + package-global hash (set at `RuntimeBaseProxy.set` time when the + parent `RuntimeHashProxyEntry`'s hash is `isGlobalPackageHash`); +3. `WeakRefRegistry.hasWeakRefsTo(base)` — at least one outstanding + weak ref points at it. + +When all three hold, the walker is consulted. If the object is still +reachable from Perl roots, `MortalList.flush()` rescues it (skips +DESTROY, restores `refCount = 1`, and re-marks it as owned by the +appropriate package-hash entry). Otherwise the normal destruction path +runs. + +The same gate is mirrored in `RuntimeScalar.setLargeRefCounted` on the +overwrite path so a transient-zero during reassignment doesn't fire +DESTROY for a still-reachable package-global object. + +This replaces the earlier class-name-based heuristic +(`DestroyDispatch.classNeedsWalkerGate`, hard-coded list of +`Class::MOP` / `Moose` / `Moo`) with a structural property that +correctly captures *why* those classes need rescuing: their metaclass +instances live as values in `our %METAS`-style package hashes whose +selective refcount transient-zeros mid-statement during method +dispatch. `classNeedsWalkerGate` is retained only for opt-in +diagnostic tracing (`PJ_REFCOUNT_TRACE` / `PJ_WEAKCLEAR_TRACE`); no +production code path consults it any more. + +**Per-flush reachable-set cache.** Because the gate now fires for +DBIC's many blessed-into-package-globals objects (Schema, +ResultSource, Storage::DBI, …) under heavy weaken usage, an O(N×G) +naïve implementation (`isReachableFromRoots(target)` per gate fire, +where G = #globals and N = #flush targets) would surface as a 100% +CPU spin on `t/100populate*.t`. `MortalList.flush()` therefore walks +the reachable set **once** per outer flush invocation via +`ReachabilityWalker.walk()` and reuses it for O(1) membership lookups +on subsequent gate fires within the same flush. The cache is cleared +in the `finally` block so the next flush sees fresh global state. + +**Manual sweep is still opt-in.** `Internals::jperl_gc()` remains the +only caller-driven full sweep. The auto-gate above only fires inside +`MortalList.flush()` on the narrow `storedInPackageGlobal` & +`hasWeakRefsTo` slice, which is safe to run from any flush point. ### 10a. ScalarRefRegistry (Phase B1) @@ -632,6 +678,66 @@ walker reads the snapshot. Opt out for benchmarking: `JPERL_NO_SCALAR_REGISTRY=1`. +### 10b. Active-Owner Tracking (D-W6.14 / D-W6.16) + +**Path:** `RuntimeBase.activeOwners` / `recordActiveOwner` / +`releaseActiveOwner` / `activeOwnerCount` / `reachableOwnerCount` + +A precise per-referent owner set, parallel to `refCount` but tracking +*which* `RuntimeScalar`s currently own each increment rather than just +the count. `activeOwners` is an `IdentityHashMap`-backed +`Set`, lazily allocated via `activateOwnerTracking()`. + +Every `++base.refCount` increment site is paired with a +`base.recordActiveOwner(scalar)` call, and every `--base.refCount` +decrement site with a matching `base.releaseActiveOwner(scalar)`. The +audit covers the full set: + +- `RuntimeScalar.setLargeRefCounted` (store + overwrite + undefine) +- `RuntimeScalar.incrementRefCountForContainerStore` +- `RuntimeArray.shift` / `pop` (deferred-decrement path) +- `RuntimeList` materialized-copy undo (4 sites: undef-target, + scalar-target, array-target, hash-target) +- `Storable.releaseApplyArgs` +- `DestroyDispatch.doCallDestroy` (args.push balance) +- `MortalList.deferDecrementIfTracked` / + `deferDecrementRecursive` / `deferDestroyForContainerClear` +- `WeakRefRegistry.weaken` +- `RuntimeStash.dynamicRestoreState` + +`reachableOwnerCount()` walks `activeOwners`, filters to scalars that +still satisfy `refCountOwned == true && value == this`, and asks +`ReachabilityWalker.isScalarReachable(scalar)` whether each owning +scalar is reachable from Perl roots. The resulting count is a precise +"how many genuinely-live owners does this referent still have" +metric — strictly more accurate than raw `refCount`, which can be +inflated by JVM temporaries and stale-but-not-yet-GC'd lexicals. + +**Status: instrumented but not yet activated as the production rescue +criterion.** The property-based walker gate in section 10 (which +checks `storedInPackageGlobal` + `hasWeakRefsTo`) is sufficient for +all currently-exercised modules (Class::MOP, Moose, Moo, DBIx::Class, +Template::Toolkit). `reachableOwnerCount()` is available for future +work that needs a finer-grained "is this referent really still +owned?" check — for example, replacing the `storedInPackageGlobal` +property with `reachableOwnerCount() > 0` once owner-set population +catches all increment sites that pre-date `activateOwnerTracking()`. + +**Diagnostic tracing.** Three env-flag-gated tracers print to stderr: + +- `PJ_REFCOUNT_TRACE` — per-`RuntimeBase` refCount transitions, with + abbreviated stack snippets and owner record/release events. Activated + at bless time for classes matching the legacy + `DestroyDispatch.classNeedsWalkerGate` list (Class::MOP / Moose / + Moo). A JVM shutdown hook dumps each tracked base's surviving + `activeOwners` set. +- `PJ_WEAKCLEAR_TRACE` — `WeakRefRegistry.weaken` / + `removeWeakRef` / `clearWeakRefsTo` events. +- `PJ_DESTROY_TRACE` — `DestroyDispatch.callDestroy` invocations. + +All three are zero-cost when their env flags are unset (single +boolean check at the call site). + ### 11. `Internals::*` Perl-Visible API **Path:** `org.perlonjava.runtime.perlmodule.Internals` @@ -766,7 +872,7 @@ my @kept; ### Example 6: Reachability Sweep (Phase 4) -For leak-tracer-style scripts where cooperative refcount inflates beyond +For leak-tracer-style scripts where selective refcount inflates beyond what `callDestroy` alone resolves. ```perl @@ -783,7 +889,7 @@ sub register { my $obj = DBICTest::Artist->new(...); register($obj); # ... lots of DBIC machinery creates JVM temporaries that inflate - # $obj's cooperative refCount ... + # $obj's selective refCount ... } # At this point $obj's lexical is gone, but refCount > 0 due to inflation. @@ -873,7 +979,7 @@ decrement per reference assignment), but this is by design. | Aspect | Perl 5 | PerlOnJava | |--------|--------|------------| | Tracking scope | Every SV has a refcount | Only blessed-into-DESTROY objects, anonymous containers, closures with captures, and weaken targets | -| GC model | Deterministic refcounting + cycle collector | JVM tracing GC + cooperative refcounting overlay + opt-in reachability sweep | +| GC model | Deterministic refcounting + cycle collector | JVM tracing GC + selective refcounting overlay + opt-in reachability sweep | | Circular references | Leak without weaken | Handled by JVM GC (weaken still needed for DESTROY timing) | | `weaken()` on the only ref | Immediate DESTROY | Same behavior | | DESTROY timing | Immediate when refcount hits 0 | Same for tracked objects; untracked objects rely on JVM GC | @@ -930,11 +1036,15 @@ decrement per reference assignment), but this is by design. 7. **Reachability walker can't see live JVM-call-stack lexicals.** Phase 4's `ReachabilityWalker` walks from globals and `rescuedObjects` but not into - per-frame Java locals. Running an auto-triggered sweep is therefore - unsafe (it would clear weak refs to objects that are alive in some live - lexical). `Internals::jperl_gc()` is opt-in for exactly this reason — - the caller is responsible for ensuring the current frame's lexicals - aren't holding objects that should survive. + per-frame Java locals. A *full* auto-triggered sweep is therefore still + unsafe in general — it would clear weak refs to objects that are alive + in some live lexical. The narrow auto-gate at `MortalList.flush()` + (D-W6.18: blessed + `storedInPackageGlobal` + `hasWeakRefsTo`) is the + exception: that triple guard is structurally restricted to objects + whose canonical owner is a package-global hash entry, so reachability + from globals is sufficient. `Internals::jperl_gc()` remains opt-in for + the same reason — the caller is responsible for ensuring the current + frame's lexicals aren't holding objects that should survive. 8. **Reachability walker does not follow `RuntimeCode.capturedScalars` by default.** Sub::Quote and Moo generate accessor closures that capture @@ -944,7 +1054,7 @@ decrement per reference assignment), but this is by design. captures. Opt in via `ReachabilityWalker.withCodeCaptures(true)` if you need the more conservative traversal. -9. **`Internals::SvREFCNT` is approximate.** Cooperative refCount +9. **`Internals::SvREFCNT` is approximate.** Selective refCount under-counts stack / JVM temporaries vs native Perl. `B::SV::REFCNT` (in `bundled-modules/B.pm`) relies on the +1 inflation from `$self->{ref}` hash storage to compensate for this under-counting; removing either @@ -989,7 +1099,7 @@ Tests are organized in four tiers: ## See Also - [dev/design/destroy_weaken_plan.md](../design/destroy_weaken_plan.md) -- Design document with implementation history, strategy analysis, and evolution of the WEAKLY_TRACKED design -- [dev/design/refcount_alignment_plan.md](../design/refcount_alignment_plan.md) -- 2026-04 plan for aligning cooperative refcount with Perl semantics (phases 0-7) +- [dev/design/refcount_alignment_plan.md](../design/refcount_alignment_plan.md) -- 2026-04 plan for aligning selective refcount with Perl semantics (phases 0-7) - [dev/design/refcount_alignment_progress.md](../design/refcount_alignment_progress.md) -- Per-phase progress log - [dev/design/perf-dbic-safe-port.md](../design/perf-dbic-safe-port.md) -- 2026-04-24 post-merge branch plan - [dev/modules/moo.md](../modules/moo.md) -- Moo test tracking and category-by-category fix log diff --git a/dev/cpan-reports/Scalar-Util.md b/dev/cpan-reports/Scalar-Util.md index 7a96b8eec..9f4cd9832 100644 --- a/dev/cpan-reports/Scalar-Util.md +++ b/dev/cpan-reports/Scalar-Util.md @@ -34,7 +34,7 @@ All 14 standard Scalar::Util EXPORT_OK functions are declared and registered. | `blessed` | Full | Handles blessed refs and `qr//` (implicit "Regexp" blessing) | | `refaddr` | Full | Uses `System.identityHashCode()` (JVM -- not real memory address) | | `reftype` | Full | Handles SCALAR, REF, ARRAY, HASH, CODE, GLOB, FORMAT, REGEXP, VSTRING | -| `weaken` | Full | Cooperative reference counting on JVM GC. Well tested. | +| `weaken` | Full | Selective reference counting on JVM GC. Well tested. | | `unweaken` | Full | Restores strong reference | | `isweak` | Full | Delegates to `WeakRefRegistry.isweak()` | | `dualvar` | Full | Creates `DualVar` record with separate numeric/string values | diff --git a/dev/design/refcount_alignment_plan.md b/dev/design/refcount_alignment_plan.md index 8d357d811..4623c14ca 100644 --- a/dev/design/refcount_alignment_plan.md +++ b/dev/design/refcount_alignment_plan.md @@ -18,7 +18,7 @@ destruction semantics: diagnostics/leak detection. DBIC, Moose/Moo, Sub::Quote, File::Temp, Devel::StackTrace, and many cache/ORM -modules all depend on these semantics. Today PerlOnJava's **cooperative +modules all depend on these semantics. Today PerlOnJava's **selective refcount** approximates them, but it diverges in enough places that several real-world tests fail (DBIC t/52leaks tests 12–18, txn_scope_guard test 18, etc.), and further real-world modules fail silently. This limits PerlOnJava's @@ -35,7 +35,7 @@ This document lays out a phased plan to close the gap so that: ## 2. Why the Current Scheme Falls Short -PerlOnJava uses **cooperative reference counting** layered on top of JVM GC: +PerlOnJava uses **selective reference counting** layered on top of JVM GC: - `RuntimeBase.refCount` is an `int` with state machine values: `-1` (untracked), `0` (tracked, no counted refs), `>0` (N counted refs), @@ -99,7 +99,7 @@ Specifically: ## 4. Strategy Overview -Keep cooperative refcounting as the *primary* mechanism, but add: +Keep selective refcounting as the *primary* mechanism, but add: - **Scope-exit decrement completeness** — ensure every path that increments has a matching path that decrements when the holder goes out of scope. @@ -337,14 +337,14 @@ Each phase is independently shippable. Rollback is per-commit. | 1 (Scope exit) | Could break closures/eval/goto by over-decrementing | Large test corpus from Phase 0; feature-flag behind `JPERL_STRICT_SCOPE_EXIT=1` during validation | | 2 (`@_` aliasing) | XS / C-level assumptions could break | Feature-flag `JPERL_ALIASED_AT_UNDERSCORE=1`; keep old behavior as fallback for first release | | 3 (DESTROY FSM) | Resurrection cycles if state machine has bugs | Loop detection (fail fast with RuntimeException above 1000 DESTROY calls on same object) | -| 4 (Reachability) | Cost; rarely-triggered edge cases (tied vars, weak refs into globs) | Profile extensively; amortize via periodic not per-op; keep current cooperative refcount as source of truth, reachability as fallback | +| 4 (Reachability) | Cost; rarely-triggered edge cases (tied vars, weak refs into globs) | Profile extensively; amortize via periodic not per-op; keep current selective refcount as source of truth, reachability as fallback | | 5 (REFCNT API) | CPAN modules with specific REFCNT expectations might break | Opt-in via `JPERL_ACCURATE_REFCNT=1` for one release; default-on in next | | 6 (CPAN validation) | Modules may need small patches for their own test bugs | Apply via `dev/patches/cpan/` if module's test is jperl-unaware | | 7 (Interpreter) | Double the work | Share semantic helpers between backends via `runtime` classes | ## 7. What Stays the Same -- JVM GC remains the memory manager. Cooperative refCount is *metadata*, +- JVM GC remains the memory manager. Selective refCount is *metadata*, not storage. - `MortalList` / `DynamicState` stack discipline unchanged. - Existing compile-time optimizations (constant folding, type propagation) diff --git a/dev/modules/anyevent_fixes.md b/dev/modules/anyevent_fixes.md index df847dbb6..29289f6fb 100644 --- a/dev/modules/anyevent_fixes.md +++ b/dev/modules/anyevent_fixes.md @@ -18,7 +18,7 @@ including low-priority ones. **Note**: `./jcpan -t AnyEvent` stops at `t/02_signals.t` because that test outputs `Bail out!` on failure, which aborts the entire harness run after only 3 files. The signal failure is downstream of the -`weaken`/cooperative-refcount limitation documented in `AGENTS.md` +`weaken`/selective-refcount limitation documented in `AGENTS.md` (timer/io watchers are destroyed immediately because `weaken` too eagerly clears the last strong ref). This is being addressed in a separate branch. Running tests individually would reveal the per-file @@ -177,7 +177,7 @@ not ok 5 # weakened timer still fires not ok 6 # twin (expected/unexpected) of 5 ``` -Our `weaken` is cooperative-refcount based (per AGENTS.md) and doesn't +Our `weaken` is selective-refcount based (per AGENTS.md) and doesn't match Perl's eager-free semantics in this specific pattern: ```perl @@ -232,7 +232,7 @@ Result: 82/83 → 13/83 test-program failures; subtests 24 → 157. - `fork`: implement, or reach agreement that fork-dependent tests are exempt from the "all tests must pass" rule? -- `weaken` semantics: cooperative-refcount is documented in `AGENTS.md` +- `weaken` semantics: selective-refcount is documented in `AGENTS.md` — do we strengthen it for this test, or treat t/13_weaken #5–#6 as a known deviation? diff --git a/dev/modules/dbix_class.md b/dev/modules/dbix_class.md index 6845cdd9f..98d80586a 100644 --- a/dev/modules/dbix_class.md +++ b/dev/modules/dbix_class.md @@ -141,7 +141,7 @@ When scope exits, scalar releases 1 reference but hash stays at refCount > 0. `c ### Next Steps -Both remaining failures (t/52leaks.t tests 12-18 and t/storage/txn_scope_guard.t test 18) hit **fundamental limitations** of PerlOnJava's cooperative refCounting that can't be solved without a major architectural change: +Both remaining failures (t/52leaks.t tests 12-18 and t/storage/txn_scope_guard.t test 18) hit **fundamental limitations** of PerlOnJava's selective refCounting that can't be solved without a major architectural change: #### Why t/52leaks.t tests 12-18 Are Blocked @@ -162,7 +162,7 @@ Attempted fix (Fix 10n attempt #2): Set `refCount = 0` during DESTROY body (not **Failure mode**: `my $self = shift` inside DESTROY body increments `refCount` to 1 via `setLargeRefCounted` when `$self` is assigned. When DESTROY returns, `$self` is a Java local that goes out of scope without triggering a corresponding decrement (PerlOnJava lexicals don't hook scope-exit decrements for scalar copies). Post-DESTROY `refCount=1` → false resurrection detection → loops indefinitely on File::Temp DESTROY during DBIC test loading. -Root cause: PerlOnJava's cooperative refCount scheme can't accurately track the net delta from a DESTROY body, because lexical assignments increment but lexical destruction doesn't always decrement. +Root cause: PerlOnJava's selective refCount scheme can't accurately track the net delta from a DESTROY body, because lexical assignments increment but lexical destruction doesn't always decrement. #### What Would Fix Both @@ -180,7 +180,7 @@ Deferred until such architectural work becomes practical. 2. **`createReference()` audit** — Fixed: Storable, DBI. Other deserializers (JSON, XML::Parser) don't appear in the DBIC leak pattern. 3. **Targeted refcount inflation sources** — function-arg copies tracked via `originalArgsStack` (Fix 10l), @DB::args preservation works; but inflation in `map`/`grep`/`keys` temporaries remains. -### Cooperative Refcounting Internals (reference) +### Selective Refcounting Internals (reference) **States**: `-1`=untracked; `0`=tracked, 0 counted refs; `>0`=N counted refs; `-2`=WEAKLY_TRACKED; `MIN_VALUE`=DESTROY called. diff --git a/dev/modules/list_moreutils.md b/dev/modules/list_moreutils.md index 6f377e61c..8e079ccb4 100644 --- a/dev/modules/list_moreutils.md +++ b/dev/modules/list_moreutils.md @@ -217,7 +217,7 @@ All 4492 subtests must pass. Rerun `make` to ensure no unit-test regressions. Scalar::Util::weaken($ref); is($ref, undef, "weakened away"); ``` - In real perl the temporary returned by `indexes(...)` has a refcount of 1 held by `$ref`; weakening that ref drops the refcount to 0 and the temporary is freed, so `$ref` becomes undef. PerlOnJava's cooperative-refcount overlay (see `dev/architecture/weaken-destroy.md`) only tracks objects blessed into a class with `DESTROY`. For an unblessed numeric scalar like this one, weaken transitions it to `WEAKLY_TRACKED` but does not clear the weak ref at scope exit because we can't distinguish "last strong ref was this one" from "symbol table still holds a ref" without full refcounting. This is a known architectural limitation being addressed on a separate branch; this PR does not touch it. + In real perl the temporary returned by `indexes(...)` has a refcount of 1 held by `$ref`; weakening that ref drops the refcount to 0 and the temporary is freed, so `$ref` becomes undef. PerlOnJava's selective-refcount overlay (see `dev/architecture/weaken-destroy.md`) only tracks objects blessed into a class with `DESTROY`. For an unblessed numeric scalar like this one, weaken transitions it to `WEAKLY_TRACKED` but does not clear the weak ref at scope exit because we can't distinguish "last strong ref was this one" from "symbol table still holds a ref" without full refcounting. This is a known architectural limitation being addressed on a separate branch; this PR does not touch it. ### Final summary diff --git a/dev/modules/moose_support.md b/dev/modules/moose_support.md index e0c5e1730..5c1bc0c9b 100644 --- a/dev/modules/moose_support.md +++ b/dev/modules/moose_support.md @@ -48,7 +48,7 @@ will do the real port — they're complementary, not alternatives. These are listed only because they were "out of scope" / "blockers" in earlier revisions of this document; they no longer are: -- **`weaken` / `isweak`** — implemented in core (cooperative reference +- **`weaken` / `isweak`** — implemented in core (selective reference counting on top of JVM GC). See `dev/architecture/weaken-destroy.md`. - **`DESTROY` / `DEMOLISH` timing** — implemented in core; fires deterministically for tracked blessed objects. Moose's `DEMOLISH` @@ -859,7 +859,7 @@ during `Class::MOP.pm`'s self-bootstrap). Without the fix The auto-sweep (`MortalList.maybeAutoSweep` → `ReachabilityWalker.sweepWeakRefs(true)`) was clearing weak refs to -blessed objects whose cooperative `refCount > 0` simply because the +blessed objects whose selective `refCount > 0` simply because the walker couldn't see them as reachable. The walker only seeds from package globals and `ScalarRefRegistry`; it doesn't seed from `my` lexical hashes or arrays. A blessed object held only by a `my %REG` @@ -870,7 +870,7 @@ in the caller's scope is therefore invisible to the walker — `ReachabilityWalker.sweepWeakRefs(quiet=true)` now skips clearing weak refs whose referent has `refCount > 0`. Reasoning: PerlOnJava's -cooperative refCount can drift due to JVM temporaries, but a positive +selective refCount can drift due to JVM temporaries, but a positive refCount means at least one tracked container thinks it's holding a strong reference. Auto-sweep should be conservative; explicit `Internals::jperl_gc()` (non-quiet) still clears, since the user @@ -965,7 +965,7 @@ guard, the cycle stays alive forever. Reverted. **Lesson**: there's no simple predicate that distinguishes "transient refCount drift during heavy reference shuffling" from -"genuine end-of-life with weak refs about to clear". The cooperative +"genuine end-of-life with weak refs about to clear". The selective refCount system doesn't carry enough information at the destroy gate to make this call. The fix has to be in the **accounting itself**, not at the destroy gate. @@ -1218,7 +1218,7 @@ After Paths 1 and 2 land: ####### What success does NOT mean -The cooperative refCount may still over-count in some cases (objects +The selective refCount may still over-count in some cases (objects hold refCount > 0 after they're truly dead). That's acceptable: the existing auto-sweep will reap them on the next walker cycle. The problematic direction — under-counting that fires DESTROY too early @@ -1634,7 +1634,7 @@ fix-level work: `Class/MOP/PurePerl.pm`. 6. **Patch Class::MOP::Method::Accessor** to skip - `weaken($self->{attribute})`. The cooperative refCount can't + `weaken($self->{attribute})`. The selective refCount can't keep the attribute alive across the brief window between `weaken` and `_initialize_body`. Trade: leaks attribute objects at global destruction. @@ -1810,7 +1810,7 @@ Moose stays at 412/478, refcount unit tests stay green. untracked to refCount=0+ tracked, scan ScalarRefRegistry for scalars holding the object and back-increment). - 2. Replace the cooperative refCount mechanism with a more + 2. Replace the selective refCount mechanism with a more reliable scheme (e.g. JVM-level identity hashmap keyed by referent, counting actual scalar holders). @@ -1991,7 +1991,7 @@ Tests fixed: - A handful of cmop/method introspection edge cases (constants, forward declarations, eval-defined subs). -### D-W6.7: Pinpointed root cause — %METAS storage doesn't bump cooperative refCount +### D-W6.7: Pinpointed root cause — %METAS storage doesn't bump selective refCount **Date:** 2026-04-26 **Branch:** `fix/d-w6-precise-die-probe` @@ -2027,7 +2027,7 @@ install_accessors: assoc=UNDEF **The bug:** the metaclass `RuntimeHash` (id=1746570062) is stored in `our %METAS` (`store_metaclass_by_name $METAS{$name} = $self`). -Despite `%METAS` strongly holding it, the cooperative refCount drops +Despite `%METAS` strongly holding it, the selective refCount drops to 0 at end-of-statement when the temporary returned by `HasMethods->meta` falls out of scope and `MortalList.flush()` processes its mortals. `clearWeakRefsTo` is called on the metaclass, @@ -2042,7 +2042,7 @@ remove_accessors → _remove_accessor` at `Class/MOP/Attribute.pm:475`, which dies again with the visible `Can't call method "get_method" on an undefined value` message. -**Root cause statement:** the cooperative refCount is failing to +**Root cause statement:** the selective refCount is failing to count the strong reference held by the package variable hash slot `$METAS{$package_name}`. When the temporary metaclass return-value from `->meta` expires, refCount goes from 1 → 0 even though `%METAS` @@ -2066,7 +2066,7 @@ package-variable-hash reachability. Two options for a real fix: 1. **Fix the refCount discipline:** ensure `RuntimeHash.put()` / - slot-assignment increments the cooperative refCount of the + slot-assignment increments the selective refCount of the referent when storing a blessed/tracked RuntimeBase value. Find why this doesn't happen for `$METAS{$name} = $meta`. @@ -2149,7 +2149,7 @@ The 1→0 transition occurs during `MortalList.flush()` triggered at the end of statement at `Class/MOP/Class.pm:260` (`my $super_meta = Class::MOP::get_metaclass_by_name(...)`). At that moment the metaclass should have refCount==1 (held by -`our %METAS`), but the cooperative count goes to 0 because one of +`our %METAS`), but the selective count goes to 0 because one of the 50 increment sites does **not** end up paired with the right decrement (one extra `pending.add(metaclass)` queued without a paired increment, or one increment lost without queueing a paired @@ -2191,7 +2191,7 @@ the instrumentation. The user-stated requirement: **the class-name heuristic is not acceptable**. We must find and fix the actual unpaired -increment/decrement in cooperative refCount discipline. +increment/decrement in selective refCount discipline. #### What we know from the tracer @@ -2298,7 +2298,7 @@ Once the underlying bug is fixed, the gate at } ``` -Or even simpler: remove the gate entirely once cooperative refCount +Or even simpler: remove the gate entirely once selective refCount is correct and DBIC's leak detection passes without rescue. **Step 5: Acceptance gates.** @@ -2350,7 +2350,7 @@ The two surviving owners both came from `setLargeRefCounted store`: This proves the imbalance is **not in the tracer instrumentation**: those owner scalars genuinely still hold strong refs to the -destroyed metaclass. Cooperative refCount said "0 strong refs" +destroyed metaclass. Selective refCount said "0 strong refs" while in fact 2 strong refs exist. #### Smoking-gun candidates @@ -2374,7 +2374,7 @@ this.value = lvalue.value; // proxy ALSO points to base, but no recordOwner ``` The proxy's `this.value` field then holds a strong reference to the -base **invisible to cooperative refcounting**. When the proxy is +base **invisible to selective refcounting**. When the proxy is later assigned a new value, the proxy's `set()` calls `lvalue.set(new_value)` which decrements old base's refCount via the overwrite path — but only because `lvalue.refCountOwned` is true. If @@ -2397,7 +2397,7 @@ through the queueing-then-flushing protocol): original increment), or b. removing the decrement (it was unpaired in the first place). -Once all sites are cleanly paired, the cooperative refCount becomes +Once all sites are cleanly paired, the selective refCount becomes self-consistent and the walker-gate heuristic can be removed. #### Files committed on this branch @@ -2449,7 +2449,7 @@ heuristic): | Unit tests: PASS | all green | | DBIC `t/52leaks.t`: 9–10 of 18 fail | leak rescue too aggressive — keeps cycle members alive that real Perl correctly leaks | -The fundamental issue: cooperative refCount cannot tolerate +The fundamental issue: selective refCount cannot tolerate cycles without weaken. My filter `refCountOwned && value == this` finds true strong owners, including cycle members that real Perl also leaks but DBIC's leak test expects to see destroyed (likely @@ -2489,7 +2489,7 @@ side-effect. #### Next step (D-W6.14) -The cooperative refCount underflow is real (D-W6.10 trace shows +The selective refCount underflow is real (D-W6.10 trace shows the metaclass refCount going to 0 with 2 surviving strong owners — D-W6.12). The 2 unpaired increments are still unidentified. The audit of direct `--refCount` sites pointed at all known paths, @@ -2524,7 +2524,7 @@ allowing each reference-counted graph to collapse properly when an external strong reference is dropped. For PerlOnJava to behave the same way, we need: -1. **Precise cooperative refCount** — every increment paired with +1. **Precise selective refCount** — every increment paired with exactly one decrement, no transient zeros. 2. **Effective weaken()** — weakening must decrement refCount and exclude the slot from owner-counting (already done correctly). @@ -2533,7 +2533,7 @@ For PerlOnJava to behave the same way, we need: #### What's wrong today -Cooperative refCount has **transient zeros**. The deferred +Selective refCount has **transient zeros**. The deferred decrement model (`MortalList.flush`) means a sequence like `inc → queue → inc → flush → flush → inc` can have refCount briefly hit 0 between the second flush and the third inc — even though the @@ -2555,7 +2555,7 @@ strong owners. But: 2. **For DBIC row objects in test cycles**: `populate_weakregistry` weak-refs the row. Then test does `undef $row` in some scope. Real Perl: refcount drops to 0, DESTROY fires, weak ref clears. - PerlOnJava: cooperative refCount has phantom owners — typically + PerlOnJava: selective refCount has phantom owners — typically container element scalars whose containers are themselves transient/dying but haven't yet released their elements. @@ -2568,7 +2568,7 @@ Rescue keeps them alive, breaking the leak detection. System Perl's approach (precise refcount + programmer-controlled weaken) maps to PerlOnJava as: -**Goal**: eliminate transient zeros from cooperative refCount +**Goal**: eliminate transient zeros from selective refCount without changing the deferred-decrement architecture. **Algorithm options:** @@ -2692,7 +2692,7 @@ The walker doesn't find Schema's owner. Likely candidates: RuntimeStash for perf). The walker is fundamentally **a snapshot of live JVM state at -this exact moment**. Cooperative refCount can hit 0 transiently +this exact moment**. Selective refCount can hit 0 transiently DURING method call frames where strong owners are sitting on the JVM stack but haven't been pushed into MyVarCleanupStack (e.g., intermediate `RuntimeScalar` values during method dispatch). @@ -2714,7 +2714,7 @@ The right fix would be one of: ScalarRefRegistry; survivors are real strong owners. 3. **Eliminate transient zeros at the source**: ensure - cooperative refCount never goes to 0 except at true scope + selective refCount never goes to 0 except at true scope exits, by making MortalList drain only AFTER the destination scalar's increment has happened. Requires re-ordering JVM- emitted bytecode for assignments (probably very invasive). @@ -2768,7 +2768,7 @@ strongly). Possible sources: 1. **DBIC's internal method dispatch left a my-var holding the row** - that wasn't released because cooperative refCount has a + that wasn't released because selective refCount has a pre-existing imbalance. 2. **Closure captures**: a closure created during DBIC processing captured the row; the closure is reachable from globals. @@ -2814,7 +2814,7 @@ must be addressed first — see D-W6.17 (next session). ### D-W6.17: Plan revision — FREETMPS-style flushing is necessary but NOT sufficient -The original D-W6.15 plan suggested making cooperative refCount precise +The original D-W6.15 plan suggested making selective refCount precise by ensuring transient zeros only happen at scope-exit boundaries (matching Perl 5's FREETMPS). This session attempted that and learned the plan needs revision. @@ -2849,7 +2849,7 @@ not WHETHER they're paired. #### Why the plan needs revision The user's plan asked for "transient zeros only at scope-exit -boundaries". This would be sufficient IF the cooperative refCount +boundaries". This would be sufficient IF the selective refCount were otherwise balanced. But it's NOT balanced — somewhere a decrement fires for which no matching increment ever happened (or vice versa). @@ -2858,7 +2858,7 @@ The transient-zero hypothesis came from observing a specific trace: refCount went 1→0 mid-statement during `Class/MOP/Class.pm:260`. But that 1→0 was the CONSEQUENCE of the imbalance, not a cause — the metaclass's "true" refCount should be 2 at end of run (matching -2 surviving owners), but PerlOnJava's cooperative count went to +2 surviving owners), but PerlOnJava's selective count went to MIN_VALUE because there were 2 extra decrement events somewhere. #### Revised plan: find the REAL unpaired site @@ -2939,7 +2939,7 @@ Master's heuristic gate: → 0 → DESTROY. Works. The heuristic is empirically correct for the modules we care about. -Its shape is: "for these specific classes, cooperative refCount may +Its shape is: "for these specific classes, selective refCount may have transient zeros; consult walker before firing DESTROY." #### Is the heuristic actually wrong? @@ -2978,7 +2978,7 @@ Then the gate becomes: && WeakRefRegistry.hasWeakRefsTo(base) && ReachabilityWalker.isReachableFromRoots(base)) { // Rescue: this object's lifetime is module-global, but - // cooperative refCount has transient zeros. Walker confirms + // selective refCount has transient zeros. Walker confirms // reachability; suppress DESTROY. } ``` diff --git a/dev/modules/ppi.md b/dev/modules/ppi.md index cb91b536e..4240b4f49 100644 --- a/dev/modules/ppi.md +++ b/dev/modules/ppi.md @@ -78,7 +78,7 @@ cross-links), which empties the Structure's hash. When the lexer next calls `_continues`, it sees `$LastChild->{start}` as undef, falls through to the `while` branch, and throws `Illegal state in 'while' compound statement`. -The root cause is **not** in PPI; it is in PerlOnJava's cooperative refcount +The root cause is **not** in PPI; it is in PerlOnJava's selective refcount bookkeeping for containers that hold DESTROY-tracked objects. **Minimal PerlOnJava-only repro (no PPI):** @@ -186,7 +186,7 @@ Investigative steps before coding a fix: 3. If object identity is lost: it's a PerlOnJava-level bug in hash/blessed-ref handling or in `Scalar::Util::weaken` / refaddr. 4. If `weaken` prematurely collects the start brace (unlikely given - `_PARENT` weak ref, but possible): that points to our `weaken` cooperative + `_PARENT` weak ref, but possible): that points to our `weaken` selective refcount interacting badly with PPI's hand-rolled weak-parent table. Fix will depend on what (1)–(4) reveal. Document the resolution back in this @@ -230,7 +230,7 @@ file before closing the phase. ### Next Steps 1. **Phase 2 is blocked** on a broader refcount-parity investigation. The root - cause is in PerlOnJava's cooperative refcount (container stores do not + cause is in PerlOnJava's selective refcount (container stores do not increment refcount for DESTROY-tracked refs), not in PPI. Needed work, in order, as its own branch: - Make container-store ops increment refcount consistently: `push`, diff --git a/dev/patches/cpan/DBIx-Class-0.082844/LeakTracer-README.md b/dev/patches/cpan/DBIx-Class-0.082844/LeakTracer-README.md index dbefd08b6..7fd8617da 100644 --- a/dev/patches/cpan/DBIx-Class-0.082844/LeakTracer-README.md +++ b/dev/patches/cpan/DBIx-Class-0.082844/LeakTracer-README.md @@ -7,7 +7,7 @@ but only for the outer test-wide registry (more than 5 entries). ## Why DBIC's leak tracer uses `weaken()` + `defined` to detect orphan objects. -PerlOnJava's cooperative refCount inflates vs native Perl's reference +PerlOnJava's selective refCount inflates vs native Perl's reference counting, so weak refs that *should* become undef at Perl-level (because the object is unreachable) remain defined. diff --git a/docs/about/changelog.md b/docs/about/changelog.md index 5c7ee5419..b49dff608 100644 --- a/docs/about/changelog.md +++ b/docs/about/changelog.md @@ -38,7 +38,7 @@ Release history of PerlOnJava. See [Roadmap](roadmap.md) for future plans. - Add `attributes` pragma with `MODIFY_*_ATTRIBUTES`/`FETCH_*_ATTRIBUTES` callbacks for subroutines and variables. - Add modules: `Filter::Simple` with `FILTER` and `FILTER_ONLY` support. -- Add `DESTROY` method support with cooperative reference counting on blessed objects, cascading destruction, closure capture tracking, and global destruction phase. +- Add `DESTROY` method support with selective reference counting on blessed objects, cascading destruction, closure capture tracking, and global destruction phase. - Add `Scalar::Util` functions: `weaken`, `isweak`, `unweaken`. - Add `Internals::SvREFCNT` for compatibility with reference-counting introspection (e.g. Sub::Quote, Moo, DBIx::Class internals). - **Bundled Moose 2.4000 and Class::MOP 2.4000**: the upstream Moose source tree is shipped in `src/main/perl/lib/{Moose,Class/MOP}/`. Tested by installing `DBIx::Class` 0.082843 via `jcpan` (DBIx::Class itself uses `Moo`, fetched from CPAN) and running its test suite — it passes 100% (314 files / 13858 asserts). Upstream Moose's own test suite passes ~99% (≥396/478 files, ≥13413/13550 asserts). See [bundled modules](../reference/bundled-modules.md#moose--classmop) and [dev/modules/moose_support.md](../../dev/modules/moose_support.md) for the full status and the small set of remaining failure clusters (numeric-arg warnings, anon-class GC timing, threads/fork tests). diff --git a/docs/about/relation-perlito.md b/docs/about/relation-perlito.md index 892b3d48e..74920f964 100644 --- a/docs/about/relation-perlito.md +++ b/docs/about/relation-perlito.md @@ -4,5 +4,5 @@ The key difference between PerlOnJava and Perlito (https://github.com/fglock/Per From an architectural standpoint, PerlOnJava is more mature. However, Perlito is currently more feature-rich due to its longer development history. PerlOnJava, however, doesn't support JavaScript like Perlito does. -Both compilers share certain limitations imposed by the JVM, such as the lack of support for XS modules and auto-closing filehandles, among others. PerlOnJava implements `DESTROY` via cooperative reference counting. +Both compilers share certain limitations imposed by the JVM, such as the lack of support for XS modules and auto-closing filehandles, among others. PerlOnJava implements `DESTROY` via selective reference counting. diff --git a/docs/about/roadmap.md b/docs/about/roadmap.md index 16cc91cdd..a3eb05432 100644 --- a/docs/about/roadmap.md +++ b/docs/about/roadmap.md @@ -76,7 +76,7 @@ Work currently in progress: ### Core Language Gaps -- ~~**`DESTROY` Support**~~ — Implemented with cooperative reference counting. Supports cascading destruction, closure capture tracking, and global destruction phase. +- ~~**`DESTROY` Support**~~ — Implemented with selective reference counting. Supports cascading destruction, closure capture tracking, and global destruction phase. - ~~**Weak References**~~ — Implemented: `Scalar::Util::weaken`/`isweak`/`unweaken` with external WeakRefRegistry. - **Taint Mode (`-T`)** — Track external data provenance using a `TAINTED` wrapper type (no extra storage for untainted scalars). Required for security-sensitive Perl applications. See `dev/design/TAINT_MODE.md`. - **Dynamically-Scoped Regex Variables** — `$1`, `$2`, etc. should be localized per regex match in the dynamic scope. diff --git a/docs/reference/architecture.md b/docs/reference/architecture.md index 98c86528d..d8bc6b113 100644 --- a/docs/reference/architecture.md +++ b/docs/reference/architecture.md @@ -178,6 +178,16 @@ Java implementations of Perl operators, organized by category. #### Perl Modules (`runtime/perlmodule/`) Java XS implementations for modules that normally use C XS code (DateTime, DBI, JSON, etc.). +#### Memory Management +PerlOnJava relies on the JVM's tracing garbage collector for general memory +reclamation, including circular references. On top of that, a small +**selective reference-counting overlay** provides Perl 5's deterministic +`DESTROY` timing and `Scalar::Util::weaken` semantics for the narrow set of +objects that need them. See [Memory Management](memory-management.md) for +the user-facing summary, the relationship to the GC literature +(Bacon 2004, Blackburn & McKinley 2003, *The Garbage Collection Handbook*), +and a comparison with Perl 5 / JVM finalization. + ## Interpreter Backend Details The interpreter provides an alternative execution path: @@ -230,9 +240,11 @@ Tests use Perl's TAP (Test Anything Protocol) format and are executed via JUnit ## Related Documentation +- [Memory Management](memory-management.md) - Selective reference-counting + overlay for `DESTROY` and `weaken`, with literature context. - `dev/architecture/` - Deep-dive architecture documents for contributors: - [Overview and index](../../dev/architecture/README.md) - - [DESTROY and weak references](../../dev/architecture/weaken-destroy.md) - Cooperative reference counting overlay + - [DESTROY and weak references](../../dev/architecture/weaken-destroy.md) - Implementation details - [Dynamic scoping](../../dev/architecture/dynamic-scope.md) - `local` and DynamicVariableManager - [Lexical pragmas](../../dev/architecture/lexical-pragmas.md) - Warnings, strict, and features - [Control flow](../../dev/architecture/control-flow.md) - die/eval, loop control, exceptions diff --git a/docs/reference/feature-matrix.md b/docs/reference/feature-matrix.md index 5122ae100..f6681cb1e 100644 --- a/docs/reference/feature-matrix.md +++ b/docs/reference/feature-matrix.md @@ -248,7 +248,7 @@ my @copy = @{$z}; # ERROR Upstream Moose tests pass ~99%; DBIx::Class (installed via `jcpan`) passes 100%. See [bundled modules](bundled-modules.md#moose--classmop). -- ✅ **`DESTROY`**: Destructor methods with cooperative reference counting. +- ✅ **`DESTROY`**: Destructor methods with selective reference counting. --- @@ -788,7 +788,7 @@ The DBI module provides seamless integration with JDBC drivers: ## Features Incompatible with JVM - ❌ **`fork` operator**: `fork` is not implemented. Calling `fork` will always fail and return `undef`. -- ✅ **`DESTROY`**: Implemented with cooperative reference counting on top of JVM GC. Supports cascading destruction, closure capture tracking, `weaken`/`isweak`/`unweaken`, global destruction phase, and `Internals::SvREFCNT` introspection. +- ✅ **`DESTROY`**: Implemented with selective reference counting on top of JVM GC. Supports cascading destruction, closure capture tracking, `weaken`/`isweak`/`unweaken`, global destruction phase, and `Internals::SvREFCNT` introspection. - ❌ **Perl `XS` code**: XS code interfacing with C is not supported on the JVM. - ❌ **Auto-close files**: File auto-close depends on handling of object destruction, may be incompatible with JVM garbage collection. All files are closed before the program ends. - ❌ **Keywords related to the control flow of the Perl program**: `dump` operator. diff --git a/docs/reference/memory-management.md b/docs/reference/memory-management.md new file mode 100644 index 000000000..bfc9ddb61 --- /dev/null +++ b/docs/reference/memory-management.md @@ -0,0 +1,135 @@ +# Memory Management in PerlOnJava + +PerlOnJava lives inside the JVM, so the underlying memory manager is always +the JVM's [tracing garbage collector][tracing-gc]. Perl 5, however, expects +two semantics that a tracing GC alone cannot deliver: + +- **`DESTROY`** — destructor methods that fire deterministically when an + object becomes unreachable, in a predictable order, before the program + observes any subsequent statement; +- **`Scalar::Util::weaken`** — weak references that are nulled out at the + moment the referent's last strong reference disappears. + +To implement these on top of a tracing GC, PerlOnJava layers a small +**selective reference-counting overlay** on top of the JVM heap. Most +allocations remain pure JVM objects with zero bookkeeping; only the small +subset of objects that actually need deterministic destruction or weak-ref +support carry refcount metadata. Everything else — cycles, large object +graphs, transient closures — is left to the JVM's regular garbage collector. + +The full implementation is documented in +[`dev/architecture/weaken-destroy.md`][weaken-destroy] (lifecycle examples, +state machine, all components, edge cases, and limitations). This page is +the user-facing summary and the literature context. + +## How the overlay works + +| Concern | Mechanism | +|---|---| +| Which objects carry refcounts | Blessed into a class with a `DESTROY` method, anonymous containers (`{}`, `[]` when birth-tracked), closures with captures, and the targets of `weaken()`. Unblessed scalars / unblessed plain data carry no refcount and are managed solely by the JVM GC. | +| Increment / decrement | Hooks at every reference-assignment site (`setLarge`), container-store (`incrementRefCountForContainerStore`), and scope exit (`scopeExitCleanup`) for tracked referents only — short-circuited for the untracked majority. | +| Statement boundary cleanup | A `MortalList` (the analogue of Perl 5's `FREETMPS` / mortal stack) defers decrements to a flush point so temporaries survive their use. | +| Weak references | Tracked in `WeakRefRegistry` (forward set + reverse identity map). When a tracked referent's count reaches zero, all weak refs to it are cleared to `undef` and `DESTROY` runs. | +| Cycle handling | The JVM tracing GC reclaims cycles as usual; refcount alone cannot. Where Perl programs use `weaken()` to break cycles for *DESTROY-timing* reasons, the overlay honors that. | +| Reachability reconciliation | A `ReachabilityWalker` walks Perl roots when refcount drifts from real reachability — most often during deferred-decrement flushes for blessed objects stored as values of package-global hashes (e.g. `our %METAS = ...`). It rescues objects that the walker proves are still reachable, preventing premature `DESTROY`. | + +This is a deliberately *narrow* refcount: only what's needed to honour +Perl's `DESTROY` and `weaken` contracts, and not the full Perl 5 +reference-counting discipline. In particular, plain scalars, plain +containers, and unblessed data carry no refcount at all — those code +paths cost the same as untracked Java objects. + +## Where this fits in the GC literature + +Combining reference counting with a tracing garbage collector is a +well-studied design. The standard reference is *The Garbage Collection +Handbook* by Jones, Hosking & Moss (CRC Press, 2nd ed. 2023): + +- [www.gchandbook.org][gc-handbook] — book home page with chapter list and + bibliography. + +Two foundational papers describe the design space: + +- **David F. Bacon, Perry Cheng, and V. T. Rajan**, *"A unified theory of + garbage collection"*, OOPSLA 2004 + ([DOI: 10.1145/1028976.1028982][bacon-2004]). Frames tracing and + reference counting as duals of the same algorithm and characterizes + hybrid collectors as points on a spectrum between them. +- **Stephen M. Blackburn and Kathryn S. McKinley**, *"Ulterior reference + counting: fast garbage collection without a long wait"*, OOPSLA 2003 + ([author PDF][urc-paper] · [DOI: 10.1145/949305.949336][urc-doi]). + Pioneered partitioning the heap so that one part is reference-counted + and another is tracing-collected, with each managing the other's + inter-partition references. + +Wikipedia provides the encyclopaedic introductions: + +- [Reference counting][wiki-rc] +- [Tracing garbage collection][wiki-tracing] +- [Garbage collection (computer science)][wiki-gc] +- [Finalizer][wiki-finalizer] — discusses the deterministic-destructor / + non-deterministic-finalizer distinction that motivates PerlOnJava's + overlay in the first place. + +### How PerlOnJava fits the spectrum + +In the Bacon/Cheng/Rajan framing, a hybrid collector partitions the heap +and applies tracing to one part and reference counting to the other. The +Blackburn/McKinley *ulterior* design picks the partition along a +generational boundary (mature vs. nursery). PerlOnJava's overlay picks +the partition along a **per-class behavioural boundary**: *"does this +class need finalization or weak-reference semantics that the JVM cannot +already provide?"* If yes, the object is added to the refcounted side at +`bless` time; if no, it stays on the pure tracing side forever. The +refcount is then strictly local — it exists only to schedule `DESTROY` +calls and to clear weak references at the right moment, while the JVM's +tracing GC remains the actual memory manager. + +This is also why we describe the overlay as **selective** rather than +*deferred* or *ulterior*: the partition criterion is neither a write-barrier +optimisation nor a generational boundary, but the runtime question of +whether deterministic finalization is required for this object's class. + +## Comparison with other Perl runtimes and the JVM + +| | Perl 5 (perl) | JVM finalization | PerlOnJava | +|---|---|---|---| +| Primary GC | Reference counting + cycle collector | Tracing | Tracing (JVM) | +| `DESTROY` timing | Deterministic, immediate at refcount 0 | Non-deterministic; `Object.finalize` is deprecated | Deterministic for tracked classes; matches Perl 5 timing for the cases users observe | +| Cycles | Leak unless broken with `weaken` | Reclaimed by tracing GC | Reclaimed by tracing GC; `weaken` still needed for `DESTROY` *timing* | +| Weak refs | `Scalar::Util::weaken` (built into refcount) | [`java.lang.ref.WeakReference`][java-ref] / `PhantomReference` / `Cleaner` | `Scalar::Util::weaken` implemented via `WeakRefRegistry` and refcount hooks | +| Cost when feature unused | Per-op refcount on every SV | Zero | Zero — `refCount == -1` short-circuit on every untracked object | + +For Perl-language semantics, see also `perlobj`'s +[Destructors][perl-destructors] section. + +## Limitations + +The full set of edge cases lives in +[`dev/architecture/weaken-destroy.md`][weaken-destroy]. The user-visible +ones are: + +- `DESTROY` is delivered for tracked objects (those blessed into a class + with `DESTROY`) at the same moment Perl 5 would deliver it; for + untracked objects (e.g. unblessed data with weak refs) it relies on JVM + GC and timing is approximate. +- `Internals::SvREFCNT($ref)` returns an approximate count rather than + the raw value Perl 5 would report — useful for `weaken`/`DESTROY` + invariants, not for byte-for-byte refcount fidelity. +- `fork` and Perl-level `threads` are not supported by the JVM backend; + the refcount overlay is single-threaded. +- `local($@, $!, $?)` around `DESTROY`: only `$@` is currently saved and + restored. + +[tracing-gc]: https://en.wikipedia.org/wiki/Tracing_garbage_collection +[weaken-destroy]: ../../dev/architecture/weaken-destroy.md +[gc-handbook]: https://www.gchandbook.org/ +[bacon-2004]: https://doi.org/10.1145/1028976.1028982 +[urc-paper]: https://users.cecs.anu.edu.au/~steveb/pubs/papers/urc-oopsla-2003.pdf +[urc-doi]: https://doi.org/10.1145/949305.949336 +[wiki-rc]: https://en.wikipedia.org/wiki/Reference_counting +[wiki-tracing]: https://en.wikipedia.org/wiki/Tracing_garbage_collection +[wiki-gc]: https://en.wikipedia.org/wiki/Garbage_collection_(computer_science) +[wiki-finalizer]: https://en.wikipedia.org/wiki/Finalizer +[java-ref]: https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/ref/package-summary.html +[perl-destructors]: https://perldoc.perl.org/perlobj#Destructors diff --git a/jperl b/jperl index 6f6c8b4ea..1bae414a7 100755 --- a/jperl +++ b/jperl @@ -33,7 +33,7 @@ JVM_OPTS="--enable-native-access=ALL-UNNAMED" # Note on JVM heap settings: do NOT set -XX:SoftMaxHeapSize below -Xmx. # That combination triggers an aggressive G1 GC cadence that interacts -# pathologically with PerlOnJava's weak-ref / cooperative-refcount +# pathologically with PerlOnJava's weak-ref / selective-refcount # machinery — DBIx-Class t/96_is_deteministic_value.t hangs at 100% CPU # under -XX:SoftMaxHeapSize=2g -Xmx4g while passing in seconds without # the soft cap (or with SoftMax >= Xmx). The JVM's auto-default diff --git a/jperl.bat b/jperl.bat index e47eaa0a7..42c5c1cbb 100755 --- a/jperl.bat +++ b/jperl.bat @@ -20,7 +20,7 @@ set JVM_OPTS=--enable-native-access=ALL-UNNAMED rem Note on JVM heap settings: do NOT set -XX:SoftMaxHeapSize below -Xmx. rem That combination triggers an aggressive G1 GC cadence that interacts -rem pathologically with PerlOnJava's weak-ref / cooperative-refcount +rem pathologically with PerlOnJava's weak-ref / selective-refcount rem machinery (DBIx-Class t/96_is_deteministic_value.t hangs at 100% CPU rem under SoftMaxHeapSize=2g + Xmx=4g). The JVM auto-default rem (MaxHeapSize = 1/4 of system RAM) is honored when nothing is set. diff --git a/src/main/java/org/perlonjava/core/Configuration.java b/src/main/java/org/perlonjava/core/Configuration.java index 5dd980f0f..566786e29 100644 --- a/src/main/java/org/perlonjava/core/Configuration.java +++ b/src/main/java/org/perlonjava/core/Configuration.java @@ -33,7 +33,7 @@ public final class Configuration { * Automatically populated by Gradle/Maven during build. * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String gitCommitId = "723dfee80"; + public static final String gitCommitId = "2c91dd8bb"; /** * Git commit date of the build (ISO format: YYYY-MM-DD). @@ -48,7 +48,7 @@ public final class Configuration { * Parsed by App::perlbrew and other tools via: perl -V | grep "Compiled at" * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String buildTimestamp = "Apr 30 2026 10:12:03"; + public static final String buildTimestamp = "Apr 30 2026 10:33:02"; // Prevent instantiation private Configuration() { diff --git a/src/main/java/org/perlonjava/runtime/operators/ReferenceOperators.java b/src/main/java/org/perlonjava/runtime/operators/ReferenceOperators.java index da637e624..54669bfb4 100644 --- a/src/main/java/org/perlonjava/runtime/operators/ReferenceOperators.java +++ b/src/main/java/org/perlonjava/runtime/operators/ReferenceOperators.java @@ -118,7 +118,7 @@ public static RuntimeScalar bless(RuntimeScalar runtimeScalar, RuntimeScalar cla // count these as strong references so scopeExitCleanupHash // correctly decrements them when the container is destroyed. // Without this, references stored before bless are invisible to - // cooperative refcounting, causing premature destruction of + // selective refcounting, causing premature destruction of // objects held only by this container (e.g., DBIC ResultSource // held by a ResultSet's {result_source} hash element). if (referent instanceof RuntimeHash hash) { diff --git a/src/main/java/org/perlonjava/runtime/runtimetypes/DestroyDispatch.java b/src/main/java/org/perlonjava/runtime/runtimetypes/DestroyDispatch.java index 7e58f716f..2708a4497 100644 --- a/src/main/java/org/perlonjava/runtime/runtimetypes/DestroyDispatch.java +++ b/src/main/java/org/perlonjava/runtime/runtimetypes/DestroyDispatch.java @@ -179,7 +179,7 @@ public static void callDestroy(RuntimeBase referent) { } // Perl 5 semantics: DESTROY CAN be called multiple times for resurrected - // objects. However, in PerlOnJava, cooperative refCount inflation means + // objects. However, in PerlOnJava, selective refCount inflation means // rescue detection fires more broadly than in Perl 5, so we keep // destroyFired=true after rescue to prevent infinite loops. // The destroyFired flag acts as a one-shot guard: once DESTROY has fired, @@ -209,7 +209,7 @@ public static void callDestroy(RuntimeBase referent) { // This allows captured variables to be properly cleaned up // (e.g., blessed objects in captured scalars can fire DESTROY). // However, skip releaseCaptures if the CODE ref is still installed in the - // stash (stashRefCount > 0). The cooperative refCount can falsely reach 0 + // stash (stashRefCount > 0). The selective refCount can falsely reach 0 // for stash-installed closures because glob assignments, closure captures, // and other JVM-level references aren't always counted. Releasing captures // prematurely would cascade to clear weak references (e.g., in Sub::Defer's diff --git a/src/main/java/org/perlonjava/runtime/runtimetypes/MortalList.java b/src/main/java/org/perlonjava/runtime/runtimetypes/MortalList.java index 6f4039a49..20526f65f 100644 --- a/src/main/java/org/perlonjava/runtime/runtimetypes/MortalList.java +++ b/src/main/java/org/perlonjava/runtime/runtimetypes/MortalList.java @@ -133,7 +133,7 @@ private static void removeFromDeferredSet(RuntimeScalar scalar) { *

* This is safe because at this point ALL lexical scopes have exited * (the main script has returned). Closures installed in stashes still - * hold JVM references to the RuntimeScalar, but the cooperative + * hold JVM references to the RuntimeScalar, but the selective * refCount should reflect that the declaring scope is gone. */ public static void flushDeferredCaptures() { @@ -154,7 +154,7 @@ public static void flushDeferredCaptures() { // Final sweep: clear weak refs for ALL remaining blessed objects. // At this point the main script has returned and all lexical scopes - // have exited. Some objects may still have inflated cooperative + // have exited. Some objects may still have inflated selective // refCounts (due to JVM temporaries, method-call copies, interpreter // captures) that prevent DESTROY from firing. Their weak refs would // remain defined forever, causing DBIC's leak tracer to report false @@ -613,7 +613,7 @@ && isReachableCached(base)) { // Replaces the class-name heuristic // (classNeedsWalkerGate). Object's lifetime is // module-global metadata (stored in a package- - // global hash like %METAS), so cooperative + // global hash like %METAS), so selective // refCount transient zeros must not fire DESTROY. // Walker confirms reachability; suppress destroy. // D-W6.16: heuristic walker gate (primary). diff --git a/src/main/java/org/perlonjava/runtime/runtimetypes/ReachabilityWalker.java b/src/main/java/org/perlonjava/runtime/runtimetypes/ReachabilityWalker.java index e54c53c0e..725907aef 100644 --- a/src/main/java/org/perlonjava/runtime/runtimetypes/ReachabilityWalker.java +++ b/src/main/java/org/perlonjava/runtime/runtimetypes/ReachabilityWalker.java @@ -12,7 +12,7 @@ * Walks the live object graph from Perl-visible roots and identifies which * objects in the weak-ref registry are unreachable. Clears weak refs for * those objects, simulating Perl 5's refcount-based collection when - * PerlOnJava's cooperative refCount has drifted due to JVM temporaries. + * PerlOnJava's selective refCount has drifted due to JVM temporaries. *

* Roots: *

    @@ -335,7 +335,7 @@ private void addReachable(RuntimeBase b, java.util.ArrayDeque todo) * without enumerating the full live set. *

    * Used by {@link MortalList#flush} to avoid prematurely firing - * DESTROY on a blessed object whose cooperative refCount dipped to + * DESTROY on a blessed object whose selective refCount dipped to * 0 transiently while the object is still held by a container the * walker can see (globals, hash/array elements registered in * {@link ScalarRefRegistry}). Concrete failure mode without this diff --git a/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeBase.java b/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeBase.java index 36adb4254..7f3e25db1 100644 --- a/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeBase.java +++ b/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeBase.java @@ -36,7 +36,7 @@ public abstract class RuntimeBase implements DynamicState, Iterable * When stashRefCount > 0, the CODE ref should NOT be considered dead even if - * the cooperative refCount reaches 0, because the stash still holds a live + * the selective refCount reaches 0, because the stash still holds a live * reference. This prevents premature {@code releaseCaptures()} which would * cascade to clear weak references (e.g., in Sub::Defer's %DEFERRED hash). */ @@ -533,7 +533,7 @@ public void releaseCaptures() { // capture, decrement refCount to balance the original increment. // // Only cascade for BLESSED referents. For unblessed containers - // (arrays, hashes), the cooperative refCount from releaseCaptures + // (arrays, hashes), the selective refCount from releaseCaptures // can falsely reach 0 (because closure captures hold JVM references // not counted in refCount). Cascading to callDestroy for such // containers would clear weak references prematurely, breaking diff --git a/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeGlob.java b/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeGlob.java index 5fadac3ff..9e590911a 100644 --- a/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeGlob.java +++ b/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeGlob.java @@ -236,7 +236,7 @@ public RuntimeScalar set(RuntimeScalar value) { // Increment stashRefCount on the new CODE ref installed in the stash. // This tracks that the stash holds a reference to this CODE object, - // which is invisible to the cooperative refCount mechanism. + // which is invisible to the selective refCount mechanism. if (value.value instanceof RuntimeCode newCode) { newCode.stashRefCount++; } diff --git a/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeHash.java b/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeHash.java index 38639bf6b..95c4286e9 100644 --- a/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeHash.java +++ b/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeHash.java @@ -43,7 +43,7 @@ public class RuntimeHash extends RuntimeBase implements RuntimeScalarReference, * the value's referent is marked as {@code storedInPackageGlobal} — * which then allows the walker-gate rescue at refCount→0 to * preserve module-global metadata (Class::MOP %METAS, similar caches) - * across transient cooperative-refCount zeros. + * across transient selective-refCount zeros. *

    * Set by {@code GlobalVariable.markAsGlobalPackageHash} when the hash * is registered into globalHashes. diff --git a/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeScalar.java b/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeScalar.java index 20b555e33..e0646e439 100644 --- a/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeScalar.java +++ b/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeScalar.java @@ -1218,7 +1218,7 @@ private RuntimeScalar setLargeRefCounted(RuntimeScalar value) { // flag set at hash-store time. // Phase D / Step W3-Path 2: mirror of the gate in // MortalList.flush(). Blessed object with outstanding - // weak refs whose cooperative refCount dipped to 0 + // weak refs whose selective refCount dipped to 0 // under an overwrite, but the walker says it's still // reachable from roots (e.g. held by `our %METAS`). // Treat as transient refCount drift; don't fire @@ -1236,7 +1236,7 @@ private RuntimeScalar setLargeRefCounted(RuntimeScalar value) { && WeakRefRegistry.weakRefsExist) { // Phase D: inside a DESTROY body, an explicit undef // assignment released our strong ref to another - // blessed-with-DESTROY object but cooperative refCount + // blessed-with-DESTROY object but selective refCount // didn't drop to 0 (cycles). Flag a deferred sweep to // run once at the end of the outermost DESTROY. // Narrow gating (only inside DESTROY, only value==UNDEF, @@ -2332,7 +2332,7 @@ public RuntimeScalar undefine() { } } else if (oldBase.blessId != 0 && oldBase.refCount > 0 && WeakRefRegistry.weakRefsExist) { - // Phase D: cooperative refCount suggests this object still has + // Phase D: selective refCount suggests this object still has // strong references, but those may all be internal cycles // (e.g. DBIC's Schema <-> source_registrations). Defer to the // reachability walker if the class has DESTROY — it's the @@ -2345,7 +2345,7 @@ public RuntimeScalar undefine() { } } else if (oldBase.blessId != 0 && WeakRefRegistry.weakRefsExist) { // Phase D: no owned-count decrement (refCountOwned was false, or - // refCount was already 0 from prior cooperative drift). The + // refCount was already 0 from prior selective drift). The // object is blessed — if its class has DESTROY, let the walker // decide whether this undef just released the last live lexical // handle. @@ -2365,7 +2365,7 @@ public RuntimeScalar undefine() { // Phase D: undef-of-blessed auto-trigger for the reachability walker. // When the user explicitly undef's a blessed ref with DESTROY but - // cooperative refCount stays > 0 (internal cycles), ask the walker + // selective refCount stays > 0 (internal cycles), ask the walker // to determine real reachability. Bypasses the MortalList auto-sweep // throttle because this is an explicit release, not an opportunistic // check. Skips when we're in module-init to avoid clearing weak refs diff --git a/src/main/java/org/perlonjava/runtime/runtimetypes/WeakRefRegistry.java b/src/main/java/org/perlonjava/runtime/runtimetypes/WeakRefRegistry.java index a2ae2d93a..ba1d43dc7 100644 --- a/src/main/java/org/perlonjava/runtime/runtimetypes/WeakRefRegistry.java +++ b/src/main/java/org/perlonjava/runtime/runtimetypes/WeakRefRegistry.java @@ -247,7 +247,7 @@ public static void clearWeakRefsTo(RuntimeBase referent) { * Clear weak refs for ALL blessed, non-CODE objects in the registry. * Called after flushDeferredCaptures() — at this point the main script * has returned and all lexical scopes have exited. Objects with inflated - * cooperative refCounts (due to JVM temporaries, method-call argument + * selective refCounts (due to JVM temporaries, method-call argument * copies, etc.) may still appear "alive" even though no Perl code holds * a reference. Clearing their weak refs allows DBIC's leak tracer * (which runs in an END block) to see them as "collected". diff --git a/src/main/perl/lib/B.pm b/src/main/perl/lib/B.pm index 9777fe1eb..dede76a0a 100644 --- a/src/main/perl/lib/B.pm +++ b/src/main/perl/lib/B.pm @@ -51,7 +51,7 @@ package B::SV { sub new { # IMPORTANT: Avoid `my ($class, $ref) = @_` or `shift` — each local # variable assignment that holds a reference inflates the referent's - # cooperative refcount by 1 (via setLargeRefCounted). Instead, use + # selective refcount by 1 (via setLargeRefCounted). Instead, use # $_[0]/$_[1] (aliases into @_) which don't increment refcount. # This keeps the only inflation to the `$self->{ref}` hash slot, # which REFCNT compensates for with its -1 adjustment. @@ -61,28 +61,28 @@ package B::SV { } sub REFCNT { - # Return the cooperative refcount via Internals::SvREFCNT. + # Return the selective refcount via Internals::SvREFCNT. # # In PerlOnJava, B::SV stores the reference in $self->{ref} which is # a tracked (blessed) hash element. This inflates the referent's - # cooperative refCount by +1 via setLargeRefCounted. + # selective refCount by +1 via setLargeRefCounted. # # In Perl 5, B::svref_2object() stores the SV pointer directly (a C # pointer), so it does NOT inflate the referent's refcnt. However, # Perl 5 has higher refcounts overall because ALL references count # (hash elements, stack temporaries, mortal slots). PerlOnJava's - # cooperative refCount is lower because: + # selective refCount is lower because: # 1. Stack/JVM temporaries don't contribute # 2. Method call argument copies don't contribute # # The B::SV inflation (+1) roughly compensates for these deficits, - # so we return the raw cooperative refCount WITHOUT subtracting 1. + # so we return the raw selective refCount WITHOUT subtracting 1. # # For Schema::DESTROY's `refcount($source) > 1` check: - # - Source with 1 cooperative ref (e.g., source_registrations only): + # - Source with 1 selective ref (e.g., source_registrations only): # B::SV inflation → 2, REFCNT = 2 → > 1 → rescue ✓ # (In Perl 5 this source also shows > 1 because stack temps add refs) - # - Source with 0 cooperative refs (untracked): + # - Source with 0 selective refs (untracked): # B::SV inflation → 1, REFCNT = 1 → no rescue ✓ Internals::SvREFCNT($_[0]->{ref}); } @@ -373,7 +373,7 @@ sub class { # Main introspection function sub svref_2object { # IMPORTANT: Do NOT do `my $ref = shift` — that creates a local variable - # holding a reference, which inflates the referent's cooperative refcount + # holding a reference, which inflates the referent's selective refcount # by 1 (via setLargeRefCounted). Use $_[0] (an alias into @_) instead, # which doesn't increment refcount. This is critical for DBIC's # refcount() function which calls B::svref_2object($_[0])->REFCNT diff --git a/src/main/perl/lib/Class/MOP/Method/Accessor.pm b/src/main/perl/lib/Class/MOP/Method/Accessor.pm index d918a4af3..81fd3d5ba 100644 --- a/src/main/perl/lib/Class/MOP/Method/Accessor.pm +++ b/src/main/perl/lib/Class/MOP/Method/Accessor.pm @@ -38,7 +38,7 @@ sub new { # we don't want this creating # a cycle in the code, if not # needed - # PerlOnJava: weaken disabled — cooperative refCount is too fragile + # PerlOnJava: weaken disabled — selective refCount is too fragile # to keep the attribute alive across the Method::Accessor's # _initialize_body call. Loses cycle detection at global destruction # but we accept the leak since our DESTROY semantics differ anyway. diff --git a/src/test/resources/unit/refcount/drift/our_metas_underflow.t b/src/test/resources/unit/refcount/drift/our_metas_underflow.t index e2f57b2b8..7d8ed3e11 100644 --- a/src/test/resources/unit/refcount/drift/our_metas_underflow.t +++ b/src/test/resources/unit/refcount/drift/our_metas_underflow.t @@ -4,9 +4,9 @@ use Test::More; use Scalar::Util qw(weaken); # ============================================================================= -# our_metas_underflow.t — Reproducer for the cooperative refCount bug: +# our_metas_underflow.t — Reproducer for the selective refCount bug: # storing a blessed reference in a package hash (`our %METAS`) does NOT -# bump the cooperative refCount, so the object's refCount underflows to 0 +# bump the selective refCount, so the object's refCount underflows to 0 # at end-of-statement when the temporary holding it expires. # # This is the root cause of the Class::MOP bootstrap failure that the