Phase J: performance investigation plan + baseline tooling#518
Closed
Phase J: performance investigation plan + baseline tooling#518
Conversation
Two regressions from the DESTROY/weaken merge (PR #464): 1. BytecodeInterpreter: SCOPE_EXIT_CLEANUP_ARRAY/HASH/scalar opcodes crash with ClassCastException when the interpreter fallback path reuses registers with unexpected types. Add instanceof guards before casting. Fixes Sub::Exporter::Progressive (used by Devel::GlobalDestruction, needed by DBIx::Class). 2. GlobalDestruction: runGlobalDestruction() iterates global variable HashMaps while DESTROY callbacks can modify them, causing ConcurrentModificationException. Snapshot collections with toArray() before iterating. Fixes DBIx::Class Makefile.PL. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…weaken - Updated branch/PR references for feature/dbix-class-destroy-weaken - Added Phase 9 section documenting post-DESTROY/weaken assessment - Documented 645 ok / 183 not ok across 92 test files - Identified premature DESTROY blocker (20 tests) and GC leak blocker - Catalogued improvements from DESTROY/weaken merge (PR #464) - Updated Next Steps with new priorities (P0-P2) - Marked obsoleted items (Phase 7, old GC/DESTROY sections) Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ings Add localBindingExists flag to RuntimeBase that tracks when a named hash/array (my %hash, my @array) has had a reference created via the \ operator. This flag indicates a JVM local variable slot holds a strong reference not counted in refCount. When refCount reaches 0, the flag prevents premature callDestroy since the local variable may still be alive. The flag is cleared at scope exit (scopeExitCleanupHash/Array), allowing subsequent refCount==0 to correctly trigger callDestroy. This fixes the DBIx::Class bug where \%reg stored via accessor caused premature DESTROY of Schema objects when %reg went out of scope, even though the hash was still alive through the stored reference. The localBindingExists check is applied consistently across all refCount decrement paths: setLargeRefCounted, undefine, weaken, MortalList.flush, and MortalList.popAndFlush. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Sub::Exporter::Progressive's import() relies on caller() to determine
the target package. When Sub::Uplevel overrides CORE::GLOBAL::caller
(used by Test::Exception via Test::Builder), PerlOnJava's caller()
returns wrong frames during `use` processing, causing SEP to install
exports into the wrong package. This prevented in_global_destruction
from being exported to Devel::GlobalDestruction, which namespace::clean
then removed, causing a bareword error in DESTROY methods.
Fix by bundling a simplified Devel::GlobalDestruction that uses plain
Exporter instead of Sub::Exporter::Progressive. Since PerlOnJava always
has ${^GLOBAL_PHASE} (Perl 5.14+ feature), the implementation is a
simple one-liner.
Generated with [Devin](https://cli.devin.ai/docs)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Add the complete DBI::Const module hierarchy needed by DBIx::Class: - DBI::Const::GetInfo::ANSI - ANSI SQL/CLI constants (from DBI 1.643) - DBI::Const::GetInfo::ODBC - ODBC constants (from DBI 1.643) - DBI::Const::GetInfoType - merged name-to-number mapping - DBI::Const::GetInfoReturn - upgraded from stub to real implementation These are pure Perl constant-data modules from the DBI distribution. DBIx::Class uses them to translate info type names (e.g. SQL_DBMS_NAME) to numeric codes for $dbh->get_info(), which our JDBC-based DBI already implements with matching numeric codes. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
314-test analysis: 155 blocked by "detached result source" (weak ref cleared during clone -> _copy_state_from), ~10 GC-only, ~25 real+GC, ~6 errors. Root cause traced to Schema->connect's shift->clone->connection chain where the clone temporary's refCount drops to 0 mid-operation. Added reference to dev/architecture/weaken-destroy.md for refCount internals needed for debugging. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Corrected categorization: 27 GC-only (was ~10), only 4 real functional failures across all 40 non-detached test files. Added DESTROY trace confirming Schema::DESTROY fires during _copy_state_from in clone(). Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Prevents premature DESTROY of return values from chained method calls like shift->clone->connection(@_). During list assignment materialization (my ($self, @info) = @_), setLargeRefCounted calls MortalList.flush() which processes pending decrements from inner scope exits (e.g., clone's $self). This can drop the return value's refCount to 0 before the caller's LHS variables capture it, triggering DESTROY and clearing weak refs to still-live objects. The fix: - Adds MortalList.suppressFlush() to temporarily block flush() execution - RuntimeList.setFromList() suppresses flushing around the materialization and LHS assignment phase, then restores the previous state - Also adds reentrancy guard to flush() itself (try/finally with flushing flag) to prevent cascading DESTROY from re-entering flush() This fixes the DBIx::Class "detached result source" error where Schema->connect() returned an undefined value because the Schema clone was destroyed mid-construction. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Condensed P0 done section, corrected P1 hypothesis (callDestroy sets MIN_VALUE permanently, not a transient underflow), replaced speculative fix approaches with concrete debugging plan. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ised approach Step 11.2 (popMark + flush in setLargeRefCounted) was implemented and failed: mark-aware flush prevents DESTROY from firing for delete/untie/undef because subroutine calls push marks that hide earlier entries. 4 unit test regressions. Changes reverted. Added Step 11.3 with root cause analysis, comparison to Perl 5 FREETMPS model, and 4 possible approaches. Recommends Approach D (targeted GC leak fix) since P0 premature DESTROY is already solved by suppressFlush. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Blessed objects whose class has no DESTROY method (e.g., Moo objects like DBIx::Class::Storage::BlockRunner) were set to refCount=-1 (untracked) at bless time, so when they went out of scope their hash/array elements' refcounts were never decremented. Changes: - ReferenceOperators.bless(): always track all blessed objects regardless of whether DESTROY exists in the class hierarchy. Previously, classes without DESTROY got refCount=-1 (untracked). - DestroyDispatch.doCallDestroy(): when no DESTROY method is found, still cascade into hash/array elements via scopeExitCleanupHash/ scopeExitCleanupArray + flush() to decrement contained references. Test: dev/sandbox/destroy_weaken/destroy_no_destroy_method.t (13/13) All unit tests pass (make). Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…tails Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
- Step 11.4 fix committed and verified (all unit tests pass, 13/13 sandbox) - GC-only failures explained: Sub::Quote closure walk differences, not refcount bugs - Documented B::svref_2object->REFCNT method chain leak (separate bug) - Updated Next Steps and Open Questions with investigation results Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Complete handoff plan with 13 work items covering: - GC object liveness at END (146 files, 658 assertions) - DBI shim fixes (statement handles, transactions, numeric formatting, DBI_DRIVER, stringification, table locking, error handler) - Transaction/savepoint depth tracking - Detached ResultSource weak ref cleanup - B::svref_2object method chain refcount leak - UTF-8 byte-level string handling - Bless/overload performance Full suite baseline: 27 pass, 146 GC-only, 25 real fail, 43 skipped. 11,646 ok / 746 not-ok assertions. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Work Item 4: Added toJdbcValue() helper in DBI.java to convert
whole-number Doubles to Long before JDBC setObject(), fixing
10.0 vs 10 issue. Also handles overloaded object stringification.
Work Item 5: Fixed DBI.pm connect() to support empty driver in DSN,
$ENV{DBI_DRIVER} fallback, $ENV{DBI_DSN} fallback, proper error
messages, and require DBD::$driver for missing driver errors.
Work Item 6: Overloaded object stringification fixed by toJdbcValue().
Work Item 8: Added HandleError callback support in DBI.pm execute
wrapper, enabling DBIx::Class custom error handler.
Updated design doc with investigation findings for Work Item 2
(DBI statement handle finalization via cascading DESTROY).
Generated with [Devin](https://cli.devin.ai/docs)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Avoid fork exhaustion by limiting parallel processes. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
do { BLOCK } return values were being prematurely destroyed when the
block contained lexical variables. The scope-exit flush processed
deferred decrements from inner subroutine returns before the caller
could capture the do-block result via assignment.
This fixes 11 of 12 "Unreachable cached statement still active" failures
in DBIx::Class t/60core.t. The Cursor DESTROY now fires at the correct
time, calling finish() on cached statement handles.
Root cause: do-blocks were treated as regular bare blocks (flush=true),
but like subroutine bodies, their return value is on the JVM operand
stack and must not be destroyed before the caller captures it.
Fix: Annotate do-blocks with blockIsDoBlock and skip mortal flush at
scope exit, matching the existing blockIsSubroutine behavior. Both
JVM backend (EmitBlock) and bytecode interpreter (BytecodeCompiler)
are updated.
Generated with [Devin](https://cli.devin.ai/docs)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
When die throws inside eval{}, lexical variables between the die point
and the eval boundary go out of scope. Previously, their DESTROY methods
were never called because the SCOPE_EXIT_CLEANUP opcodes were skipped
by Java exception handling.
This fix adds scope-exit cleanup in both backends:
Bytecode interpreter:
- EVAL_TRY now records the first body register index
- The exception catch handler iterates registers from that index,
calling scopeExitCleanup for each RuntimeScalar/Hash/Array,
then flushes the mortal list to trigger DESTROY
JVM backend:
- During eval body compilation, all my-variable local indices are
recorded via emitScopeExitNullStores into evalCleanupLocals
- The catch handler emits MortalList.evalExceptionScopeCleanup()
for each recorded local, then flushes
New runtime helper: MortalList.evalExceptionScopeCleanup(Object)
dispatches to the appropriate cleanup method based on runtime type.
This is critical for DBIx::Class TxnScopeGuard, which relies on
DESTROY firing during eval exception unwinding to rollback transactions.
Generated with [Devin](https://cli.devin.ai/docs)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Implement runtime cleanup stack (MyVarCleanupStack) to ensure DESTROY fires for blessed objects in my-variables when die propagates through regular subroutines without an enclosing eval. Approach: register my-variables at creation time on a runtime stack, and unwind (running scopeExitCleanup) when exceptions propagate through RuntimeCode.apply(). Normal scope exit uses existing bytecodes and discards registrations via popMark. Key changes: - New MyVarCleanupStack class with pushMark/register/unwindTo/popMark - EmitVariable.java: emit register() after my-variable ASTORE - RuntimeCode.java: wrap 3 static apply() overloads with cleanup - BytecodeInterpreter.java: propagatingException for interpreter backend This replaces the failed emitter try/catch approach which caused try_catch.t failures due to JVM exception table ordering. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Sub bodies use flush=false in emitScopeExitNullStores to protect
return values on the JVM operand stack. This caused DESTROY to fire
outside the caller's dynamic scope -- e.g., after local $SIG{__WARN__}
unwinds, causing Test::Warn to miss warnings from DESTROY.
In void context there is no return value to protect, so we can safely
flush deferred decrements immediately after the sub returns. Added
MortalList.flush() after mortalizeForVoidDiscard in all three static
apply() overloads.
Fixes: destroy.t (0/14 -> 14/14), weaken.t (3/4 -> 4/4),
txn_scope_guard.t test 8
Generated with [Devin](https://cli.devin.ai/docs)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Three fixes for DBI handle garbage collection and resource management: 1. Use createReferenceWithTrackedElements() for Java-created DBI handles instead of createReference(). The latter incorrectly sets localBindingExists=true (designed for Perl lexical `my %hash`), which prevents DESTROY from firing in MortalList.flush(). This affected all 43+ DBIx::Class test files with GC-only failures. 2. Add Java-side DBI::finish() that closes the JDBC PreparedStatement, releasing database locks (e.g., SQLite table locks). Also add $sth->finish() call in DBI::do() for temporary statement handles. Fixes: t/storage/on_connect_do.t test 8 (table locking). 3. Break circular reference between dbh.sth (stores full sth ref) and sth.Database (stores dbh ref). Now dbh.sth stores only the raw JDBC Statement object needed for the last_insert_id() fallback. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…poraries - Add pushMark/popMark/flushAboveMark to MortalList for scoped mortal boundaries (analogous to Perl 5's SAVETMPS/FREETMPS) - Emit flushAboveMark at statement boundaries in EmitBlock to process deferred DESTROY within the current function scope only - Fix bless() to mortalize newly blessed refs (refCount=1 + deferDecrement) so method chain temporaries like Foo->new()->method() get properly destroyed at the caller's statement boundary - Fix hash/array setFromList() materialization to avoid spurious refCount increments from push() — use direct list.add() instead - Fix RuntimeArray push/pop/shift to properly track refCount for container store/remove operations - RuntimeCode.apply/callCached push/pop marks around function execution to isolate mortal scopes between caller and callee - EmitStatement scope-exit always flushes pending entries even when no my-variables exist, to handle inner sub scope exit temporaries - Remove debug tracing (JPERL_DEBUG_MORTAL env var checks) These changes fix: - Hash clear not triggering DESTROY (materialization refCount leak) - Test2::API::Context premature DESTROY breaking subtests - Method chain temporaries leaking (never reaching refCount=0) Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
grep/map/sort/all/any block closures are compiled as anonymous subs that capture lexical variables, incrementing captureCount. Unlike eval blocks, these closures were never having releaseCaptures() called after execution. This caused captureCount to stay elevated, preventing scopeExitCleanup from decrementing blessed ref refCounts — objects could never reach refCount 0 and DESTROY would never fire. The fix adds eager releaseCaptures() calls in ListOperators after each operation completes. Only closures flagged as isMapGrepBlock (temporary block closures) are affected — named subs and user closures are not touched. Sort blocks now also receive the isMapGrepBlock annotation. Also includes MortalList.flush() before END blocks (from prior session) to ensure file-scoped lexical cleanup fires before END block dispatch. Impact: DBIx::Class Storage object refCount dropped from 237 to 1. Moo-generated constructors with grep-based required attribute validation no longer leak refCounts. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
When `local @array` or `local %hash` scope exits, dynamicRestoreState() replaces the current elements with the saved ones. Previously, the current (local scope's) elements were simply discarded by JVM GC, without decrementing refCounts for any tracked blessed references they owned. This caused refCount leaks when Moo-generated writers used `local @_ = ($self, $value)` for inlined qsub triggers — each call leaked +1 on tracked objects stored in the same Moo object's hash. In DBIx::Class, this manifested as Storage refCount climbing by +1 per dbh_do() call (e.g., 108 after init_schema instead of 2). The fix calls MortalList.deferDestroyForContainerClear() on the outgoing elements before replacing them, matching the cleanup done by scopeExitCleanupArray/Hash for my-variable scope exit. Impact: Storage refCount stays at 2 after 100 dbh_do() calls. dbi_env.t failures reduced from 18 to 11 (remaining failures are DBI::db objects held by a different retention path). Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The callCached() method (used for all method dispatch via $obj->method)
was missing MyVarCleanupStack management. When a called method died,
my-variables registered inside the method's bytecode were never cleaned
up via unwindTo(), causing their refCount decrements to be lost. This
meant blessed objects held in those my-variables would leak (DESTROY
never fires).
Root cause: Regular function calls go through the static apply() which
wraps execution with MyVarCleanupStack.pushMark()/unwindTo()/popMark().
Method calls via callCached() bypassed this wrapper, calling either the
raw PerlSubroutine.apply() (cache hit) or the instance RuntimeCode.apply()
(cache miss) - neither of which manages MyVarCleanupStack.
Fix: Add MyVarCleanupStack.pushMark()/unwindTo()/popMark() to callCached()
by extracting the body into callCachedInner() and wrapping it with the
cleanup try-catch-finally.
Also fixes DBI.pm circular reference: weaken $sth->{Database} back-link
to $dbh, matching Perl 5's XS-based DBI which uses weak child→parent refs.
Together these fix all 4 remaining refCount leaks in DBIx::Class dbi_env.t
(tests 28-31), bringing it to 27/27 pass + 38 leak checks clean.
Generated with [Devin](https://cli.devin.ai/docs)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Anonymous arrays created by [...] were not birth-tracked (refCount stayed at -1/untracked), unlike anonymous hashes which properly set refCount=0 in createReferenceWithTrackedElements(). This caused blessed object references stored inside anonymous arrays to leak: the containerStore INC was never matched by a DEC when the array went out of scope, so the blessed object's refCount never reached 0 and DESTROY was never called. This was the root cause of DBIx::Class leak detection failures, where connect_info(\@info) wraps args in an arrayref. The fix adds the same birth-tracking logic to RuntimeArray that RuntimeHash already had: set refCount=0 for anonymous arrays so setLargeRefCounted can properly count references and callDestroy can cascade element cleanup when the array is no longer referenced. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
21 tests covering blessed objects inside anonymous arrayrefs and hashrefs: - Basic scope exit, function argument passing, weak ref clearing - Multiple objects, nested containers, return values - DBIx::Class connect_info pattern (object in anon arrayref arg) - Reassignment releasing previous contents These prevent regression of the birth-tracking fix in RuntimeArray.createReferenceWithTrackedElements(). Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
splice() called deferDecrementIfTracked() on removed elements without checking runtimeArray.elementsOwned. For @_ arrays (populated via setArrayOfAlias), the elements are aliases to the caller's variables, not owned copies. The alias shares the same RuntimeScalar as the caller's variable, so refCountOwned reflects the caller's ownership, not @_'s. This caused splice to decrement refCounts that @_ never incremented, triggering premature DESTROY while the object was still in scope. The fix adds the elementsOwned guard to splice's removal loop, matching the pattern already used by shift() and pop(). For @_ arrays where elementsOwned is false, the DEC is skipped. This is the exact pattern used by Class::Accessor::Grouped::get_inherited in DBIx::Class: splice @_, 0, 1, ref($_[0]) Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Add stashRefCount to RuntimeCode to track glob/stash references that are invisible to the cooperative refCount mechanism. Stash assignments (*Foo::bar = $coderef) bypass setLarge(), so the stash reference was not counted, causing false refCount==0 and premature releaseCaptures. Key changes: - RuntimeCode.stashRefCount tracks stash references (inc/dec in glob set, dynamicSaveState/dynamicRestoreState, stash delete paths) - DestroyDispatch skips releaseCaptures when stashRefCount > 0 - releaseCaptures only cascades deferDecrementIfTracked for blessed referents, not unblessed containers (whose cooperative refCount can falsely reach 0 because closure captures hold JVM references not counted in refCount) This fixes infinite recursion in DBIx::Class/Moo/Sub::Quote where premature releaseCaptures cascaded to clear weak references in Sub::Defer's %DEFERRED hash, triggering re-vivification loops. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The per-object memory footprint increased with RuntimeBase refcounting and DESTROY support, causing code_too_large.t (10K lines, ~5K tests) to OOM in the test runner. Setting maxHeapSize = '1g' for test tasks. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Full ./jcpan -t DBIx::Class run after Phase H H4 fix: - Files=314, Tests=13813 - Only 1 file fails: t/52leaks.t (11/13813 subtests) - All previously hanging/failing files now pass: 60core.t, 08pager.t, storage/error.t, zzzzzzz_perl_perf_bug.t Bonus TODO passes preserved: - generic_subq.t: 9, 11, 13, 15, 17 - txn_scope_guard.t: 13, 15, 17 Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
… quiet sweep Previously `sweepWeakRefs(quiet=true)` (auto-sweep) skipped draining `DestroyDispatch.rescuedObjects`, deferring rescue-pin cleanup to explicit `Internals::jperl_gc()` or the pre-END `clearAllBlessedWeakRefs`. That meant DBIC Schema/Source/Row objects that self-saved during DESTROY remained pinned through the main script, so DBIC's `assert_empty_weakregistry` (called mid-script, at line 526 of t/52leaks.t) saw them as still-defined weak-registry slots. Fix: drain rescuedObjects at the start of every sweep (both quiet and non-quiet). Rescued objects are BLESSED — independent of H2's skip- unblessed-containers rule. Each rescue indicates the DESTROY FSM already ran, so clearing the rescue-pin weak refs is safe. Result ------ t/52leaks.t: **11 failures → 2 failures** (20 subtests → 11 subtests, because many slot_names are now properly cleaned and removed from the registry before the assert). Remaining 2 failures: - test 9: ARRAY | basic random_results (H2 trade-off) - test 10: DBICTest::Artist=HASH (one object not yet reached) No regression: - t/60core.t: 125/125 - t/cdbi/sweet/08pager.t: 9/9 - t/storage/error.t: 49/49 (test 49 still fixed via H4) - sandbox/destroy_weaken: 213/213 - `make`: PASS Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Phase H is essentially complete: - H2 (60core.t hang): FIXED - H3 (08pager.t hang): FIXED - H4 (storage/error.t test 49): FIXED - H5 (perl_perf_bug.t): passes (was cascade of H2/H3) - H1 (52leaks.t): 11 failures -> 2 failures (99.9% reduction) Remaining 2 failures in t/52leaks.t are edge cases: - test 9: unblessed ARRAY (H2 trade-off) - test 10: single DBICTest::Artist (needs DBIC-specific analysis) Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Phase I targets the 2 residual failures from Phase H:
I1 — unblessed ARRAY leak (basic random_results): revert H2's
skip-unblessed rule and implement two-phase walker where
phase 1 seeds from globalCodeRefs with capture-walking
(stash-installed subs) and phase 2 seeds from other roots
without capture-walking (anon instance-held closures).
I2 — DBICTest::Artist leak (refcnt 2): add jperl_trace_to
diagnostic to the test, investigate the reachability path,
then apply Option A (source_registrations as weak edge),
Option B (expand pre-assert trigger), or Option C (accept
as DBIC test-harness artifact).
Also adds Phase J as a forward-looking placeholder for performance
optimization after Phase I lands — the final step before merging.
Generated with [Devin](https://cli.devin.ai/docs)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Investigated 4 approaches for closing the last 2 t/52leaks.t failures. All regressed t/60core.t (Sub::Defer dispatch chain). Key findings: - Two-phase walker alone (phase 1 globalCodeRefs-with-captures, phase 2 other-roots-without) breaks 60core.t test 109 - Removing captureCount>0 skip in Phase B1 lexical seeds + removing H2 skip-unblessed rule: fixes test 9 but still breaks 60core.t - Skip-HASH-only H2 variant: 60core.t still breaks - Even with H2 skip retained, additional reachability changes don't improve the 2 failing tests Test 9 (ARRAY) is held by DBIC's $weak_registry; H2's skip rule (needed to keep Sub::Defer's $deferred_info intact) prevents clearing. Distinguishing "reachable-only-via-closure-capture" from "reachable-via-data" needs a walker provenance-tracking refactor. Test 10 (DBICTest::Artist) clears correctly under explicit Internals::jperl_gc() (verified via /tmp/artist_leak.pl minimal repro). It's a timing issue — 5-s auto-sweep throttle doesn't fire between base_collection scope exit and the assertion. Prior short-throttle attempts were reverted for DBIC slowdowns. Recommendation: defer to post-Phase-J (performance optimization) and re-attempt if measurements reveal a cleaner path. Current 99.985% pass rate is acceptable for merging. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Root cause investigation
------------------------
Instrumented the auto-sweep walker to dump every cleared weak-ref
referent with its type and value. Found the critical pattern in
Sub::Quote/Sub::Defer:
sub quote_sub {
...
my $unquoted;
weaken($quoted_info->{unquoted} = \$unquoted); # <---
...
}
`$quoted_info->{unquoted}` is a weak scalar-ref to the LEXICAL
`$unquoted`. `$unquoted` starts as UNDEF and is later filled with
the compiled sub:
sub unquote_sub {
...
$captures{'$_UNQUOTED'} = \$unquoted;
...
eval ".. $$_UNQUOTED = sub { ... } .."; # assigns to $unquoted
}
When the auto-sweep walker marked `$unquoted` as unreachable
(walker doesn't follow captures), it was clearing the weak ref
in `$quoted_info->{unquoted}`. At next invocation, the scalar-ref
was gone and Sub::Quote's re-dispatch lost its compiled-code slot,
producing "Not a CODE reference" at line 510 of 60core.t.
The same pattern is present for Sub::Defer's `$deferred` and
`$undeferred` slots.
Fix
---
1. **Two-phase walker** in `ReachabilityWalker.walk()`:
- Phase 1: seed globalCodeRefs, BFS WITH closure-capture walking.
This ensures Sub::Defer's `$deferred_info` ARRAY and Sub::Quote's
`$quoted_info` HASH are seen as reachable via their stash-
installed accessor's closure captures.
- Phase 2: seed remaining roots (globalVariables/Arrays/Hashes,
rescuedObjects, lexical seeds), BFS without capture walking.
Anon instance-held closures stay opaque so 52leaks can still
detect Schema instances captured only by them.
2. **Skip clearing weak refs to CODE-ref-holding / UNDEF scalars**
in both `ReachabilityWalker.sweepWeakRefs` and
`WeakRefRegistry.clearAllBlessedWeakRefs`. These scalars are
typically Sub::Quote's `$unquoted` slot or Sub::Defer's
`$undeferred` slot — empty slots waiting to hold compiled subs.
Clearing weak refs to them breaks the re-dispatch chain.
3. Removed the **Phase H2 skip-unblessed-in-quiet-sweep** and
**Phase H3 skip-unblessed-in-pre-END** rules, since the new
precise skip (based on scalar content) subsumes both.
Result
------
- t/52leaks.t: 2 fails -> **1 fail** (only DBICTest::Artist remaining)
- t/60core.t: **125/125** (preserved)
- t/cdbi/sweet/08pager.t: **9/9** (preserved)
- t/storage/error.t: **49/49** (preserved)
- Sandbox: **213/213**
- `make`: **PASS**
Generated with [Devin](https://cli.devin.ai/docs)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The previous commit fixed the ARRAY leak but inadvertently re- introduced the 08pager.t END-block hang by fully removing Phase H3's skip-unblessed rule in `WeakRefRegistry.clearAllBlessedWeakRefs`. Restored that rule — but narrowed to `!(referent instanceof RuntimeScalar)` so the Phase I CODE-ref/UNDEF scalar skip still applies. Together: - H3: skip unblessed ARRAY/HASH at pre-END time (Sub::Defer $deferred_info, Sub::Quote $quoted_info). - Phase I: skip UNDEF and CODE-ref-holding scalars anywhere (Sub::Quote $unquoted / Sub::Defer $undeferred slots). - H1 rescue-drain (unchanged): drain rescuedObjects in both quiet and non-quiet sweeps. Result ------ t/52leaks.t: **11/11 PASS** (was 2/11 before Phase H, down to 1/11 after H1, now 0/11). Test 9 reports "No leaks found". All other DBIC headline tests preserved: - t/60core.t: 125/125 - t/cdbi/sweet/08pager.t: 9/9 in 6s - t/storage/error.t: 49/49 Sandbox 213/213, `make` PASS. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Thorough root-cause investigation (instrumented walker with clear-
batch dumps showing type, bless, and scalar contents) identified
the Sub::Quote `$unquoted` slot pattern as the critical chain
being broken by earlier 60core.t fix attempts:
sub quote_sub {
...
my $unquoted; # starts UNDEF
weaken($quoted_info->{unquoted} = \$unquoted);
...
# Later: unquote_sub compiles and fills the slot:
eval "\$\$_UNQUOTED = sub { ... }"
}
The LEXICAL `$unquoted` was being weak-referenced by the hash entry.
When the walker marked `$unquoted` unreachable and cleared the weak
ref, Sub::Quote lost its compiled-code slot. At the next accessor
dispatch: "Not a CODE reference at t/60core.t line 510".
Phase I fix:
1. Two-phase walker: phase 1 seeds globalCodeRefs and walks captures
(catches Sub::Defer `$deferred_info`, Sub::Quote `$quoted_info`).
2. Skip clearing weak refs to scalars that hold CODE refs OR are
UNDEF — these are the Sub::Quote/Sub::Defer slot pattern.
3. Keep H3 skip-unblessed rule at pre-END for the same protection.
Result: t/52leaks.t goes from 2 residual fails to 1 (just
DBICTest::Artist, a DBIC-internal `related_resultsets` cache issue
confirmed via jperl_trace_to diagnostic — it's held by a hash-ref
created in a Sub::Quote-generated accessor at
ResultSource.pm:19712 → Relationship/Base.pm:6743).
Generated with [Devin](https://cli.devin.ai/docs)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The refcount-alignment branch's merge criteria should cover all three major CPAN modules whose correctness depends on refcount/ DESTROY semantics: - DBIx::Class — Schema self-save, weaken back-refs, leak tracer - Moo — Sub::Defer %DEFERRED, Sub::Quote %QUOTED $unquoted slots - Template::Toolkit — context→stash DESTROY ordering, plugin back-refs Added a scope table at the top of the plan document listing each module, what it depends on, and its primary test files. Both Moo and Template::Toolkit runs are tracked as TODO items that must show 0 failures before merging. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…CountOwned) Thorough root-cause trace via Internals::jperl_trace_to showed the residual DBICTest::Artist leak comes from ScalarRefRegistry entries that are effectively dead but still held Java-alive by: - MortalList.deferredCaptures (static ArrayList) - MortalList.pending (deferred decrements) Walker seed filter updated to skip: 1. `sc.scopeExited == true` — scalar whose Perl-level scope has already ended (typical of scalars parked in MortalList.deferredCaptures awaiting end-of-main flush). 2. `!sc.refCountOwned` — scalar whose refCount-ownership was released (e.g. queued into MortalList.pending for deferred decrement). The cooperative refCount will drop on next flush. These scalars were previously seeding the walker as "live lexicals" and falsely pinning their referents. After the fix: - Minimum unaffected tests: 60core 125/125, 08pager 9/9, storage/error 49/49 - 52leaks.t: still 1 residual failure — DBICTest::Artist is held by an rcO=true scalar reachable via `createHashRef` inside a Sub::Quote-generated Relationship accessor. The reachability is a *real* DBIC-internal caching chain (related_resultsets or similar); Internals::jperl_gc() can clear it because forceGcAndSnapshot prunes the WeakHashMap entry, but auto-sweep (5s throttle) doesn't always run at the right time in the 52leaks.t body. Also enhanced findPathTo to walk RuntimeCode captures (mirrors the live walker), so diagnostic traces produce the same graph the live sweep sees. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Added an IdentityHashMap-based live-count tracker in MyVarCleanupStack alongside the existing stack ArrayList. `isLive(var)` is an O(1) check that returns true iff the var's declaration scope is still registered (the scope hasn't exited). The reachability walker uses this as the primary filter on ScalarRefRegistry seeds: a scalar is only a valid live-lexical walker root if it's still in an active my-var scope. Fallback scopeExited / refCountOwned heuristics are kept for scalars not tracked by MyVarCleanupStack (e.g. interpreter-path scalars, hash/array elements). Doesn't fix the last DBICTest::Artist leak — that scalar is a HASH ELEMENT (not a my-var), held by a Sub::Quote-generated Relationship accessor's internal cache. isLive returns false for it, fallback heuristics don't match, and walker seeds from it. This is a genuine DBIC-internal caching chain; `Internals::jperl_gc` clears it (forceGcAndSnapshot prunes the WeakHashMap entry) but auto-sweep's 5s throttle sometimes fires at the wrong time. No regressions — all other DBIC tests pass, sandbox 213/213, `make` PASS. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Final t/52leaks.t Artist analysis documented. The object is a HASH ELEMENT held by a Sub::Quote-generated Relationship accessor's internal cache (DBIC's `related_resultsets` pattern). The walker is semantically correct — it IS reachable via live closure chain. Explicit Internals::jperl_gc() clears it, but auto-sweep's 5s throttle doesn't always fire at the right time. Shorter throttles (tested 500ms, 2s) show flaky interaction with DBIC's Schema registration chains (intermediate weak-refed values get cleared too eagerly). Fixing the Artist leak requires DBIC-specific knowledge not practical to encode in the walker. Documented as known limitation; accepting 99.9928% pass rate (1/13803 subtests) as final for Phase I. Next phases (per user): ./jcpan -t Moo, ./jcpan -t Template runs, then Phase J optimization. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Added MortalList.isDeferredCapture(sc) — O(1) identity-map check paired with addDeferredCapture/processReady/flushDeferredCaptures keeping the set in sync with the list. Walker now skips scalars parked in deferredCaptures when seeding lexical roots. Also enhanced findPathTo to use forceGcAndSnapshot, matching what the live walker sees (stale WeakHashMap entries are pruned before tracing, so diagnostic paths aren't misleading). Doesn't fix the final Artist leak — the holder scalar survives 3 GC cycles (forceGcAndSnapshot), isn't in deferredCaptures, isn't in MyVarCleanupStack, and isn't scope-exited. The walker semantically correctly finds it reachable because Something in the DBIC Perl-level data structure keeps it alive. No regressions: - t/60core.t 125/125, t/08pager.t 9/9, t/storage/error.t 49/49 - Sandbox 213/213, `make` PASS Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Previously `sweepWeakRefs(quiet=true)` (auto-sweep) only cleared weak
refs, deferring DESTROY and refCount=MIN_VALUE marking to the
explicit `Internals::jperl_gc()` path. This was conservative to avoid
mid-module-init DESTROY cascades — but Phase B2a's `ModuleInitGuard`
already prevents auto-sweep from firing during require/use/BEGIN,
and Phase I's walker seed filters (MyVarCleanupStack.isLive,
MortalList.isDeferredCapture, scopeExited + refCountOwned) ensure
the walker only marks genuinely unreachable blessed objects.
Thorough diagnostic traced the final DBICTest::Artist leak to rows
held only by Sub::Quote-generated internal caches (e.g.
`$row->{related_resultsets}`). Between auto-sweeps, the weak ref in
DBIC's leak tracer stayed defined because the Artist's Java-level
reachability cycled through its own cache hash. Only firing DESTROY
on the cache-wrapping Row breaks the cycle.
Now every sweep (both auto and explicit) fires DESTROY on blessed
unreachable objects and sets refCount=MIN_VALUE on all unreachable
referents.
Result
------
t/52leaks.t: **11/11 pass** stably (5 consecutive runs, no flakiness)
t/60core.t: 125/125
t/cdbi/sweet/08pager.t: 9/9
t/storage/error.t: 49/49
Sandbox: 213/213
`make`: PASS
Generated with [Devin](https://cli.devin.ai/docs)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Files=314, Tests=13804, Result: PASS (0 failures). Full DBIx::Class test suite now passes completely under PerlOnJava, with zero DBIC source patches required. This is the first time the entire suite runs clean. Milestone: - Before Phase H: 5 files failing (2 hangs = SIGKILL), 11+ failures - After Phase H1-H4: 1 file fails, 2 subtests - After Phase I walker improvements: 1 file, 1 subtest - After aggressive quiet auto-sweep: **0 failures** The eight DBIC TODO tests (RT#82942 Data::Entangled) remain passing unexpectedly — a bonus that suggests our reachability model catches some leaks native Perl's refcount GC doesn't. Next up (per plan scope): ./jcpan -t Moo, ./jcpan -t Template. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Previously `StatementResolver.handleStatementModifierWithMy` only
special-cased `my $x = EXPR if/unless COND`. For `our` and `state`,
the fallback `COND && EXPR` was used, which fails the common Perl
idiom:
our $DEBUG = 0 unless defined $DEBUG;
Under `use strict`, `defined $DEBUG` was compiled BEFORE `our $DEBUG`
took effect, producing a spurious:
Global symbol "$DEBUG" requires explicit package name
(did you forget to declare "my $DEBUG"?)
Extended the transformation to cover `our` and `state`:
our $x = EXPR unless COND → (our $x, COND || ($x = EXPR))
state $x = EXPR if COND → (state $x, COND && ($x = EXPR))
The declaration is hoisted to a top-level list expression so it
runs before the condition, matching Perl's behavior.
This unblocks `./jcpan -t Template` — Template-Toolkit's
`Template::Service` uses this idiom to set package defaults.
Generated with [Devin](https://cli.devin.ai/docs)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
… leak.t
The fast path in RuntimeList.setFromList (for `my ($a, $b) = @_` with
all simple scalar LHS) was missing the MortalList.suppressFlush(true)
guard that the slow path below it uses.
Without the guard, `lhs.set(rhsValues[i])` → setLargeRefCounted →
MortalList.flush() fires mid-assignment. If an earlier RHS element
is a blessed return value with a pending mortal decrement (e.g.,
a temp returned from `Holler->new()`), that decrement can fire
DESTROY before the corresponding LHS variable captures the value.
Minimal repro:
sub store { my ($h, $k, $v) = @_; $h->{$k} = $v }
my %h;
store(\%h, 'a', Holler->new('x')); # DESTROY x fires mid-assign
Before: DESTROY fires during `my ($h, $k, $v) = @_` (too eager).
After: DESTROY fires at statement boundary, matching Perl.
Test impact:
- Template-Toolkit t/leak.t: 11/11 passing (was 9/11 — tests 5 & 9
failed because TT stash updates with blessed temp values saw
the temps DESTROYed mid-template).
- DBIx::Class t/52leaks.t: still passing (unchanged).
- Moo: still passing (unchanged).
- All make unit tests pass.
Generated with [Devin](https://cli.devin.ai/docs)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Root cause: RuntimeScalar.addToArray and RuntimeArray.add(RuntimeScalar)
deferred refCount tracking to createReferenceWithTrackedElements, which
runs only AFTER all array literal elements are emitted.
During array literal construction `[ Parent->new(2), Parent->new(3) ]`,
each sub call's interior assignments (e.g. `my $s = bless {}; $s->{CHILD}
= Obj->new(...)`) trigger MortalList.flush() via setLargeRefCounted.
That flush processes pending decrements from earlier-constructed
elements (Parent 2, Obj 2) that were already sitting in the array's
elements list but whose refCount was still only 1 (from bless). The
flush drives their refCount to 0 and fires DESTROY — even though the
array IS going to own them once construction finishes.
Fix: incref referents at add-time (in both RuntimeScalar.addToArray's
PLAIN_ARRAY case and RuntimeArray.add(RuntimeScalar)) so intermediate
flushes see refCount ≥ 2 and leave them alone.
incrementRefCountForContainerStore is idempotent (skips
refCountOwned=true), so the final pass in
createReferenceWithTrackedElements is a no-op for eagerly-incref'd
elements.
Also reverted the explicit MortalList.flush() I added yesterday to the
setFromList fast path's finally block — it was too aggressive (firing
on every sub entry) and with the new add-time incref, is no longer
needed for the TT leak.t repro.
Minimal repro:
package P;
sub new { my ($c,$id)=@_; my $s=bless{id=>$id},$c;
$s->{CHILD}=Obj->new($id); $s }
my $arr = [ P->new(2), P->new(3) ];
# DESTROY 2 fired mid-construction before this fix
Test impact:
- Template-Toolkit: Files=106, Tests=2920, all PASS (was failing
chomp.t / directive.t with "Can't call method clone on undef").
- Moo: Files=71, Tests=841, PASS.
- DBIx::Class t/52leaks.t: 11/11 PASS, no leaks detected.
- All make unit tests pass.
Generated with [Devin](https://cli.devin.ai/docs)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ssion Replaces the add-time incref approach from c8f669b (which was too broad and incremented refCounts in arg-passing paths — causing CDBI t/02-Film.t to lose DESTROY warnings because the blessed `$btaste3` never reached refCount=0 at block exit). New approach: wrap anon array/hash literal emission in `MortalList.suppressFlush(true/false)` bytecode calls, targeted narrowly at the literal construction itself. During this window, pending mortal decrements from earlier-evaluated elements (e.g., `Foo->new()` return values) are held until after `createReferenceWithTrackedElements` has bumped element refCounts — at which point flushing resumes and decrements are processed correctly. Verified: - DBIx::Class: Files=314, Tests=13804, PASS (cdbi/02-Film.t tests 70-71 pass; txn_scope_guard.t test 18 passes). - Moo: Files=71, Tests=841, PASS. - Template-Toolkit: Files=106, Tests=2920, PASS (both leak.t and directive.t still clean). - make unit tests pass. Reverts the broad incref in RuntimeScalar.addToArray and RuntimeArray.add(RuntimeScalar). Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Kicks off Phase J after refcount-alignment merged with DBIC 13804/TT
2920/Moo 841 all passing.
Hypothesis-driven plan: match tool to question rather than blindly
applying JFR to everything.
| Question shape | Tool |
|-----------------------------|-----------------------------------|
| Hot Java methods | JFR execution sampling |
| Alloc hot spots | JFR + async-profiler -e alloc |
| Method-level ns/op | JMH microbenchmarks |
| Refcount op rate per test | Custom Java counters + jfr events |
| Walker fan-out | AtomicLong counters |
| Perl-visible ns/op | Benchmark.pm scripts |
| Regression gates | time(1) × 20 |
New tooling committed:
- dev/design/phase_j_performance_plan.md — the plan itself, with
investigation queue J1–J7 and expected-payoff ordering.
- dev/bench/run_baseline.sh — runs every benchmark_*.pl N times,
writes dev/bench/results/<sha>.json (hand-written JSON for clean
diffs, no deps).
- dev/bench/benchmark_refcount_anon.pl — exercises the suppressFlush
bytecode we added in EmitLiteral.java for array/hash literals.
- dev/bench/benchmark_refcount_bless.pl — exercises bless / DESTROY
/ MortalList / setFromList fast-path hot loops.
- dev/bench/results/baseline-6d37287f1.json — first baseline on
current branch tip (post-refcount-alignment merge).
Out of scope for this phase: interpreter rewrite, class pooling,
codegen refactoring (all tracked as future phases).
Next step: J1 complete (baseline captured). On to J2 — measure the
per-literal overhead of the suppressFlush bytecode emission against
a pre-Phase-I baseline.
Generated with [Devin](https://cli.devin.ai/docs)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Static analysis in EmitLiteral: if no element of an anon array/hash literal can allocate a blessed temp (NumberNode / StringNode / IdentifierNode / sigil-only OperatorNode / simple BinaryOperator on simple operands), skip the MortalList.suppressFlush(true) / ... suppressFlush(wasFlushing) bytecode wrapper that Phase-I commit 6d37287 added. The wrapper is still emitted unchanged for any literal that could call a sub, a method, `bless()`, or anything else that might return a blessed temp — so semantics are preserved for the cases that actually need the guard (TT test 5/9, TT directive.t, DBIC cdbi row-index). Measured on a new benchmark_anon_simple.pl, best-of-3 ops/sec: bench before after delta ----- ------ ----- ----- arr_int 3.46 M 3.90 M +12.7% arr_str 3.79 M 3.97 M + 4.7% arr_var 2.26 M 2.26 M + 0.0% hash_simple 2.67 M 2.78 M + 4.1% nested_simple 914 K 930 K + 1.7% Regression gates: * DBIx::Class: Files=314, Tests=13804, 1125s (was 1145s) — PASS * Template-Toolkit: Files=106, Tests=2920, 135s — PASS * Moo: Files=71, Tests=841, 91s — PASS * make unit tests — PASS Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
J2 done (commit 2936484). J3 investigation complete but blocked on a walker bug that surfaces only when stash-entrySet allocation pressure is removed. Documented root cause hypothesis, findings, and next-phase candidates in phase_j_performance_plan.md. Summary of JFR findings from dbic/t/52leaks.t (60s profile, 481 samples): 60% — HashSpecialVariable.entrySet (via containsKey default impl) 10% — parser / ASM emit (one-time cost per module load) 20% — ByteCodeSourceMapper + getSourceLocationAccurate 2% — MortalList.flush + refcount ops (already optimized) Proposed next-phase items (kept in plan doc): - Walker hardening (unblocks J3 safely) - Compile caching (ByteCodeSourceMapper serialization) - Stash iterator laziness Regression gates: * DBIx::Class: Files=314, Tests=13804 — PASS (1125s, -1.7%) * Template-Toolkit: Files=106, Tests=2920 — PASS (134s) * Moo: Files=71, Tests=841 — PASS (92s) * make unit tests — PASS Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Adds MortalList.flush() at the start of ReachabilityWalker.sweepWeakRefs before forceGcAndSnapshot. Without this, scalars in MortalList.pending awaiting their deferred decrement stay JVM-alive and leak into ScalarRefRegistry.snapshot() as false-live seeds, pinning genuinely unreachable objects. This was the missing step that made the stash-entrySet allocation pressure *incidentally* make auto-sweep work — the heavy allocation forced young-gen GC which ran finalization and cleared WeakHashMap entries, but we can achieve the same effect deterministically with an explicit flush at sweepWeakRefs entry. This unblocks (but does not yet apply) the J3 stash containsKey fast path. Separately measured: flush-before-snapshot does NOT change anything observable on DBIC 52leaks.t because entrySet is still being called via the un-optimized containsKey path. The fix will pay off when J3 stash-containsKey is re-attempted with this in place. Regression gates: * DBIx::Class: Files=314, Tests=13804 — PASS (1126s) * Template-Toolkit: Files=106, Tests=2920 — PASS * Moo: Files=71, Tests=841 — PASS * make unit tests — PASS Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
saveSourceLocation is called once per parsed/emitted token. In JFR profiles of DBIC 52leaks.t it accounted for 48/481 ≈ 10% of CPU samples. The hot path's two HashMap lookups per call (fileNameToId + sourceFiles.computeIfAbsent) are redundant because 99% of successive calls are for the same compilation unit. Adds a one-slot cache `(lastFileName, lastFileId, lastInfo)` compared by identity. Identity is safe because `ctx.compilerOptions.fileName` is the same String reference throughout a single compile, so different compilation units deterministically miss the cache on entry and refresh it. bench before after delta DBIC full 1126s 1107s -1.7% Template 144s 133s -7.6% Moo 97s 90s -7.2% Cumulative Phase-J gain vs pre-J baseline (6d37287): DBIC full 1145s 1107s -3.3% Regression gates: * DBIx::Class: Files=314, Tests=13804 — PASS (1107s) * Template-Toolkit: Files=106, Tests=2920 — PASS (133s) * Moo: Files=71, Tests=841 — PASS (90s) * make unit tests — PASS Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Summary of committed optimizations: * J2 (2936484) — Skip suppressFlush for simple anon literals * J3a (b331c5d) — Flush MortalList before walker snapshot * J4 (b84faef) — Cache fileName/info in ByteCodeSourceMapper Cumulative regression gate results: * DBIx::Class: 13804/13804 PASS, 1145s → 1107s (-3.3%) * Template-Toolkit: 2920/2920 PASS, 135s → 133s (-1.5%) * Moo: 841/841 PASS, 91s → 90s (-1.1%) J3 blocked — full root-cause analysis and next-phase candidates documented in plan doc. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This was referenced Apr 21, 2026
Owner
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Kicks off Phase J — performance investigation after the refcount-alignment work merged with DBIC 13804 / TT 2920 / Moo 841 all passing.
Guiding principle: match the tool to the hypothesis rather than blindly applying JFR to everything.
-e allocBenchmark.pmscriptstime(1)× 20 (median)What's in this PR
dev/design/phase_j_performance_plan.md— the plan itself, with investigation queue J1–J7 ordered by expected payoff.dev/bench/run_baseline.sh— runs everybenchmark_*.plN times and writesdev/bench/results/<sha>.json(hand-written JSON for clean diffs, no deps).dev/bench/benchmark_refcount_anon.pl— exercises thesuppressFlushbytecode added inEmitLiteral.javafor array/hash literals.dev/bench/benchmark_refcount_bless.pl— exercisesbless/DESTROY/MortalList/setFromListfast-path hot loops.dev/bench/results/baseline-6d37287f1.json— first baseline on current tip.Investigation queue
suppressFlushbytecode overhead (the new cost from recent fixes)MortalList.flushhot-path (fires on every ref assignment)setLargeRefCountedfast paths (type-dispatch efficiency)Out of scope (future phases)
dev/design/interpreter.md).dev/design/optimization_codegen.md).Test plan
benchmark_refcount_anon.plagainst a pre-Phase-I commit.src/jmh/for the hottest method J1 surfaces../jcpan -t DBIx::Classand archive underdev/bench/results/.Generated with Devin