Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ Example format at the end of a design doc:

| Feature | Status |
|---------|--------|
| `weaken` / `isweak` | Implemented. Uses cooperative reference counting on top of JVM GC. See `dev/architecture/weaken-destroy.md` for details. |
| `weaken` / `isweak` | Implemented. Uses selective reference counting on top of JVM GC. See `dev/architecture/weaken-destroy.md` for details. |
| `DESTROY` | Implemented. Fires deterministically for tracked objects (blessed into a class with DESTROY). See `dev/architecture/weaken-destroy.md`. |
| `Scalar::Util::readonly` | Works for compile-time constants (`RuntimeScalarReadOnly` instances). Does not yet detect variables made readonly at runtime via `Internals::SvREADONLY` (those copy type/value into a plain `RuntimeScalar` without replacing the object). |

Expand Down
2 changes: 1 addition & 1 deletion dev/architecture/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ PerlOnJava is a Perl 5 implementation that compiles Perl source code to JVM byte
| Document | Description |
|----------|-------------|
| [dynamic-scope.md](dynamic-scope.md) | Dynamic scoping via `local` and DynamicVariableManager |
| [weaken-destroy.md](weaken-destroy.md) | Cooperative reference counting, DESTROY, and weak references |
| [weaken-destroy.md](weaken-destroy.md) | Selective reference counting, DESTROY, and weak references |
| [lexical-pragmas.md](lexical-pragmas.md) | Lexical warnings, strict, and features |
| [control-flow.md](control-flow.md) | Control flow implementation (die/eval, last/next/redo, block dispatchers) |
| [block-dispatcher-optimization.md](block-dispatcher-optimization.md) | Block-level shared dispatchers for control flow |
Expand Down
164 changes: 137 additions & 27 deletions dev/architecture/weaken-destroy.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion dev/cpan-reports/Scalar-Util.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ All 14 standard Scalar::Util EXPORT_OK functions are declared and registered.
| `blessed` | Full | Handles blessed refs and `qr//` (implicit "Regexp" blessing) |
| `refaddr` | Full | Uses `System.identityHashCode()` (JVM -- not real memory address) |
| `reftype` | Full | Handles SCALAR, REF, ARRAY, HASH, CODE, GLOB, FORMAT, REGEXP, VSTRING |
| `weaken` | Full | Cooperative reference counting on JVM GC. Well tested. |
| `weaken` | Full | Selective reference counting on JVM GC. Well tested. |
| `unweaken` | Full | Restores strong reference |
| `isweak` | Full | Delegates to `WeakRefRegistry.isweak()` |
| `dualvar` | Full | Creates `DualVar` record with separate numeric/string values |
Expand Down
10 changes: 5 additions & 5 deletions dev/design/refcount_alignment_plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ destruction semantics:
diagnostics/leak detection.

DBIC, Moose/Moo, Sub::Quote, File::Temp, Devel::StackTrace, and many cache/ORM
modules all depend on these semantics. Today PerlOnJava's **cooperative
modules all depend on these semantics. Today PerlOnJava's **selective
refcount** approximates them, but it diverges in enough places that several
real-world tests fail (DBIC t/52leaks tests 12–18, txn_scope_guard test 18,
etc.), and further real-world modules fail silently. This limits PerlOnJava's
Expand All @@ -35,7 +35,7 @@ This document lays out a phased plan to close the gap so that:

## 2. Why the Current Scheme Falls Short

PerlOnJava uses **cooperative reference counting** layered on top of JVM GC:
PerlOnJava uses **selective reference counting** layered on top of JVM GC:

- `RuntimeBase.refCount` is an `int` with state machine values:
`-1` (untracked), `0` (tracked, no counted refs), `>0` (N counted refs),
Expand Down Expand Up @@ -99,7 +99,7 @@ Specifically:

## 4. Strategy Overview

Keep cooperative refcounting as the *primary* mechanism, but add:
Keep selective refcounting as the *primary* mechanism, but add:

- **Scope-exit decrement completeness** — ensure every path that increments
has a matching path that decrements when the holder goes out of scope.
Expand Down Expand Up @@ -337,14 +337,14 @@ Each phase is independently shippable. Rollback is per-commit.
| 1 (Scope exit) | Could break closures/eval/goto by over-decrementing | Large test corpus from Phase 0; feature-flag behind `JPERL_STRICT_SCOPE_EXIT=1` during validation |
| 2 (`@_` aliasing) | XS / C-level assumptions could break | Feature-flag `JPERL_ALIASED_AT_UNDERSCORE=1`; keep old behavior as fallback for first release |
| 3 (DESTROY FSM) | Resurrection cycles if state machine has bugs | Loop detection (fail fast with RuntimeException above 1000 DESTROY calls on same object) |
| 4 (Reachability) | Cost; rarely-triggered edge cases (tied vars, weak refs into globs) | Profile extensively; amortize via periodic not per-op; keep current cooperative refcount as source of truth, reachability as fallback |
| 4 (Reachability) | Cost; rarely-triggered edge cases (tied vars, weak refs into globs) | Profile extensively; amortize via periodic not per-op; keep current selective refcount as source of truth, reachability as fallback |
| 5 (REFCNT API) | CPAN modules with specific REFCNT expectations might break | Opt-in via `JPERL_ACCURATE_REFCNT=1` for one release; default-on in next |
| 6 (CPAN validation) | Modules may need small patches for their own test bugs | Apply via `dev/patches/cpan/` if module's test is jperl-unaware |
| 7 (Interpreter) | Double the work | Share semantic helpers between backends via `runtime` classes |

## 7. What Stays the Same

- JVM GC remains the memory manager. Cooperative refCount is *metadata*,
- JVM GC remains the memory manager. Selective refCount is *metadata*,
not storage.
- `MortalList` / `DynamicState` stack discipline unchanged.
- Existing compile-time optimizations (constant folding, type propagation)
Expand Down
6 changes: 3 additions & 3 deletions dev/modules/anyevent_fixes.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ including low-priority ones.
**Note**: `./jcpan -t AnyEvent` stops at `t/02_signals.t` because that
test outputs `Bail out!` on failure, which aborts the entire harness
run after only 3 files. The signal failure is downstream of the
`weaken`/cooperative-refcount limitation documented in `AGENTS.md`
`weaken`/selective-refcount limitation documented in `AGENTS.md`
(timer/io watchers are destroyed immediately because `weaken` too
eagerly clears the last strong ref). This is being addressed in a
separate branch. Running tests individually would reveal the per-file
Expand Down Expand Up @@ -177,7 +177,7 @@ not ok 5 # weakened timer still fires
not ok 6 # twin (expected/unexpected) of 5
```

Our `weaken` is cooperative-refcount based (per AGENTS.md) and doesn't
Our `weaken` is selective-refcount based (per AGENTS.md) and doesn't
match Perl's eager-free semantics in this specific pattern:

```perl
Expand Down Expand Up @@ -232,7 +232,7 @@ Result: 82/83 → 13/83 test-program failures; subtests 24 → 157.

- `fork`: implement, or reach agreement that fork-dependent tests are
exempt from the "all tests must pass" rule?
- `weaken` semantics: cooperative-refcount is documented in `AGENTS.md`
- `weaken` semantics: selective-refcount is documented in `AGENTS.md`
— do we strengthen it for this test, or treat t/13_weaken #5–#6 as a
known deviation?

Expand Down
6 changes: 3 additions & 3 deletions dev/modules/dbix_class.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ When scope exits, scalar releases 1 reference but hash stays at refCount > 0. `c

### Next Steps

Both remaining failures (t/52leaks.t tests 12-18 and t/storage/txn_scope_guard.t test 18) hit **fundamental limitations** of PerlOnJava's cooperative refCounting that can't be solved without a major architectural change:
Both remaining failures (t/52leaks.t tests 12-18 and t/storage/txn_scope_guard.t test 18) hit **fundamental limitations** of PerlOnJava's selective refCounting that can't be solved without a major architectural change:

#### Why t/52leaks.t tests 12-18 Are Blocked

Expand All @@ -162,7 +162,7 @@ Attempted fix (Fix 10n attempt #2): Set `refCount = 0` during DESTROY body (not

**Failure mode**: `my $self = shift` inside DESTROY body increments `refCount` to 1 via `setLargeRefCounted` when `$self` is assigned. When DESTROY returns, `$self` is a Java local that goes out of scope without triggering a corresponding decrement (PerlOnJava lexicals don't hook scope-exit decrements for scalar copies). Post-DESTROY `refCount=1` → false resurrection detection → loops indefinitely on File::Temp DESTROY during DBIC test loading.

Root cause: PerlOnJava's cooperative refCount scheme can't accurately track the net delta from a DESTROY body, because lexical assignments increment but lexical destruction doesn't always decrement.
Root cause: PerlOnJava's selective refCount scheme can't accurately track the net delta from a DESTROY body, because lexical assignments increment but lexical destruction doesn't always decrement.

#### What Would Fix Both

Expand All @@ -180,7 +180,7 @@ Deferred until such architectural work becomes practical.
2. **`createReference()` audit** — Fixed: Storable, DBI. Other deserializers (JSON, XML::Parser) don't appear in the DBIC leak pattern.
3. **Targeted refcount inflation sources** — function-arg copies tracked via `originalArgsStack` (Fix 10l), @DB::args preservation works; but inflation in `map`/`grep`/`keys` temporaries remains.

### Cooperative Refcounting Internals (reference)
### Selective Refcounting Internals (reference)

**States**: `-1`=untracked; `0`=tracked, 0 counted refs; `>0`=N counted refs; `-2`=WEAKLY_TRACKED; `MIN_VALUE`=DESTROY called.

Expand Down
2 changes: 1 addition & 1 deletion dev/modules/list_moreutils.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,7 @@ All 4492 subtests must pass. Rerun `make` to ensure no unit-test regressions.
Scalar::Util::weaken($ref);
is($ref, undef, "weakened away");
```
In real perl the temporary returned by `indexes(...)` has a refcount of 1 held by `$ref`; weakening that ref drops the refcount to 0 and the temporary is freed, so `$ref` becomes undef. PerlOnJava's cooperative-refcount overlay (see `dev/architecture/weaken-destroy.md`) only tracks objects blessed into a class with `DESTROY`. For an unblessed numeric scalar like this one, weaken transitions it to `WEAKLY_TRACKED` but does not clear the weak ref at scope exit because we can't distinguish "last strong ref was this one" from "symbol table still holds a ref" without full refcounting. This is a known architectural limitation being addressed on a separate branch; this PR does not touch it.
In real perl the temporary returned by `indexes(...)` has a refcount of 1 held by `$ref`; weakening that ref drops the refcount to 0 and the temporary is freed, so `$ref` becomes undef. PerlOnJava's selective-refcount overlay (see `dev/architecture/weaken-destroy.md`) only tracks objects blessed into a class with `DESTROY`. For an unblessed numeric scalar like this one, weaken transitions it to `WEAKLY_TRACKED` but does not clear the weak ref at scope exit because we can't distinguish "last strong ref was this one" from "symbol table still holds a ref" without full refcounting. This is a known architectural limitation being addressed on a separate branch; this PR does not touch it.

### Final summary

Expand Down
Loading
Loading