From 41e739cb5d601e53b3febef84eeed5ee4726e1a1 Mon Sep 17 00:00:00 2001 From: "Flavio S. Glock" Date: Thu, 23 Apr 2026 14:30:57 +0200 Subject: [PATCH 1/6] docs(dbi): fresh jcpan -t DBI baseline + revised phase 10+ plan Full suite ran cleanly in 192s (no Gofer STORE/set_err infinite loop this time), so we now have real numbers: 5566/5944 passing (94%), 76/200 files failing, 378 failing subtests. Key finding: the per-file failure distribution is extremely skewed. t/10examp.t alone accounts for ~25% of all failures because the file dies at test 50 with "Undefined subroutine &main::test_dir" after `do "./t/lib.pl"` -- a PerlOnJava interpreter bug, not a DBI issue. Isolated repros work. Revised plan: - Drop the "Full Phase 10" big-scope plan (Profile + Kids + Executed + swap_inner_handle + take_imp_data, then flip $DBI::PurePerl = 0). Keeping the flag true means all current skip paths stay intact, so that plan would only land net-zero until every XS-only feature is reimplemented. Deferred indefinitely. - Phase 10 (new scope): debug the t/10examp.t `do "./t/lib.pl"` symbol-table issue. Up to ~965 subtests (193 x 5 wrappers). - Phase 11: file locking for DBM tests (~95 subtests). - Phase 12: execute_array (~25 subtests). - Phase 13: small triage fix-ups (~75 subtests). No code changes in this commit -- plan update only. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- dev/modules/dbi_test_parity.md | 248 ++++++++++-------- .../org/perlonjava/core/Configuration.java | 4 +- 2 files changed, 142 insertions(+), 110 deletions(-) diff --git a/dev/modules/dbi_test_parity.md b/dev/modules/dbi_test_parity.md index e70428a64..bf55d9a46 100644 --- a/dev/modules/dbi_test_parity.md +++ b/dev/modules/dbi_test_parity.md @@ -5,6 +5,20 @@ DBI test suite, 200 test files) pass on PerlOnJava. ## Current Baseline +After Phase 9 + 9b (upstream DBI 1.647 + DBI::PurePerl, JDBC path via +DBD::JDBC base driver). Fresh full-suite `jcpan -t DBI` run +(2026-04-23, master HEAD `720a04db3`): + +| | Files | Subtests | Passing | Failing | +|---|---|---|---|---| +| `jcpan -t DBI` | 200 | **5944** | **5566 (94 %)** | 378 | + +See "Fresh baseline (2026-04-23)" section below for per-file +failure distribution and the revised phase 10+ plan (skipped +tests stay skipped — no `$DBI::PurePerl` flag flip). + +--- + After Phase 7 (trace/TraceLevel semantics, DBI->internal tied-handle, `_concat_hash_sorted` rewrite, dbh default attributes, unknown-attr warnings): @@ -778,114 +792,132 @@ upcoming Phase 10 will reimplement in Java. `my $x = { ternary-returning-list }`; the guard is the minimal-risk fix, the proper emitter fix is tracked separately). Now 99/99. -2. **Full `jcpan -t DBI` baseline not yet re-run.** Per-test numbers - extrapolate to ~5800–6300 passing subtests (from the 4940/6570 - Phase 7 baseline), but a full run would confirm. - -### Phase 10 (planned): reimplement XS-only features in Java - -Upstream DBI::PurePerl explicitly skips some XS features with -warnings like `"$h->{Profile} attribute not supported for DBI::PurePerl"`. -These are the roadmap for the next round of Java work: - -- **Profile dispatch hook** — single biggest block (91 tests in - t/40..43_prof_*.t). Upstream XS wraps every dispatched method in - a timing frame that bumps `$h->{Profile}{Data}{$path...}`. We'd - hook `DBI::dispatch` (via method wrapping in the Java shim) to - do the same. -- **Callbacks** — 65-test block (t/70callbacks.t). Fire - `$h->{Callbacks}{$method}` (or `*`) before/around dispatch. -- **Kids/ActiveKids/CachedKids** auto-bookkeeping on parent handles. -- **swap_inner_handle**, **take_imp_data** round-trip. -- **XS-level trace formatter** (per-handle trace fh + PerlIO layers). - -### Next Steps - -Remaining high-signal individual-test failures (running -`./jperl ~/.cpan/build/DBI-1.647-5/t/X.t` directly; failing-count -before the test process halts): - -| Test file | Pass/Total | Area | -|---|---|---| -| `t/03handle.t` | 94/137 (43 fail) | `ActiveKids`, `CachedKids`, `swap_inner_handle`, Kids bookkeeping after DESTROY | -| `t/06attrs.t` | 142/166 (24 fail) | driver-private attr semantics (`delete` on `examplep_*`), `Statement` attr on failed `do`, `ErrCount` bump-on-error | -| `t/08keeperr.t` | 84/91 (7 fail) | `set_err` + `RaiseError` stack-trace in `$@`; `$DBI::err` undef after disconnect | -| `t/14utf8.t` | 10/16 (6 fail) | `NAME_lc`/`NAME_uc` hash derivation for ExampleP's computed column list | -| `t/15array.t` | 16/55 (39 fail) | `execute_array` / `bind_param_array` — needs DBD bulk-execute path | -| `t/16destroy.t` | 17/20 (2 fail, 1 SKIP) | `Active` read inside a user-defined `DESTROY` (stray pre-connect DESTROY is firing with Active=0) | -| `t/19fhtrace.t` | 20/27 (7 fail) | `trace($level, "STDERR")` string-target, PerlIO layer preservation on the installed trace fh | -| `t/30subclass.t` | 19/43 (24 fail) | `RootClass` connect attribute: rebless outer handles into the subclass hierarchy | -| `t/40profile.t` | 3/60 (17 fail, then halts) | `DBI::Profile` data capture — needs method-dispatch hook | -| `t/41prof_dump.t` | 7/9 (2 fail, halts) | `DBI::ProfileDumper::flush_to_disk` writes to disk + round-trip | -| `t/42prof_data.t` | 3/4 (1 fail, halts) | depends on ProfileDumper output | -| `t/43prof_env.t` | 0/11 | `DBI_PROFILE` env-var instrumentation | -| `t/70callbacks.t` | 65/81 (16 fail) | fatal-callback die propagation; reblessing of `$_[0]` in callbacks | - -1. **Profile capture** (40/41/42/43). This is the biggest - remaining block — 91 failing tests concentrated in 4 files. - Real DBI's XS hooks `DBD::_::common::AUTOLOAD` (among other - things) to bump the Profile tree on every method call. Options: - - Add a dispatch-time hook in - `DBI::_::OuterHandle::AUTOLOAD` that, when - `$h->{Profile}` is set, walks the Profile Path, builds the - node, and increments timings around the call. - - Inherit `Profile` to sth at prepare time (we already do - this) and bump child counts the same way. - - `DBI::ProfileDumper::flush_to_disk` needs to actually see - data in `{Data}` before it can write anything — the above - hook is the prerequisite. - -2. **`RootClass`** (`t/30subclass.t`). When `connect($dsn, u, p, - { RootClass => 'MyDBI' })` is used, real DBI reblesses the - outer handles into `${RootClass}::db` / `::st` / `::dr` so - user subclasses get method dispatch. Currently we ignore - `RootClass`. Fix: in `DBI.pm`'s `connect` wrapper, if - `RootClass` is set, `require` it and rebless the returned - outer handles. _new_sth / _new_drh should honour the same. - -3. **`t/03handle.t` Kids / ActiveKids / CachedKids**. After - `$sth->finish` / `$dbh->disconnect` / `undef $dbh`, the - counters on the parent handle aren't updated. Needs - systematic bump/decrement in `execute`, `finish`, - `disconnect`, and the DBD destructor. - -4. **`t/15array.t` `execute_array`**. Currently the - `execute_array` in our DBI.pm is a thin loop over - `execute(@row)` but many subtests depend on fine-grained - error handling (tuple_status), `ArrayTupleFetch` coderef - sources, and RaiseError propagation across rows. This is a - self-contained chunk. - -5. **`t/06attrs.t` driver-private `delete` semantics**. - `delete $dbh->{examplep_private_dbh_attrib}` should return - 42 but leave the value in place (the driver re-computes it - on each FETCH). This requires a DELETE override in - `DBI::_::Tie` that consults the implementor class before - actually removing the key. - -6. **`t/16destroy.t`**. Two subtests fail because a stray dbh - DESTROY fires with Active=0 between `install_driver` and - the user's `$drh->connect`. Need to trace where that extra - handle comes from (likely a temporary dbh built during - install_driver / setup_driver that we don't InactiveDestroy). - -7. **`t/19fhtrace.t` PerlIO layers**. `trace(undef, $fh)` with a - `$fh` that has custom layers (e.g. `:utf8`) must preserve - them when DBI writes. Also `trace(0, "STDERR")` should parse - the string "STDERR" as an alias for `*STDERR`. - -8. **`t/08keeperr.t` `$DBI::err` cleanup on disconnect**. - After `$dbh->disconnect`, `$DBI::err` should revert to - undef. Currently it keeps the last value. - -9. **Full-suite `jcpan -t DBI` run.** The last attempt at - a fresh baseline got stuck in what looks like an infinite - loop inside Gofer's STORE / set_err chain. To be - investigated on a separate branch (the hot-loop symptom was - `DBD::_::common::set_err` → `DBD::Gofer::db::STORE` → - `_Handles.pm:816`). Once that's resolved the next baseline - number should reflect Phase 7's gains (est. ~+100 passes - from the per-test deltas). + +--- + +## Fresh baseline (2026-04-23): full `jcpan -t DBI` + +Re-ran the complete DBI test suite after Phase 9/9b landed +(master HEAD `720a04db3`). The infinite-loop symptom in the +Gofer `STORE`/`set_err` chain did **not** reproduce this time — +the suite completed cleanly in 192s. + +| | Files | Subtests | Passing | Failing | +|---|---|---|---|---| +| `jcpan -t DBI` (2026-04-23) | 200 | **5944** | **5566** | **378** | +| Failed files | | | 76/200 | | + +Compared with the Phase 7 baseline (4940/6570 passing), subtest +count is lower because the upstream `DBI::PurePerl` switch (Phase 9) +caused many tests to hit early skip paths that the old home-grown +`_Handles.pm` did not honour (`$DBI::PurePerl` guards for `Kids`, +`swap_inner_handle`, `Executed`, Profile, Callbacks, ...). The +raw pass rate went from 75 % → **94 %**. + +### Failure distribution by base test file + +Each base test is run through 5 wrappers (`t/`, `zvg_*`, `zvn_*`, +`zvp_*`, `zvxg{n,p}*`, `zvxgp_*`). The counts below are per base +file — multiply by wrapper count for raw subtest impact. + +| Base test | Per-variant fail | ~Variants failing | Rough total | Area | +|---|---|---|---|---| +| `t/10examp.t` | 193/242 | 5 | **~965** | **Crash at test 50**: `test_dir()` undefined after `do "./t/lib.pl"`. Blocks 80 % of the file across all variants — **single biggest lever**. | +| `t/50dbm_simple.t` | 16/38 | 5 | ~80 | `flock`/`fcntl` locking — "Resource deadlock avoided" on `.lck` files in `DBI::DBD::SqlEngine`. PerlOnJava file-locking semantics gap. | +| `t/85gofer.t` | 9/20 | 3 | ~27 | Gofer transport error-path handling (`set_err` propagation over serialised calls). | +| `t/49dbd_file.t` | 9/65 | 3 | ~27 | `DBD::File` table directory traversal / column naming edge cases. | +| `t/04mods.t` | 7/12 | 5 | ~35 | Missing bundled modules / CPAN deps not installed in ports tree. | +| `t/72childhandles.t`| 6/16 | 5 | ~30 | `ChildHandles` weakref list — depends on Kids bookkeeping (PurePerl skip stays). | +| `t/15array.t` | 5/55 | 5 | ~25 | `execute_array` / `bind_param_array` — sth-level bulk-execute gaps. | +| `t/51dbm_file.t` | 2–5/7–10| 5 | ~15 | same locking family as 50dbm_simple. | +| `t/19fhtrace.t` | 4/27 | 5 | ~20 | `trace($l, "STDERR")` string alias, PerlIO layers on installed trace fh. | +| `t/16destroy.t` | 3/20 | 5 | ~15 | Stray pre-connect DESTROY with `Active=0`. | +| `t/03handle.t` | 3/137 | 5 | ~15 | Residual handle edge cases (most Kids tests now stay skipped). | +| `t/08keeperr.t` | 2–3/91 | 5 | ~13 | `$DBI::err` cleanup on disconnect; `RaiseError` $@ stack trace. | +| `t/02dbidrv.t` | 2/54 | 5 | ~10 | | +| `t/06attrs.t` | 2/166 | 5 | ~10 | `ErrCount` bump-on-error, `Statement` attr on failed `do`. | +| `t/73cachedkids.t` | 2/11 | 5 | ~10 | `CachedKids` weakref semantics. | +| `t/14utf8.t` | 1/16 | 5 | ~5 | `NAME_lc`/`NAME_uc` derivation for ExampleP. | +| `t/53sqlengine_adv.t` | setup fail | 3 | 0 | Test file aborts before any assertions — needs triage. | + +`t/10examp.t` alone accounts for an estimated **~25 % of all +remaining failures**. Fixing the single `test_dir()` crash unlocks +190+ more assertions per wrapper. + +### Revised priority order (skipped tests stay skipped) + +All `$DBI::PurePerl`-gated `skip` / `skip_all` paths are left in +place — we do **not** flip `$DBI::PurePerl = 0`. This means the +old "Phase 10 big scope" (Profile/Kids/Executed/swap_inner_handle +reimplementation in Java to flip the flag) is **deferred +indefinitely**. Focus is on failures that aren't flag-gated. + +#### Phase 10 (new scope): unblock t/10examp.t + +The test does `do "./t/lib.pl"` to pull in `test_dir()`. Under +PerlOnJava the `do` at line 172 returns without an error but +`main::test_dir` ends up undefined when called at line 174 +("Undefined subroutine &main::test_dir"). Isolated repros work +fine (`jperl -e 'do "./t/lib.pl"; test_dir()'` succeeds), so +there's something about the surrounding script state that +disrupts the symbol-table installation. + +**Hypotheses to investigate:** +- A preceding `use DBI qw(:sql_types); use Config; use strict;` + plus `require File::Basename/Spec/VMS::Filespec` somehow + changes package-symbol-table behaviour around the `do`. +- `strict` + the presence of a `package Test::Secret { ... }` + block earlier in the file may be corrupting `%main::`. +- `do FILE` semantics when the called file contains `my $test_dir` + + `END { ... }` + `sub test_dir` at file scope may install the + sub in the wrong package or clobber it post-`do`. + +**Effort:** small/unknown — this is a PerlOnJava interpreter bug, +not a DBI bug. Likely 1–2 days. + +**Impact:** up to ~965 subtests (193 × 5 wrappers). + +#### Phase 11: filesystem locking for DBM tests + +`t/50dbm_simple.t`, `t/51dbm_file.t`, and the `49dbd_file.t` +family die with "Resource deadlock avoided" from +`DBI::DBD::SqlEngine` at the `flock`/`fcntl` call. Needs: +- Verify our `flock`/`fcntl` behave on the same fd/inode pair + the way CPython/Perl does on macOS (FD-based vs inode-based). +- Either match Perl's behaviour or skip these test families on + PerlOnJava. + +**Impact:** ~95 subtests (combined DBM + DBD::File variants). + +#### Phase 12: execute_array (t/15array.t) + +Already scoped in previous Next-Steps section. Still ~25 subtests. + +#### Phase 13: small triage (pure fix-ups) + +- `t/08keeperr.t`: `$DBI::err` cleanup on disconnect (~13 subtests). +- `t/06attrs.t`: `ErrCount` bump-on-error, `Statement` on failed + `do` (~10). +- `t/16destroy.t`: stray pre-connect DESTROY (~15). +- `t/19fhtrace.t`: `trace(undef, $fh)` PerlIO layer preservation, + `trace(0, "STDERR")` string alias (~20). +- `t/14utf8.t`: ExampleP `NAME_lc`/`NAME_uc` hash derivation (~5). +- `t/02dbidrv.t`: 2 subtests (~10 across variants). + +**Combined impact:** ~75 subtests. + +#### Deferred / out of scope + +- **Profile / Callbacks / Kids / swap_inner_handle / Executed** + reimplementation in Java. Would only help if we flipped + `$DBI::PurePerl = 0`, which in turn would require all five to + work first, and would expose tests that currently stay in the + safe skip paths. Not a win until someone asks for it. +- **Gofer** (`t/85gofer.t` et al.) — deferred unless a consumer + needs it; fix scope is non-trivial. +- **`t/80proxy.t`** — needs `RPC::PlClient`; already skipped. +- **`zvp_*`** (PurePerl-on-PurePerl) variants — redundant once + the base tests pass; no extra effort required. ### Open Questions diff --git a/src/main/java/org/perlonjava/core/Configuration.java b/src/main/java/org/perlonjava/core/Configuration.java index b626e3938..20185dc05 100644 --- a/src/main/java/org/perlonjava/core/Configuration.java +++ b/src/main/java/org/perlonjava/core/Configuration.java @@ -33,7 +33,7 @@ public final class Configuration { * Automatically populated by Gradle/Maven during build. * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String gitCommitId = "1cdf0926f"; + public static final String gitCommitId = "720a04db3"; /** * Git commit date of the build (ISO format: YYYY-MM-DD). @@ -48,7 +48,7 @@ public final class Configuration { * Parsed by App::perlbrew and other tools via: perl -V | grep "Compiled at" * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String buildTimestamp = "Apr 23 2026 13:55:28"; + public static final String buildTimestamp = "Apr 23 2026 14:30:03"; // Prevent instantiation private Configuration() { From 21c612dbc27c2570ca0373d490611b7b7411b0b3 Mon Sep 17 00:00:00 2001 From: "Flavio S. Glock" Date: Thu, 23 Apr 2026 15:48:36 +0200 Subject: [PATCH 2/6] fix(runtime): `package Foo;` must be lexically scoped at runtime Perl 5 lexically scopes `package Foo;` to the enclosing block / sub / eval / file, not just to the file. PerlOnJava's ScopedSymbolTable already scoped it correctly at COMPILE time via packageStack, but both backends were emitting an unscoped runtime update of InterpreterState.currentPackage: - JVM backend: InterpreterState.setCurrentPackageStatic(name) - Interpreter: SET_PACKAGE opcode Only the block form `package Foo { BLOCK }` was correctly scoped (via PUSH_PACKAGE + DynamicVariableManager). Bare `package Foo;` leaked into the caller's scope for the entire rest of the process. The leak was latent until we switched to upstream DBI 1.647, which pulls in Carp::caller_info. Carp does: { package DB; @call_info{...} = caller($i); } as a standard debugger-compatibility shim. After any Test::More call fired, the runtime current-package tracker was stuck at "DB". Later `do "./t/lib.pl"` in t/10examp.t then compiled the loaded file in "DB", installing `sub test_dir` as `DB::test_dir` instead of `main::test_dir`. Line 174's `test_dir()` call failed with "Undefined subroutine &main::test_dir". Fix: - New InterpreterState.setCurrentPackageLocal(name): push the current package scalar onto DynamicVariableManager, then set the new value. Scope teardown (localTeardown / POP_LOCAL_LEVEL) auto-restores. - EmitOperator.handlePackageOperator (JVM): emit the scoped variant. - CompileOperator (interpreter): always emit PUSH_PACKAGE; the `isScoped` annotation is redundant since all `package X;` are scoped in Perl 5. Impact on `jcpan -t DBI` full suite: - Before: 5566/5944 passing (94%), 378 fail - After: 6210/6600 passing (94%), 390 fail - Delta: +644 more subtests pass (+656 executed overall) Per-file: `t/10examp.t` and its 4 wrappers each went from ~49/242 executed subtests to ~200+/242. `make` passes; no unit-test regressions. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- dev/modules/dbi_test_parity.md | 106 ++++++++++++------ .../backend/bytecode/CompileOperator.java | 7 +- .../backend/bytecode/InterpreterState.java | 17 +++ .../perlonjava/backend/jvm/EmitOperator.java | 11 +- .../org/perlonjava/core/Configuration.java | 4 +- 5 files changed, 106 insertions(+), 39 deletions(-) diff --git a/dev/modules/dbi_test_parity.md b/dev/modules/dbi_test_parity.md index bf55d9a46..b21416baf 100644 --- a/dev/modules/dbi_test_parity.md +++ b/dev/modules/dbi_test_parity.md @@ -5,17 +5,14 @@ DBI test suite, 200 test files) pass on PerlOnJava. ## Current Baseline -After Phase 9 + 9b (upstream DBI 1.647 + DBI::PurePerl, JDBC path via -DBD::JDBC base driver). Fresh full-suite `jcpan -t DBI` run -(2026-04-23, master HEAD `720a04db3`): +After Phase 10 (2026-04-23): PerlOnJava bug fix — `package Foo;` +now lexically scoped at runtime (not just at compile time). See +Phase 10 section below. | | Files | Subtests | Passing | Failing | |---|---|---|---|---| -| `jcpan -t DBI` | 200 | **5944** | **5566 (94 %)** | 378 | - -See "Fresh baseline (2026-04-23)" section below for per-file -failure distribution and the revised phase 10+ plan (skipped -tests stay skipped — no `$DBI::PurePerl` flag flip). +| `jcpan -t DBI` (2026-04-23, post-Phase-10) | 200 | **6600** | **6210 (94 %)** | 390 | +| (pre-Phase-10 baseline) | 200 | 5944 | 5566 (94 %) | 378 | --- @@ -854,30 +851,75 @@ indefinitely**. Focus is on failures that aren't flag-gated. #### Phase 10 (new scope): unblock t/10examp.t -The test does `do "./t/lib.pl"` to pull in `test_dir()`. Under -PerlOnJava the `do` at line 172 returns without an error but -`main::test_dir` ends up undefined when called at line 174 -("Undefined subroutine &main::test_dir"). Isolated repros work -fine (`jperl -e 'do "./t/lib.pl"; test_dir()'` succeeds), so -there's something about the surrounding script state that -disrupts the symbol-table installation. - -**Hypotheses to investigate:** -- A preceding `use DBI qw(:sql_types); use Config; use strict;` - plus `require File::Basename/Spec/VMS::Filespec` somehow - changes package-symbol-table behaviour around the `do`. -- `strict` + the presence of a `package Test::Secret { ... }` - block earlier in the file may be corrupting `%main::`. -- `do FILE` semantics when the called file contains `my $test_dir` - + `END { ... }` + `sub test_dir` at file scope may install the - sub in the wrong package or clobber it post-`do`. - -**Effort:** small/unknown — this is a PerlOnJava interpreter bug, -not a DBI bug. Likely 1–2 days. - -**Impact:** up to ~965 subtests (193 × 5 wrappers). - -#### Phase 11: filesystem locking for DBM tests +**Status: done (2026-04-23).** Root-caused to a PerlOnJava package- +scoping bug (not a DBI bug). Fixed in this branch. + +### Root cause + +`Carp::caller_info` contains: +```perl +{ + package DB; + @call_info{...} = caller($i); +} +``` + +In Perl 5, `package DB;` inside a bare block is lexically scoped — +it only affects that block, and the outer package is restored on +block exit. PerlOnJava's JVM backend was emitting +`InterpreterState.setCurrentPackageStatic("DB")` for `package X;` +statements without any scope-exit restore. Only the block form +`package X { ... }` was correctly scoped via `PUSH_PACKAGE`. + +Consequence: once Test::More called `Carp::caller_info` (which it +does during early setup), the runtime current-package tracker was +left as `"DB"`. The next `do "./t/lib.pl"` then compiled the loaded +file in package `DB`, so `sub test_dir` ended up as +`DB::test_dir` — invisible to `main::test_dir` calls. + +### Fix + +- New `InterpreterState.setCurrentPackageLocal(String)` helper: + pushes the current package scalar onto `DynamicVariableManager` + and sets the new value. Restored automatically when the + enclosing block / sub / file exits (via the existing + `localTeardown` / `POP_LOCAL_LEVEL` machinery). +- `EmitOperator.handlePackageOperator` (JVM backend) now emits + `setCurrentPackageLocal` instead of `setCurrentPackageStatic`. +- `CompileOperator` (interpreter backend) always emits + `PUSH_PACKAGE` — the `isScoped` annotation is no longer needed + to distinguish scoped vs unscoped since all `package X;` + declarations in Perl 5 are lexically scoped. + +Files: `InterpreterState.java`, `EmitOperator.java`, +`CompileOperator.java`. + +### Impact + +Fresh `jcpan -t DBI` after the fix: + +| | Files | Subtests | Passing | Failing | +|---|---|---|---|---| +| Before Phase 10 | 200 | 5944 | 5566 (94 %) | 378 | +| **After Phase 10** | 200 | **6600** | **6210 (94 %)** | 390 | +| Delta | 0 | +656 | **+644** | +12 | + +Per-file deltas for the `t/10examp.t` family (executed subtests +went from ~49 to ~200+ per wrapper): + +| Variant | Before | After (approx) | +|---|---|---| +| `t/10examp.t` | 49/242 executed | 200+/242 | +| `t/zvg_10examp.t` | 48/242 executed | 200+/242 | +| `t/zvp_10examp.t` | 49/242 executed | 200+/242 | +| `t/zvxgp_10examp.t` | 48/242 executed | 200+/242 | + +The fix also eliminates a whole class of latent bugs in any CPAN +module that uses `{ package X; ... }` — Carp itself being a +prominent example, but the pattern is common for debugger- +compatibility shims. + +### Phase 11: filesystem locking for DBM tests `t/50dbm_simple.t`, `t/51dbm_file.t`, and the `49dbd_file.t` family die with "Resource deadlock avoided" from diff --git a/src/main/java/org/perlonjava/backend/bytecode/CompileOperator.java b/src/main/java/org/perlonjava/backend/bytecode/CompileOperator.java index d208a6076..05156ad27 100644 --- a/src/main/java/org/perlonjava/backend/bytecode/CompileOperator.java +++ b/src/main/java/org/perlonjava/backend/bytecode/CompileOperator.java @@ -828,9 +828,12 @@ public static void visitOperator(BytecodeCompiler bytecodeCompiler, OperatorNode } bytecodeCompiler.symbolTable.setCurrentPackage(packageName, isClass); if (isClass) ClassRegistry.registerClass(packageName); - boolean isScoped = Boolean.TRUE.equals(node.getAnnotation("isScoped")); + // Always emit PUSH_PACKAGE so the runtime tracker is restored when + // the enclosing block/sub/file exits. Perl 5's `package Foo;` is + // lexically scoped; the `isScoped` annotation used to distinguish + // `package Foo { BLOCK }` but bare `package Foo;` is equally scoped. int nameIdx = bytecodeCompiler.addToStringPool(packageName); - bytecodeCompiler.emit(isScoped ? Opcodes.PUSH_PACKAGE : Opcodes.SET_PACKAGE); + bytecodeCompiler.emit(Opcodes.PUSH_PACKAGE); bytecodeCompiler.emit(nameIdx); bytecodeCompiler.lastResultReg = -1; } else { diff --git a/src/main/java/org/perlonjava/backend/bytecode/InterpreterState.java b/src/main/java/org/perlonjava/backend/bytecode/InterpreterState.java index a4326f8ee..4b5192026 100644 --- a/src/main/java/org/perlonjava/backend/bytecode/InterpreterState.java +++ b/src/main/java/org/perlonjava/backend/bytecode/InterpreterState.java @@ -54,6 +54,23 @@ public class InterpreterState { public static void setCurrentPackageStatic(String name) { currentPackage.get().set(name); } + + /** + * Scoped variant of {@link #setCurrentPackageStatic}: pushes the current + * package value onto the DynamicVariableManager stack so it will be + * restored when the enclosing scope exits, then sets the new value. + *

+ * Matches Perl 5 semantics: {@code package Foo;} is lexically scoped to + * the enclosing block / eval / file. Without the push, a {@code package Foo;} + * inside e.g. {@code Carp::caller_info}'s {@code { package DB; ... }} block + * would leak "DB" past the block, corrupting subsequent {@code do FILE} + * calls (which inherit the caller's package). + */ + public static void setCurrentPackageLocal(String name) { + RuntimeScalar pkg = currentPackage.get(); + org.perlonjava.runtime.runtimetypes.DynamicVariableManager.pushLocalVariable(pkg); + pkg.set(name); + } private static final ThreadLocal> frameStack = ThreadLocal.withInitial(ArrayDeque::new); // Use ArrayList of mutable int holders for O(1) PC updates (no pop/push overhead) diff --git a/src/main/java/org/perlonjava/backend/jvm/EmitOperator.java b/src/main/java/org/perlonjava/backend/jvm/EmitOperator.java index 5ed164e1a..fa85e2982 100644 --- a/src/main/java/org/perlonjava/backend/jvm/EmitOperator.java +++ b/src/main/java/org/perlonjava/backend/jvm/EmitOperator.java @@ -1123,13 +1123,18 @@ static void handlePackageOperator(EmitterVisitor emitterVisitor, OperatorNode no // `require FILE` (which inspects InterpreterState.currentPackage to // compile the required file in the correct namespace) see the right // package after a `package Foo;` declaration in JVM-compiled code. - // Without this, the runtime tracker stays at "main" in compiled code, - // and `require FILE` incorrectly installs subs in main::. + // + // Use the *scoped* (local) variant so the runtime tracker is restored + // when the enclosing block / sub / file exits. Perl 5's `package Foo;` + // is lexically scoped; without the restore, a `package DB;` inside + // e.g. Carp::caller_info's inner `{ package DB; ... }` block would + // leak past the block and break subsequent `do FILE` calls which + // compile the loaded file in the *current* runtime package. emitterVisitor.ctx.mv.visitLdcInsn(name); emitterVisitor.ctx.mv.visitMethodInsn( org.objectweb.asm.Opcodes.INVOKESTATIC, "org/perlonjava/backend/bytecode/InterpreterState", - "setCurrentPackageStatic", + "setCurrentPackageLocal", "(Ljava/lang/String;)V", false); // Set debug information for the file name. diff --git a/src/main/java/org/perlonjava/core/Configuration.java b/src/main/java/org/perlonjava/core/Configuration.java index 20185dc05..b98d57854 100644 --- a/src/main/java/org/perlonjava/core/Configuration.java +++ b/src/main/java/org/perlonjava/core/Configuration.java @@ -33,7 +33,7 @@ public final class Configuration { * Automatically populated by Gradle/Maven during build. * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String gitCommitId = "720a04db3"; + public static final String gitCommitId = "41e739cb5"; /** * Git commit date of the build (ISO format: YYYY-MM-DD). @@ -48,7 +48,7 @@ public final class Configuration { * Parsed by App::perlbrew and other tools via: perl -V | grep "Compiled at" * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String buildTimestamp = "Apr 23 2026 14:30:03"; + public static final String buildTimestamp = "Apr 23 2026 15:46:48"; // Prevent instantiation private Configuration() { From 5b96dd99beda8c8f9ae066adebc62df21af9fd3d Mon Sep 17 00:00:00 2001 From: "Flavio S. Glock" Date: Thu, 23 Apr 2026 16:08:20 +0200 Subject: [PATCH 3/6] docs(dbi): Phase 13 triage results - no code changes, all failures out of scope Triaged t/14utf8.t, t/02dbidrv.t, t/06attrs.t, t/08keeperr.t, t/19fhtrace.t. Each remaining failure maps to one of: - PerlOnJava infra gaps out of DBI scope: - t/14utf8.t: Encode::_utf8_on flag not preserved across hash keys. - t/19fhtrace.t: PerlIO ':via' and ':scalar' custom layers not implemented. - Work already tracked elsewhere: - t/02dbidrv.t: $dbh->DESTROY -> $drh err propagation, covered by a separate DESTROY PR. - Deep PerlOnJava interpreter issue flagged for future dive: - t/06attrs.t and t/08keeperr.t both affected by a bug in DBI::PurePerl's _install_method eval-STRING wrapper where `$h->{dbi_pp_last_method} = $method_name;` persists inside the wrapper (verified with injected print) but vanishes after the wrapper returns. Minimal reproductions outside DBI all work correctly - the bug only triggers inside DBI's actual wrapper with full DBI initialisation. Not cost-effective to debug for ~8 subtests. Documented for future investigation. Docs-only commit; no behaviour change. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- dev/modules/dbi_test_parity.md | 44 ++++++++++++++----- .../org/perlonjava/core/Configuration.java | 4 +- 2 files changed, 36 insertions(+), 12 deletions(-) diff --git a/dev/modules/dbi_test_parity.md b/dev/modules/dbi_test_parity.md index b21416baf..4a65ea1af 100644 --- a/dev/modules/dbi_test_parity.md +++ b/dev/modules/dbi_test_parity.md @@ -937,16 +937,40 @@ Already scoped in previous Next-Steps section. Still ~25 subtests. #### Phase 13: small triage (pure fix-ups) -- `t/08keeperr.t`: `$DBI::err` cleanup on disconnect (~13 subtests). -- `t/06attrs.t`: `ErrCount` bump-on-error, `Statement` on failed - `do` (~10). -- `t/16destroy.t`: stray pre-connect DESTROY (~15). -- `t/19fhtrace.t`: `trace(undef, $fh)` PerlIO layer preservation, - `trace(0, "STDERR")` string alias (~20). -- `t/14utf8.t`: ExampleP `NAME_lc`/`NAME_uc` hash derivation (~5). -- `t/02dbidrv.t`: 2 subtests (~10 across variants). - -**Combined impact:** ~75 subtests. +**Status: triaged 2026-04-23, no code changes landed.** Each +failure in the targeted files traces back to one of three +pre-existing blockers: + +| Test file | Failures | Root cause | Disposition | +|---|---|---|---| +| `t/14utf8.t` | 1/16 | `Encode::_utf8_on` flag not preserved across hash-key storage | PerlOnJava infra gap (strings are JVM `String`, UTF-8 flag tracked externally). Out of DBI scope. | +| `t/02dbidrv.t` | 2/54 | `$dbh->DESTROY` not copying `err`/`errstr`/`state` up to parent `$drh` | Being addressed in a separate PR (DESTROY work). | +| `t/06attrs.t` | 2/166 | `_install_method`-generated eval-STRING wrapper: `$h->{dbi_pp_last_method} = $method_name` persists inside wrapper but is lost after wrapper returns (verified with injected `print` inside & outside wrapper). Minimal repros in isolation all pass. | Deep eval-STRING/closure/tied-hash interaction inside DBI specifically. Not cost-effective to debug for 2 subtests. | +| `t/08keeperr.t` | 3/91 | 2/3 same as `t/06attrs.t` (wrapper's `dbi_pp_last_method` loss makes error messages say "set_err failed" instead of "do failed"). 1/3 warning-count mismatch, downstream of same issue. | Same blocker. | +| `t/19fhtrace.t` | 4/27 | All 4 failing tests use `open $fh, ':via(TraceDBI)'` or `:scalar` PerlIO layers | PerlOnJava doesn't implement PerlIO custom layers. Out of DBI scope. | + +The most interesting finding is the `_install_method` wrapper bug +(see `dotest12.pl` + injected `print STDERR` diagnostics): the +assignment `$h->{'dbi_pp_last_method'} = $method_name;` at line +36 of the generated wrapper: + +- Reports `dbi_pp_last_method=prepare` immediately after the + assignment (verified via injected print). +- Reports `dbi_pp_last_method=undef` in the caller after the + wrapper returns — even though nothing between the assignment + and return modifies `dbi_pp_last_method`. + +Minimal reproductions (including eval-STRING + tied-hash ++ `local` + `&$sub` invocation patterns) all behave identically +between jperl and perl. The bug only manifests inside DBI's +actual `_install_method` wrapper with full DBI initialisation. +The issue is likely in how eval-STRING captures the +`$method_name` closure variable across PerlOnJava's DBI shim +boundary — but is not easily reproducible outside DBI, and not +worth the debug cost for the ~8 subtests it blocks. Flagged for +future deep dive. + +--- #### Deferred / out of scope diff --git a/src/main/java/org/perlonjava/core/Configuration.java b/src/main/java/org/perlonjava/core/Configuration.java index b98d57854..a66184b70 100644 --- a/src/main/java/org/perlonjava/core/Configuration.java +++ b/src/main/java/org/perlonjava/core/Configuration.java @@ -33,7 +33,7 @@ public final class Configuration { * Automatically populated by Gradle/Maven during build. * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String gitCommitId = "41e739cb5"; + public static final String gitCommitId = "21c612dbc"; /** * Git commit date of the build (ISO format: YYYY-MM-DD). @@ -48,7 +48,7 @@ public final class Configuration { * Parsed by App::perlbrew and other tools via: perl -V | grep "Compiled at" * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String buildTimestamp = "Apr 23 2026 15:46:48"; + public static final String buildTimestamp = "Apr 23 2026 16:06:54"; // Prevent instantiation private Configuration() { From 903cbc84f4971208a0a63542ccd3d8ad56f365bf Mon Sep 17 00:00:00 2001 From: "Flavio S. Glock" Date: Thu, 23 Apr 2026 16:22:08 +0200 Subject: [PATCH 4/6] docs(dbi): root-cause t/06attrs.t and t/08keeperr.t failures Investigated the DBI::PurePerl _install_method wrapper failures flagged in the previous triage commit. Traced the exact point of divergence between jperl and real-perl-plus-PurePerl. Root cause is a reproducible PerlOnJava bug in list-form `local` assignment inside eval-STRING-compiled subroutines: my $h = { x => 0 }; my $sub = eval q{ sub { local ($h->{x}) = 99; print "x=$h->{x}\n" } }; $sub->(); # prints "x=0" on jperl (WRONG); "x=99" on real perl The scope-entry/exit machinery works (values restore on sub exit) but the right-hand-side assignment is silently dropped. Scalar form `local $h->{x} = 99;` (no parens) works correctly. Same bug affects `local ($a[0]) = 99;`. Why this breaks DBI: every DBI::PurePerl method wrapper is built via eval STRING and contains `local ($h->{dbi_pp_call_depth}) = $call_depth;`. Because the assignment is a no-op, nested wrapper entries all see call_depth=0, and the innermost wrapper (set_err) incorrectly fires the "failed" error message instead of letting it bubble to the outermost (do). Result: "set_err failed" instead of "do failed". Compare the same trace on real perl vs jperl: real perl jperl [CALLDEPTH do] h.cd=0 [CALLDEPTH do] h.cd=0 [CALLDEPTH prepare] h.cd=1 [CALLDEPTH prepare] h.cd=0 <- wrong [CALLDEPTH set_err] h.cd=2 [CALLDEPTH set_err] h.cd=0 <- wrong err: "db do failed: ..." err: "db set_err failed: ..." Adds: - dev/known-bugs/local_list_assign_eval_string.pl - minimal repro demonstrating the bug with hash-element and array-element list-form local assignments inside eval STRING. - Updated Phase 13 triage section in dbi_test_parity.md to replace the "deep interaction" hand-wave with the precise root cause, reproduction, affected code surface, and expected impact if fixed. Likely fix area: JVM emitter / bytecode compiler path for LIST_ASSIGN targeting LOCAL HASH_ELEMENT / ARRAY_ELEMENT, in the eval-STRING compilation mode (file-scope compilation works). No code change in this commit -- bug report only. If fixed, expect +10-15 DBI suite subtests plus latent benefit for any CPAN module using eval-STRING-generated accessors that localize hash/array elements (Moose/MouseX/similar). Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- .../local_list_assign_eval_string.pl | 42 ++++++ dev/modules/dbi_test_parity.md | 129 +++++++++++++++--- 2 files changed, 149 insertions(+), 22 deletions(-) create mode 100644 dev/known-bugs/local_list_assign_eval_string.pl diff --git a/dev/known-bugs/local_list_assign_eval_string.pl b/dev/known-bugs/local_list_assign_eval_string.pl new file mode 100644 index 000000000..6cfef9f59 --- /dev/null +++ b/dev/known-bugs/local_list_assign_eval_string.pl @@ -0,0 +1,42 @@ +#!/usr/bin/env perl +# Minimal reproduction of PerlOnJava bug: +# `local (HASH_OR_ARRAY_ELEMENT) = value;` inside eval-STRING-compiled +# subs is a no-op for the value assignment (scope restoration still works). +# +# See dev/modules/dbi_test_parity.md "Root cause of t/06attrs.t and +# t/08keeperr.t failures" for context. Blocks proper DBI::PurePerl +# error-message formatting. +# +# Run with both: +# ./jperl dev/known-bugs/local_list_assign_eval_string.pl +# perl dev/known-bugs/local_list_assign_eval_string.pl +# and compare outputs. + +use strict; +use warnings; + +my $h = { x => 0 }; +my @a = (0); + +# Case A: direct file-scope compile — works on both +sub directA { local ($h->{x}) = 42; print "A: h->{x}=$h->{x}\n"; } +directA(); +print "A: after: h->{x}=$h->{x}\n"; + +# Case B: eval-STRING compiled sub, hash-element, list form — BUG on jperl +my $subB = eval q{ sub { local ($h->{x}) = 99; print "B: h->{x}=$h->{x}\n"; } }; +die $@ if $@; +$subB->(); + +# Case C: eval-STRING compiled sub, hash-element, SCALAR form — works +my $subC = eval q{ sub { local $h->{x} = 77; print "C: h->{x}=$h->{x}\n"; } }; +die $@ if $@; +$subC->(); + +# Case D: eval-STRING compiled sub, array-element, list form — BUG on jperl +my $subD = eval q{ sub { local ($a[0]) = 88; print "D: a[0]=$a[0]\n"; } }; +die $@ if $@; +$subD->(); + +print "\nExpected (real perl):\n"; +print "A: h->{x}=42\nA: after: h->{x}=0\nB: h->{x}=99\nC: h->{x}=77\nD: a[0]=88\n"; diff --git a/dev/modules/dbi_test_parity.md b/dev/modules/dbi_test_parity.md index 4a65ea1af..63c7b8b12 100644 --- a/dev/modules/dbi_test_parity.md +++ b/dev/modules/dbi_test_parity.md @@ -945,30 +945,115 @@ pre-existing blockers: |---|---|---|---| | `t/14utf8.t` | 1/16 | `Encode::_utf8_on` flag not preserved across hash-key storage | PerlOnJava infra gap (strings are JVM `String`, UTF-8 flag tracked externally). Out of DBI scope. | | `t/02dbidrv.t` | 2/54 | `$dbh->DESTROY` not copying `err`/`errstr`/`state` up to parent `$drh` | Being addressed in a separate PR (DESTROY work). | -| `t/06attrs.t` | 2/166 | `_install_method`-generated eval-STRING wrapper: `$h->{dbi_pp_last_method} = $method_name` persists inside wrapper but is lost after wrapper returns (verified with injected `print` inside & outside wrapper). Minimal repros in isolation all pass. | Deep eval-STRING/closure/tied-hash interaction inside DBI specifically. Not cost-effective to debug for 2 subtests. | -| `t/08keeperr.t` | 3/91 | 2/3 same as `t/06attrs.t` (wrapper's `dbi_pp_last_method` loss makes error messages say "set_err failed" instead of "do failed"). 1/3 warning-count mismatch, downstream of same issue. | Same blocker. | +| `t/06attrs.t` | 2/166 | **PerlOnJava bug**: `local ($h->{key}) = value` (list form) inside eval-STRING-compiled subs is a no-op for the assignment. `local` scope restoration works; the RHS assignment doesn't take effect. See "Root cause of t/06attrs.t and t/08keeperr.t failures" below. | **Fixable PerlOnJava bug** — flagged for follow-up. | +| `t/08keeperr.t` | 3/91 | Same bug as t/06attrs.t. DBI::PurePerl's `_install_method` wrapper uses `local ($h->{'dbi_pp_call_depth'}) = $call_depth;` to track call depth. The no-op means every wrapper sees `dbi_pp_call_depth = 0`, so nested error handling fires in the innermost wrapper (`set_err`) instead of bubbling to the outermost (`do`). Error message becomes `"set_err failed"` instead of `"do failed"`. | Same root cause — flagged. | | `t/19fhtrace.t` | 4/27 | All 4 failing tests use `open $fh, ':via(TraceDBI)'` or `:scalar` PerlIO layers | PerlOnJava doesn't implement PerlIO custom layers. Out of DBI scope. | -The most interesting finding is the `_install_method` wrapper bug -(see `dotest12.pl` + injected `print STDERR` diagnostics): the -assignment `$h->{'dbi_pp_last_method'} = $method_name;` at line -36 of the generated wrapper: - -- Reports `dbi_pp_last_method=prepare` immediately after the - assignment (verified via injected print). -- Reports `dbi_pp_last_method=undef` in the caller after the - wrapper returns — even though nothing between the assignment - and return modifies `dbi_pp_last_method`. - -Minimal reproductions (including eval-STRING + tied-hash -+ `local` + `&$sub` invocation patterns) all behave identically -between jperl and perl. The bug only manifests inside DBI's -actual `_install_method` wrapper with full DBI initialisation. -The issue is likely in how eval-STRING captures the -`$method_name` closure variable across PerlOnJava's DBI shim -boundary — but is not easily reproducible outside DBI, and not -worth the debug cost for the ~8 subtests it blocks. Flagged for -future deep dive. +The most interesting finding is a **reproducible PerlOnJava bug** +in `local (hash-or-array-element) = value` list assignment inside +eval-STRING-compiled subs. + +### Root cause of t/06attrs.t and t/08keeperr.t failures + +**Bug:** When a Perl subroutine is compiled via `eval STRING` and +contains `local ($href->{key}) = value;` (or `local ($aref->[i]) = value;`), +the `local` scope entry/exit machinery fires correctly, but the +**value assignment is silently dropped**. Inside the sub's body, +reading back the element returns the pre-local value, not the +assigned one. + +**Minimal repro** (append to a Perl script): + +```perl +my $h = { x => 0 }; + +# Case A: direct file-scope compile — works +sub directA { local ($h->{x}) = 42; print "A: x=$h->{x}\n"; } +directA(); # prints "A: x=42" (correct) + +# Case B: eval-STRING compiled sub — BUG +my $subB = eval q{ sub { local ($h->{x}) = 99; print "B: x=$h->{x}\n"; } }; +$subB->(); # prints "B: x=0" (WRONG) +``` + +Expected output (real Perl): `A: x=42`, `B: x=99`. +Actual jperl output: `A: x=42`, `B: x=0`. + +**Scalar form works correctly:** `local $h->{x} = 99;` (without +outer parens) is fine. Only the list-assignment form is broken. + +**Array-element list-form has the same bug**: +`local ($a[0]) = 99;` inside eval-STRING. + +### Why this breaks DBI + +DBI::PurePerl's `_install_method` generates wrappers via +`eval qq{#line 1 "..."\n$method_code}`. Every generated wrapper +contains: + +```perl +my $call_depth = $h->{'dbi_pp_call_depth'} + 1; +local ($h->{'dbi_pp_call_depth'}) = $call_depth; # ← bug: assignment is a no-op +``` + +Because the assignment is dropped, `$h->{dbi_pp_call_depth}` stays +at `0` for every nested wrapper entry. The error-handling +`post_call_frag` then incorrectly thinks each wrapper is the +outermost one and fires the "failed" message. For +`$dbh->do('bad sql')`, the error bubbles through +`do → prepare → ExampleP::prepare → set_err`; because set_err's +wrapper sees `call_depth <= 1`, it fires with +`"set_err failed"` instead of letting do's wrapper fire with +`"do failed"`. + +Verified trace (injected `print STDERR` at every call_depth +compute + every pre-call-frag dbi_pp_last_method set): + +``` +[CALLDEPTH do] computed_call_depth=1 h.cd_before_local=0 +[CALLDEPTH prepare] computed_call_depth=1 h.cd_before_local=0 ← should be 2 +[CALLDEPTH set_err] computed_call_depth=1 h.cd_before_local=0 ← should be 3 +err: DBD::ExampleP::db set_err failed: ... ← should say "do failed" +``` + +On real Perl + DBI::PurePerl the same trace shows +`h.cd_before_local=0, 1, 2` respectively and the error is +`"db do failed"`. + +### Affected code + +Likely in the JVM emitter's handling of `LOCAL` op on +`HASH_ELEMENT` / `ARRAY_ELEMENT` targets when the containing sub +is produced via eval-STRING. The compile-time bytecode for +list-assignment localization isn't emitting the store on the +RHS value. The pattern: + +```perl +local (LVALUE_LIST) = RVALUE_LIST +``` + +Should be semantically equivalent to: +```perl +local LVALUE_LIST[0] = RVALUE_LIST[0]; +local LVALUE_LIST[1] = RVALUE_LIST[1]; +... +``` + +The scalar-single-element variant works (`local $h->{x} = $v`), +suggesting the bug is in the list-context emitter path for +`local (...)` with a single hash/array element, most likely +specific to eval-STRING's bytecode compiler (since file-scope +compilation of the same code works). + +### Impact if fixed + +- `t/06attrs.t`: 2/166 → expected 0/166 +- `t/08keeperr.t`: 3/91 → expected 0-1/91 (1 downstream effect) +- Full suite: +10–15 subtests across wrapper variants +- Latent bug affecting any CPAN module that uses eval-STRING- + generated subs with localized hash/array elements (DBI being + the most visible, but Moose/MouseX/etc. accessors may also + rely on this pattern) --- From 7a0687aef133c0e65e566d56a4f8e2fe7629ca2d Mon Sep 17 00:00:00 2001 From: "Flavio S. Glock" Date: Thu, 23 Apr 2026 16:56:57 +0200 Subject: [PATCH 5/6] fix(interpreter): list-form local assignment on hash/array elements `local ($h->{key}) = value;` and `local ($a[i]) = value;` in the bytecode-interpreter backend silently dropped the right-hand-side assignment. The `local` scope-entry/exit machinery fired correctly (the value was restored on scope exit), but no SET happened in between -- effectively a no-op. Scalar form `local $h->{key} = v;` (no outer parens) was unaffected; the JVM backend was unaffected for file-scope code but hit the bug via eval STRING (which compiles to bytecode). Root cause: `CompileAssignment.handleLocalListAssignment` iterated over the LHS list and only emitted bytecode when an element was `OperatorNode("$", IdentifierNode)` (i.e. a plain `$name`). Elements that were `BinaryOperatorNode` -- hash-element (`$h->{k}`), array- element (`$a[i]`), arrow chains (`$obj->method->{k}`) -- fell through both the size==1 special case and the main loop with no bytecode emitted. Fix: added a `BinaryOperatorNode` branch in both paths: - size==1 case: compile element -> PUSH_LOCAL_VARIABLE -> SET_SCALAR from RHS (matches the existing scalar-context `local EXPR = RHS` handler at the top of handleLocalAssignment). - multi-element loop: compile element -> PUSH_LOCAL_VARIABLE -> ARRAY_GET RHS[i] -> SET_SCALAR. Why this matters for DBI: DBI::PurePerl's `_install_method` wraps every method in an eval-STRING'd sub containing `local ($h->{dbi_pp_call_depth}) = $call_depth;`. Because the assignment was dropped, every nested wrapper saw call_depth=0, so error/warning messages fired from the innermost wrapper (set_err) instead of bubbling to the outermost (do). Users saw "DBD::Foo::db set_err failed: ..." instead of "db do failed: ...". Impact on `jcpan -t DBI` full suite: - 6210 -> 6256 passing (+46 subtests) - 76 -> 64 failed files (-12) Latent bug also affecting any CPAN module with eval-STRING-generated subs that localize hash/array elements (DBI::PurePerl was the most visible; Moose/MouseX-style accessor generators may also benefit). Regression test added at dev/known-bugs/local_list_assign_eval_string.pl (renamed for history; still present as the reproducer). The script now passes on both JVM and --interpreter backends. `make` passes; no unit-test regressions. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- dev/modules/dbi_test_parity.md | 55 +++++++++++++++---- .../backend/bytecode/CompileAssignment.java | 38 +++++++++++++ .../org/perlonjava/core/Configuration.java | 4 +- 3 files changed, 85 insertions(+), 12 deletions(-) diff --git a/dev/modules/dbi_test_parity.md b/dev/modules/dbi_test_parity.md index 63c7b8b12..5e4246ab4 100644 --- a/dev/modules/dbi_test_parity.md +++ b/dev/modules/dbi_test_parity.md @@ -5,14 +5,18 @@ DBI test suite, 200 test files) pass on PerlOnJava. ## Current Baseline -After Phase 10 (2026-04-23): PerlOnJava bug fix — `package Foo;` -now lexically scoped at runtime (not just at compile time). See -Phase 10 section below. +After Phase 10b (2026-04-23): fixed the bytecode-interpreter +list-form `local` assignment bug (see Phase 10b section below). -| | Files | Subtests | Passing | Failing | -|---|---|---|---|---| -| `jcpan -t DBI` (2026-04-23, post-Phase-10) | 200 | **6600** | **6210 (94 %)** | 390 | -| (pre-Phase-10 baseline) | 200 | 5944 | 5566 (94 %) | 378 | +| | Files | Subtests | Passing | Failing | Files failed | +|---|---|---|---|---|---| +| `jcpan -t DBI` (post-Phase-10b) | 200 | **6600** | **6256 (95 %)** | 344 | 64/200 | +| (post-Phase-10) | 200 | 6600 | 6210 (94 %) | 390 | 76/200 | +| (pre-Phase-10) | 200 | 5944 | 5566 (94 %) | 378 | 76/200 | + +See "Fresh baseline (2026-04-23)" section below for per-file +failure distribution and the revised phase 10+ plan (skipped +tests stay skipped — no `$DBI::PurePerl` flag flip). --- @@ -945,13 +949,18 @@ pre-existing blockers: |---|---|---|---| | `t/14utf8.t` | 1/16 | `Encode::_utf8_on` flag not preserved across hash-key storage | PerlOnJava infra gap (strings are JVM `String`, UTF-8 flag tracked externally). Out of DBI scope. | | `t/02dbidrv.t` | 2/54 | `$dbh->DESTROY` not copying `err`/`errstr`/`state` up to parent `$drh` | Being addressed in a separate PR (DESTROY work). | -| `t/06attrs.t` | 2/166 | **PerlOnJava bug**: `local ($h->{key}) = value` (list form) inside eval-STRING-compiled subs is a no-op for the assignment. `local` scope restoration works; the RHS assignment doesn't take effect. See "Root cause of t/06attrs.t and t/08keeperr.t failures" below. | **Fixable PerlOnJava bug** — flagged for follow-up. | -| `t/08keeperr.t` | 3/91 | Same bug as t/06attrs.t. DBI::PurePerl's `_install_method` wrapper uses `local ($h->{'dbi_pp_call_depth'}) = $call_depth;` to track call depth. The no-op means every wrapper sees `dbi_pp_call_depth = 0`, so nested error handling fires in the innermost wrapper (`set_err`) instead of bubbling to the outermost (`do`). Error message becomes `"set_err failed"` instead of `"do failed"`. | Same root cause — flagged. | +| `t/06attrs.t` | 2/166 | **PerlOnJava bug — FIXED in Phase 10b**: `local ($h->{key}) = value` (list form) in the bytecode-interpreter backend silently dropped the RHS assignment. | **FIXED** | +| `t/08keeperr.t` | 3/91 | Same bug as t/06attrs.t. | **FIXED** | | `t/19fhtrace.t` | 4/27 | All 4 failing tests use `open $fh, ':via(TraceDBI)'` or `:scalar` PerlIO layers | PerlOnJava doesn't implement PerlIO custom layers. Out of DBI scope. | The most interesting finding is a **reproducible PerlOnJava bug** in `local (hash-or-array-element) = value` list assignment inside -eval-STRING-compiled subs. +eval-STRING-compiled subs. **Fixed in Phase 10b.** + +### Phase 10b: list-form `local` assignment on hash/array elements + +**Status: done (2026-04-23).** Fixed in +`src/main/java/org/perlonjava/backend/bytecode/CompileAssignment.java`. ### Root cause of t/06attrs.t and t/08keeperr.t failures @@ -1055,6 +1064,32 @@ compilation of the same code works). the most visible, but Moose/MouseX/etc. accessors may also rely on this pattern) +### Fix applied (Phase 10b) + +`CompileAssignment.handleLocalListAssignment` iterated over the +list elements but only emitted bytecode for +`OperatorNode("$" + IdentifierNode)` sigil-variable elements. +Elements that were `BinaryOperatorNode` (i.e. `$h->{key}`, +`$a[i]`, `$obj->method->{k}`, etc.) were silently skipped — +no assignment bytecode emitted. + +Fix: added a `BinaryOperatorNode` branch in both the +single-element special case and the multi-element loop. For each +such element, emit: +1. Compile the element as an lvalue (gets the element scalar ref). +2. `PUSH_LOCAL_VARIABLE` to save the value for scope-exit restore. +3. Multi-element: `ARRAY_GET` to pull RHS[i] from the value list. +4. `SET_SCALAR` to assign. + +Measured impact on `jcpan -t DBI`: +- 6210 → **6256 passing subtests** (+46) +- 76 → **64 failed files** (-12) + +The overshoot vs the predicted "+10-15" is because many more +DBI tests indirectly depended on `dbi_pp_call_depth` tracking +working correctly (error messages, warning messages, method- +dispatch trace format). + --- #### Deferred / out of scope diff --git a/src/main/java/org/perlonjava/backend/bytecode/CompileAssignment.java b/src/main/java/org/perlonjava/backend/bytecode/CompileAssignment.java index 9dce702b2..dc03e0414 100644 --- a/src/main/java/org/perlonjava/backend/bytecode/CompileAssignment.java +++ b/src/main/java/org/perlonjava/backend/bytecode/CompileAssignment.java @@ -261,6 +261,24 @@ private static boolean handleLocalListAssignment(BytecodeCompiler bc, BinaryOper bc.lastResultReg = localReg; return true; } + // Single-element list with an lvalue like $h->{k}, $a[i], $obj->method->{k}, etc. + // Delegate to the scalar-local handler (matches the `local EXPR = RHS` path at + // line 20). Without this, the element falls through the main loop below and + // emits nothing - a silent no-op assignment. Reproduced by: + // local ($h->{x}) = 99; inside an eval-STRING-compiled sub + if (element instanceof BinaryOperatorNode binOp) { + bc.compileNode(binOp, -1, rhsContext); + int elemReg = bc.lastResultReg; + bc.emit(Opcodes.PUSH_LOCAL_VARIABLE); + bc.emitReg(elemReg); + bc.compileNode(node.right, -1, rhsContext); + int valueReg = bc.lastResultReg; + bc.emit(Opcodes.SET_SCALAR); + bc.emitReg(elemReg); + bc.emitReg(valueReg); + bc.lastResultReg = elemReg; + return true; + } } bc.compileNode(node.right, -1, rhsContext); int valueReg = bc.lastResultReg; @@ -292,6 +310,26 @@ private static boolean handleLocalListAssignment(BytecodeCompiler bc, BinaryOper bc.emitReg(localReg); bc.emitReg(elemReg); if (i == 0) bc.lastResultReg = localReg; + } else if (element instanceof BinaryOperatorNode binOp) { + // Element is an lvalue expression (e.g. $h->{k}, $a[i], $obj->attr). + // Compile to get the element reference, localize it, and assign RHS[i]. + bc.compileNode(binOp, -1, RuntimeContextType.SCALAR); + int elemLvalReg = bc.lastResultReg; + bc.emit(Opcodes.PUSH_LOCAL_VARIABLE); + bc.emitReg(elemLvalReg); + int idxReg = bc.allocateRegister(); + bc.emit(Opcodes.LOAD_INT); + bc.emitReg(idxReg); + bc.emit(i); + int rhsElemReg = bc.allocateRegister(); + bc.emit(Opcodes.ARRAY_GET); + bc.emitReg(rhsElemReg); + bc.emitReg(valueReg); + bc.emitReg(idxReg); + bc.emit(Opcodes.SET_SCALAR); + bc.emitReg(elemLvalReg); + bc.emitReg(rhsElemReg); + if (i == 0) bc.lastResultReg = elemLvalReg; } } return true; diff --git a/src/main/java/org/perlonjava/core/Configuration.java b/src/main/java/org/perlonjava/core/Configuration.java index a66184b70..7883dab1b 100644 --- a/src/main/java/org/perlonjava/core/Configuration.java +++ b/src/main/java/org/perlonjava/core/Configuration.java @@ -33,7 +33,7 @@ public final class Configuration { * Automatically populated by Gradle/Maven during build. * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String gitCommitId = "21c612dbc"; + public static final String gitCommitId = "903cbc84f"; /** * Git commit date of the build (ISO format: YYYY-MM-DD). @@ -48,7 +48,7 @@ public final class Configuration { * Parsed by App::perlbrew and other tools via: perl -V | grep "Compiled at" * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String buildTimestamp = "Apr 23 2026 16:06:54"; + public static final String buildTimestamp = "Apr 23 2026 16:26:30"; // Prevent instantiation private Configuration() { From 93797b7d56b92659e9d5b1399cf7e02a1d359d42 Mon Sep 17 00:00:00 2001 From: "Flavio S. Glock" Date: Thu, 23 Apr 2026 18:55:37 +0200 Subject: [PATCH 6/6] feat(XSLoader): reject known pure-XS modules so require fails cleanly MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CPAN's DB_File, BerkeleyDB, SDBM_File, GDBM_File, NDBM_File, ODBM_File are pure-XS modules with no pure-Perl fallback. In PerlOnJava, our XSLoader::load was silently returning success for them, so `require DB_File` appeared to work but the XS helpers like DB_File::constant were never defined. The first real use (e.g. `tie %h, 'DB_File', ...`) triggered infinite AUTOLOAD -> constant -> AUTOLOAD recursion ending in StackOverflowError. CPAN test runners like DBI's t/50dbm_simple.t probe optional DBM backends with: @dbm_types = grep { eval { require "$_.pm" } } @dbms; plan skip_all => "No DBM modules available" unless @dbm_types; This pattern requires `require` to FAIL for unavailable backends. Our silent success broke it. Fix: - XS_ONLY_NOT_SUPPORTED blacklist in both XSLoader.pm and XSLoader.java (kept in sync). XSLoader::load dies with a clear "XS module not supported on PerlOnJava" message, which the caller's `eval` catches and the backend probe falls through. - installEndBlockStubs("BerkeleyDB"): registers a no-op Perl sub for BerkeleyDB::Term::close_everything. The module's END block is registered at compile time, BEFORE our XSLoader::load dies at runtime, so the END queue still fires at interpreter shutdown. Without the stub, the program exits non-zero and prove counts it as a failed test program even if it SKIPped all subtests. Impact on `jcpan -t DBI` full suite: - Failing subtests: 344 -> 144 (-200) - Failing files: 64 -> 48 (-16) Per-file wins: t/50dbm_simple.t + variants: 16/38 fail x 5 -> SKIP x 5 t/52dbm_complex.t: partial crash -> SKIP t/53sqlengine_adv.t: crash -> SKIP t/49dbd_file.t (base): 9/65 fail -> 65/65 pass The "passing" column drops from 6256 -> 5992 because ~464 subtests that formerly ran (and mostly failed) inside these files now skip entirely — the correct outcome for CPAN-style backend probing. make passes; no unit-test regressions. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- dev/modules/dbi_test_parity.md | 80 ++++++++++++++---- .../org/perlonjava/core/Configuration.java | 4 +- .../runtime/perlmodule/XSLoader.java | 82 +++++++++++++++++++ src/main/perl/lib/XSLoader.pm | 36 +++++++- 4 files changed, 182 insertions(+), 20 deletions(-) diff --git a/dev/modules/dbi_test_parity.md b/dev/modules/dbi_test_parity.md index 5e4246ab4..a11f097dc 100644 --- a/dev/modules/dbi_test_parity.md +++ b/dev/modules/dbi_test_parity.md @@ -5,18 +5,24 @@ DBI test suite, 200 test files) pass on PerlOnJava. ## Current Baseline -After Phase 10b (2026-04-23): fixed the bytecode-interpreter -list-form `local` assignment bug (see Phase 10b section below). +After Phase 11 (2026-04-23): XSLoader now rejects known-XS-only +modules cleanly so `eval { require SomeDBM }` probes fall through +to alternatives, and the DBM family tests SKIP instead of crashing. | | Files | Subtests | Passing | Failing | Files failed | |---|---|---|---|---|---| -| `jcpan -t DBI` (post-Phase-10b) | 200 | **6600** | **6256 (95 %)** | 344 | 64/200 | +| `jcpan -t DBI` (post-Phase-11) | 200 | 6136 | 5992 | **144** | **48/200** | +| (post-Phase-10b) | 200 | 6600 | 6256 (95 %) | 344 | 64/200 | | (post-Phase-10) | 200 | 6600 | 6210 (94 %) | 390 | 76/200 | | (pre-Phase-10) | 200 | 5944 | 5566 (94 %) | 378 | 76/200 | -See "Fresh baseline (2026-04-23)" section below for per-file -failure distribution and the revised phase 10+ plan (skipped -tests stay skipped — no `$DBI::PurePerl` flag flip). +**Phase 11 delta: -200 failing subtests, -16 failed files.** The +"passing" and "subtests" columns drop because ~464 subtests that +formerly ran (and mostly failed) inside `t/50dbm_simple.t`, +`t/52dbm_complex.t`, `t/85gofer.t` and friends now skip entirely +via `plan skip_all => "No DBM modules available"`. This is the +correct outcome — CPAN-style backend probing is supposed to SKIP +when no backend works. --- @@ -923,17 +929,61 @@ module that uses `{ package X; ... }` — Carp itself being a prominent example, but the pattern is common for debugger- compatibility shims. -### Phase 11: filesystem locking for DBM tests +### Phase 11: DBM backend probing fails cleanly -`t/50dbm_simple.t`, `t/51dbm_file.t`, and the `49dbd_file.t` -family die with "Resource deadlock avoided" from -`DBI::DBD::SqlEngine` at the `flock`/`fcntl` call. Needs: -- Verify our `flock`/`fcntl` behave on the same fd/inode pair - the way CPython/Perl does on macOS (FD-based vs inode-based). -- Either match Perl's behaviour or skip these test families on - PerlOnJava. +**Status: done (2026-04-23).** Fixed in +`src/main/java/org/perlonjava/runtime/perlmodule/XSLoader.java` +and `src/main/perl/lib/XSLoader.pm`. + +**Root cause.** CPAN's `DB_File`, `BerkeleyDB`, `SDBM_File`, +`GDBM_File`, `NDBM_File`, `ODBM_File` are pure-XS modules with +no pure-Perl fallback. In PerlOnJava, `require DB_File` succeeded +silently (XSLoader::load returned true) but the XS helpers like +`DB_File::constant` were never defined. The first real use +triggered an infinite `AUTOLOAD → constant → AUTOLOAD` recursion +ending in `StackOverflowError`. + +CPAN test runners (DBI's `t/50dbm_simple.t` et al.) probe +optional backends with: +```perl +my @dbms = qw(SDBM_File GDBM_File DB_File BerkeleyDB NDBM_File ODBM_File); +@dbm_types = grep { eval { require "$_.pm" } } @dbms; +plan skip_all => "No DBM modules available" unless @dbm_types; +``` +This pattern relies on `require` **failing** for unavailable +backends. Because XSLoader silently returned success, the test +picked DB_File, then crashed on use. + +**Fix.** +- Added an `XS_ONLY_NOT_SUPPORTED` blacklist in both + `XSLoader.pm` and `XSLoader.java` (kept in sync). When + `XSLoader::load("DB_File", ...)` etc. is called, die with + `"XS module not supported on PerlOnJava"`. The caller's + `eval { require ... }` catches it and the probe falls through. +- Added `installEndBlockStubs("BerkeleyDB")` which registers a + no-op Perl sub for `BerkeleyDB::Term::close_everything` when + BerkeleyDB fails to load. Without this, the module's END block + (registered at compile time, before our die runs) would fire + `close_everything()` on interpreter shutdown, causing a + non-zero exit status that prove/TAP::Harness flags as a failed + program — even for tests that otherwise passed or SKIPped. + +**Impact.** -**Impact:** ~95 subtests (combined DBM + DBD::File variants). +| Test file | Before | After | +|---|---|---| +| `t/50dbm_simple.t` + variants | 16/38 fail × 5 | **SKIP × 5** | +| `t/52dbm_complex.t` | partial / crash | **SKIP** | +| `t/53sqlengine_adv.t` | crash | **SKIP** | +| `t/49dbd_file.t` (base) | 9/65 fail | **passes 65/65** | + +Full-suite: +- Failing subtests: 344 → **144** (-200) +- Failing files: 64 → **48** (-16) + +Still failing in this family: `t/51dbm_file.t` (2 fails across +variants — hard-requires a DBM backend without `eval`). Would +require patching the test, out of scope. #### Phase 12: execute_array (t/15array.t) diff --git a/src/main/java/org/perlonjava/core/Configuration.java b/src/main/java/org/perlonjava/core/Configuration.java index 7883dab1b..e4fdd8a4a 100644 --- a/src/main/java/org/perlonjava/core/Configuration.java +++ b/src/main/java/org/perlonjava/core/Configuration.java @@ -33,7 +33,7 @@ public final class Configuration { * Automatically populated by Gradle/Maven during build. * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String gitCommitId = "903cbc84f"; + public static final String gitCommitId = "7a0687aef"; /** * Git commit date of the build (ISO format: YYYY-MM-DD). @@ -48,7 +48,7 @@ public final class Configuration { * Parsed by App::perlbrew and other tools via: perl -V | grep "Compiled at" * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String buildTimestamp = "Apr 23 2026 16:26:30"; + public static final String buildTimestamp = "Apr 23 2026 18:51:50"; // Prevent instantiation private Configuration() { diff --git a/src/main/java/org/perlonjava/runtime/perlmodule/XSLoader.java b/src/main/java/org/perlonjava/runtime/perlmodule/XSLoader.java index c764801d0..a0347de45 100644 --- a/src/main/java/org/perlonjava/runtime/perlmodule/XSLoader.java +++ b/src/main/java/org/perlonjava/runtime/perlmodule/XSLoader.java @@ -33,6 +33,30 @@ public class XSLoader extends PerlModuleBase { "XSLoader" ); + /** + * Modules that are pure-XS in real Perl with no PerlOnJava-side implementation. + * When XSLoader::load is called for one of these, we die cleanly so that + * {@code eval { require SomeModule }} in CPAN code (and test suites that + * probe for optional backends like DBM engines) correctly falls through + * to alternatives instead of silently "succeeding" and then crashing + * later when methods are actually called. + *

+ * Rule of thumb: the module's whole functionality lives in a shared + * library shipped with CPAN, and there is no pure-Perl or Java-backed + * replacement in PerlOnJava. Pre-registered Java modules (File::Glob, + * Encode, Time::HiRes, etc.) must NOT appear here. + *

+ * Kept in sync with the Perl-side copy in {@code lib/XSLoader.pm}. + */ + private static final Set XS_ONLY_NOT_SUPPORTED = Set.of( + "DB_File", + "BerkeleyDB", + "GDBM_File", + "NDBM_File", + "ODBM_File", + "SDBM_File" + ); + /** * Constructor for XSLoader. * Initializes the module with the name "XSLoader". @@ -57,6 +81,41 @@ public static void initialize() { } } + /** + * Installs no-op Perl subroutines for XS symbols that a failed-to-load + * module's END block is known to call. Without these, the END queue + * aborts on interpreter shutdown with a non-zero exit status, which + * prove/TAP::Harness counts as a failed test program even when the + * program's actual assertions all passed or were SKIPped. + * + * Keyed by the module name passed to {@code XSLoader::load}. + */ + private static void installEndBlockStubs(String moduleName) { + String[] symbols = switch (moduleName) { + case "BerkeleyDB" -> new String[] { "BerkeleyDB::Term::close_everything" }; + default -> null; + }; + if (symbols == null) return; + try { + java.lang.invoke.MethodHandle mh = RuntimeCode.lookup.findStatic( + XSLoader.class, "noopStub", RuntimeCode.methodType); + for (String sym : symbols) { + if (GlobalVariable.existsGlobalCodeRef(sym)) continue; + RuntimeCode code = new RuntimeCode(mh, null, null); + code.isStatic = true; + GlobalVariable.getGlobalCodeRef(sym).set(new RuntimeScalar(code)); + } + } catch (Exception e) { + // Non-fatal: the test program may still report a spurious non-zero + // exit, but the module-load failure path is unaffected. + } + } + + /** No-op Perl sub used by {@link #installEndBlockStubs}. */ + public static RuntimeList noopStub(RuntimeArray args, int ctx) { + return new RuntimeList(); + } + /** * Loads a PerlOnJava module. *

@@ -90,6 +149,29 @@ public static RuntimeList load(RuntimeArray args, int ctx) { moduleName = args.getFirst().toString(); } + // Bail out cleanly for pure-XS modules PerlOnJava can't back. + // Without this, modules like DB_File load but their XS helpers + // (constant, etc.) are undefined, leading to infinite AUTOLOAD + // recursion (StackOverflowError) the first time the module is + // actually used. CPAN test suites commonly probe optional backends + // with `eval { require SomeDBM }` and rely on require FAILING to + // fall through to alternatives; silent success breaks them. + if (XS_ONLY_NOT_SUPPORTED.contains(moduleName)) { + // Install no-op stubs for any functions the module registers in an + // END block — the `.pm` file was already compiled end-to-end before + // we reach this `load`, so its END queue entries will fire at + // interpreter shutdown regardless of whether `require` succeeds. + // Without these, CPAN prove-style runners report the (otherwise- + // passing / SKIPped) test program as "exited 1" from the END die. + installEndBlockStubs(moduleName); + + return WarnDie.die( + new RuntimeScalar("Can't load '" + moduleName + "' for module " + moduleName + + ": XS module not supported on PerlOnJava"), + new RuntimeScalar("\n") + ).getList(); + } + // Convert Perl::Module::Name to org.perlonjava.runtime.perlmodule.PerlModuleName String[] parts = moduleName.split("::"); StringBuilder className1 = new StringBuilder("org.perlonjava.runtime.perlmodule."); diff --git a/src/main/perl/lib/XSLoader.pm b/src/main/perl/lib/XSLoader.pm index c69045b3f..81dc61d85 100644 --- a/src/main/perl/lib/XSLoader.pm +++ b/src/main/perl/lib/XSLoader.pm @@ -15,26 +15,56 @@ package XSLoader; our $VERSION = "0.32"; +# Modules that are pure-XS in real Perl with no PerlOnJava-side implementation. +# When XSLoader::load is called for one of these, we die cleanly so that +# `eval { require SomeModule }` in CPAN code (and test suites that probe for +# optional backends like DBM engines) correctly falls through to alternatives +# instead of silently "succeeding" and then crashing later when methods are +# actually called. +# +# Rule of thumb for adding to this list: the module's whole functionality +# lives in a `.so`/DLL shipped with CPAN, and there is no pure-Perl or +# Java-backed replacement in PerlOnJava. Pre-registered Java modules +# (File::Glob, Encode, Time::HiRes, etc.) must NOT appear here. +our %XS_ONLY_NOT_SUPPORTED = map { $_ => 1 } qw( + DB_File + BerkeleyDB + GDBM_File + NDBM_File + ODBM_File + SDBM_File +); + # Only define our load() if it's not already defined by Java BEGIN { unless (defined &load) { *load = sub { my ($module, $version) = @_; $module = caller() unless defined $module; - + + # Bail out cleanly for pure-XS modules PerlOnJava can't back. + # Without this, modules like DB_File load but XS functions such + # as `constant` are undefined, which triggers infinite AUTOLOAD + # recursion (StackOverflowError) the first time the module is + # actually used. + if ($XS_ONLY_NOT_SUPPORTED{$module}) { + die "Can't load '$module' for module $module: " + . "XS module not supported on PerlOnJava\n"; + } + # Check if the module has a bootstrap function (like standard XSLoader) my $boots = "${module}::bootstrap"; if (defined &{$boots}) { goto &{$boots}; } - + # For Java-backed modules, the methods are already registered. # For pure-Perl modules, nothing needs to be done. # Either way, just return success. return 1; }; } - + # Alias for compatibility *bootstrap_inherit = \&load unless defined &bootstrap_inherit; }