Skip to content

Commit 44bc18b

Browse files
fix: push CallerStack for INIT/CHECK/END blocks so caller() works
INIT/CHECK/END blocks in PerlOnJava were executed directly from Java (SpecialBlock.runInitBlocks() → RuntimeCode.apply()) with no Perl caller frame above them. This caused caller(1) to return empty, breaking Test2::IPC which calls context() (using caller(1)) in an INIT block. Fix: wrap INIT/CHECK/END block execution in PerlLanguageProvider with CallerStack.push("main",...)/pop(), matching Perl 5 behavior where these blocks run from the main program scope. This unblocks ~40 App::perlbrew tests that use Test2::Tools::Spec → Test2::AsyncSubtest → Test2::IPC. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
1 parent f97aa6c commit 44bc18b

3 files changed

Lines changed: 137 additions & 59 deletions

File tree

dev/modules/app_perlbrew.md

Lines changed: 109 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# App::perlbrew CPAN Installation Plan
22

3-
## Status: Phase 1 Complete (2026-04-07)
3+
## Status: Phase 5 in progress (2026-04-07)
44

55
## Goal
66

@@ -19,20 +19,20 @@ App::perlbrew 1.02
1919
├── Test2::Plugin::IOEvents (build_requires)
2020
├── local::lib >= 2.000014 (requires)
2121
│ └── uses `perl - args` stdin idiom in Makefile.PL
22-
└── (test dependencies: Test2::V0, FindBin, English.pm)
22+
└── (test dependencies: Test2::V0, Test2::Tools::Spec, FindBin, English.pm)
2323
```
2424

25-
## Test Run Summary (2026-04-07, after Phase 1 fixes)
25+
## Test Run Summary (2026-04-07, after Phase 4 fixes)
2626

2727
| Module | Configure | Build | Test | Blocker |
2828
|--------|-----------|-------|------|---------|
2929
| Module::Build::Tiny | OK | OK | **32/32 PASS** ||
3030
| CPAN::Perl::Releases | OK | OK | 104/105 (1 fail) | `blib/arch` not created |
3131
| Devel::PatchPerl | OK | OK | 28/28 pass, 3 programs fail | Missing `File::pushd` |
32-
| File::Which | OK | OK | 14/18 (4 fail) | `catpath()` prototype bug |
33-
| Test2::Plugin::IOEvents | OK | OK | FAIL | Test2::V0 import issue |
34-
| local::lib | **NOT OK** | | | `-` stdin arg not supported |
35-
| App::perlbrew | OK | OK | 4/73 (most crash) | Test2::V0 import + FindBin + English.pm |
32+
| File::Which | OK | OK | 14/18 (4 fail) | `catpath()` prototype bug *(fixed in Phase 2)* |
33+
| Test2::Plugin::IOEvents | OK | OK | FAIL | Test2::V0 import issue *(fixed in Phase 4)* |
34+
| local::lib | OK | OK | 26/32 pass, shell.t hangs | `-` stdin *(fixed in Phase 2)*, PATH in sub-shells |
35+
| App::perlbrew | OK | OK | **18/73 pass** | Test2::IPC context depth *(Phase 5)* |
3636

3737
---
3838

@@ -133,43 +133,28 @@ MakeMaker may not create `blib/arch` during `make`. Standard Perl's `make` alway
133133

134134
---
135135

136-
## Phase 3: Parser and Import System Fixes
137-
138-
### 3.1 Test2::V0 / Test2::Util::Importer function import chain
139-
140-
**Priority: HIGH** — Root cause of most App::perlbrew test failures (syntax errors on
141-
`is`, `subtest`, `prop`, etc.).
142-
143-
**Problem:** Functions imported via `use Test2::V0` are not recognized by the parser at compile
144-
time. The parser treats them as barewords, and subsequent tokens are misinterpreted as infix
145-
operators, producing syntax errors like:
146-
- `syntax error near "(' '"` (from `is join(...)`)
147-
- `syntax error near "( @vers "` (from `is scalar(...)`)
148-
- `syntax error near "(qw/"` (from `is editdist(...)`)
149-
- `syntax error near "=> sub"` (from `subtest foo => sub { }`)
150-
151-
**Root cause chain:**
152-
1. `Test2::V0` imports via `Test2::Util::Importer->import_into()`
153-
2. `optimal_import()` uses `*{"$from\::$_"}{CODE}` to extract code refs from glob slots
154-
3. This may fail in PerlOnJava, causing the import to not register functions
155-
4. Without registered functions, `SubroutineParser.java` line ~353 checks
156-
`nextTok.type != LexerTokenType.IDENTIFIER` — when the next token IS an identifier
157-
(like `join`, `scalar`), the unknown-sub-call path is skipped, returning a bareword
158-
5. The expression parser loop gives unknown identifiers default precedence 24, consuming
159-
them as infix operators, which then fails
160-
161-
**Diagnosis step:** Run:
162-
```perl
163-
./jperl -e 'use Test2::V0; print defined(&is) ? "is: OK\n" : "is: MISSING\n"'
164-
```
165-
This determines whether the issue is import-side or parser-side.
136+
## Phase 3: Parser and Import System Fixes (COMPLETED 2026-04-07)
137+
138+
### 3.1 Test2::V0 / Test2::Util::Importer function import chain ✅
139+
140+
**Problem:** `return` inside `map`/`grep` blocks only exited the block, not the enclosing
141+
subroutine. This broke `Test2::Util::Importer::optimal_import()` which uses `return` inside
142+
`map` to exit early from the import function.
166143

167-
**Fix approach (depends on diagnosis):**
168-
- If import-side: fix `*{...}{CODE}` glob slot extraction or string eval in Importer.pm
169-
- If parser-side: improve `SubroutineParser.java` to treat unknown subs followed by known
170-
CORE functions (join, scalar, etc.) as list operator calls
144+
**Root cause:** Map/grep blocks were compiled as `SubroutineNode` (anonymous subs), so `return`
145+
inside them returned from the block rather than propagating to the enclosing sub.
171146

172-
**Files:** TBD based on diagnosis
147+
**Fix:** Two-layer approach:
148+
1. **Return-value markers**: Map/grep blocks annotated with `isMapGrepBlock`; `return` creates
149+
`RuntimeControlFlowList(RETURN, returnValue)` marker returned as block result
150+
2. **Exception propagation**: `ListOperators.map()`/`grep()` detect RETURN markers and throw
151+
`PerlNonLocalReturnException`; `RuntimeCode.apply()` catches this in normal subs
152+
153+
**Files changed:** `PerlNonLocalReturnException.java` (new), `ControlFlowType.java`,
154+
`RuntimeControlFlowList.java`, `ParseMapGrepSort.java`, `JavaClassInfo.java`,
155+
`EmitSubroutine.java`, `EmitControlFlow.java`, `ListOperators.java`,
156+
`EmitterMethodCreator.java`, `RuntimeCode.java`, `InterpretedCode.java`,
157+
`BytecodeCompiler.java`, `BytecodeInterpreter.java`
173158

174159
### 3.2 `isa` infix operator precedence when feature-gated
175160

@@ -215,34 +200,104 @@ the dependency chain to work.
215200

216201
---
217202

203+
## Phase 5: Test2::IPC CallerStack Fix
204+
205+
### 5.1 INIT/CHECK/END blocks missing CallerStack entry
206+
207+
**Priority: HIGH** — Root cause of ~40 App::perlbrew test failures.
208+
209+
**Problem:** Tests using `Test2::Tools::Spec` load `Test2::AsyncSubtest``Test2::IPC`,
210+
which has an INIT block that calls `Test2::API::context()`. The `context()` function uses
211+
`caller(1)` to find the calling package, but INIT blocks in PerlOnJava execute directly
212+
from Java (`SpecialBlock.runInitBlocks()``RuntimeCode.apply()`) with no Perl caller
213+
frame above them. `caller(1)` returns empty → confess "Could not find context at depth 1".
214+
215+
**Loading chain for failing tests:**
216+
```
217+
Test2::Tools::Spec
218+
→ Test2::Workflow::Runner (line 9: use Test2::AsyncSubtest())
219+
→ Test2::AsyncSubtest (line 5: use Test2::IPC)
220+
→ Test2::IPC INIT block (line 26-29): context()->release()
221+
```
222+
223+
**Why passing tests work:** Tests using only `Test2::V0` don't load `Test2::IPC`.
224+
225+
**Root cause in PerlLanguageProvider.java:**
226+
- Line 161-177: `CallerStack.push("main", ...)` during parse, popped after parsing completes
227+
- Line 348: `runInitBlocks()` executes AFTER CallerStack was popped
228+
- Result: no CallerStack entry when `caller(1)` falls back to CallerStack
229+
230+
**Fix:** Push a CallerStack entry around INIT/CHECK/END block execution in
231+
`PerlLanguageProvider.executeCode()`, matching Perl 5 behavior where these blocks
232+
run from the main program scope.
233+
234+
```java
235+
if (isMainProgram) {
236+
CallerStack.push("main", ctx.compilerOptions.fileName, 0);
237+
try { runInitBlocks(); } finally { CallerStack.pop(); }
238+
}
239+
```
240+
241+
**Files:** `src/main/java/org/perlonjava/app/scriptengine/PerlLanguageProvider.java`
242+
243+
**Verification:**
244+
```bash
245+
./jperl -e 'use Test2::IPC; print "ok\n"'
246+
# Should print "ok" instead of "Could not find context at depth 1"
247+
```
248+
249+
### 5.2 Other remaining App::perlbrew test failures
250+
251+
**Priority: MEDIUM** — After 5.1, some tests will still fail for other reasons:
252+
253+
| Issue | Tests affected | Root cause |
254+
|-------|---------------|------------|
255+
| `can't get_layers on tied handle` | t/12.destdir.t, t/12.sitecustomize.t | PerlIO.java line 54 |
256+
| `B::SV` not implemented | t/util-looks-like.t | Test2::Util::Stash uses B:: introspection |
257+
| PATH in sub-shells | t/shell.t (local::lib) | `local::lib` test resets PATH, jperl shell script can't find `dirname`/`java` |
258+
| `sys()` returns undef | t/sys.t | Likely missing IPC::Cmd or similar |
259+
260+
---
261+
218262
## Implementation Priority Order
219263

220-
1. **Phase 2.1**`-` stdin support (unblocks `local::lib`)
221-
2. **Phase 3.1** — Test2::V0 import chain (unblocks most App::perlbrew tests)
222-
3. **Phase 2.2** — English.pm (quick win, unblocks format_perl_version test)
223-
4. **Phase 2.3** — catpath prototype fix (quick win, improves File::Which)
224-
5. **Phase 4.1** — FindBin $0 investigation
225-
6. **Phase 2.4** — blib/arch creation
226-
7. **Phase 3.2** — isa feature gate
264+
1. ~~**Phase 2.1**`-` stdin support~~
265+
2. ~~**Phase 3.1** — Non-local return from map/grep blocks~~
266+
3. ~~**Phase 2.2** — English.pm~~
267+
4. ~~**Phase 2.3** — catpath prototype fix~~
268+
5. **Phase 5.1** — CallerStack for INIT/CHECK/END blocks (unblocks ~40 App::perlbrew tests)
269+
6. **Phase 4.1** — FindBin $0 investigation
270+
7. **Phase 2.4** — blib/arch creation
271+
8. **Phase 3.2** — isa feature gate
272+
9. **Phase 5.2** — Remaining App::perlbrew test issues
227273

228274
---
229275

230276
## Progress Tracking
231277

232-
### Current Status: Phase 2 ready to start
278+
### Current Status: Phase 5.1 in progress
233279

234280
### Completed Phases
235281
- [x] Phase 1: Foundation Fixes (2026-04-07)
236282
- Added `$Config{startperl}` and `$Config{sharpbang}` to Config.pm
237283
- Added `$DynaLoader::VERSION = '1.54'` to DynaLoader.java
238284
- Created DynaLoader.pm stub for CPAN disk-based lookups
239285
- Module::Build::Tiny now configures, builds, and tests (32/32) successfully
286+
- [x] Phase 2: CLI and Core Module Fixes (2026-04-07)
287+
- 2.1: Added `-` stdin support in ArgumentParser.java (unblocked local::lib)
288+
- 2.2: Added English.pm core module
289+
- 2.3: Fixed File::Spec catpath/splitpath/abs2rel/rel2abs prototypes
290+
- [x] Phase 3.1: Non-local return from map/grep blocks (2026-04-07)
291+
- Two-layer approach: return-value markers + PerlNonLocalReturnException
292+
- Fixed Test2::Util::Importer::optimal_import() and all map/grep return semantics
293+
- Commit: f97aa6c1c
294+
- [x] Phase 4 (partial): Test2::V0 import chain now works
240295

241296
### Next Steps
242-
1. Implement `-` stdin support in ArgumentParser.java
243-
2. Diagnose Test2::V0 import chain
244-
3. Add English.pm
297+
1. Implement CallerStack fix for INIT/CHECK/END blocks (Phase 5.1)
298+
2. Re-run `./jcpan -t App::perlbrew` to measure improvement
299+
3. Investigate remaining failures (Phase 5.2)
245300

246301
### Open Questions
247-
- Is the Test2::V0 failure import-side or parser-side?
302+
- Will the CallerStack fix also help the main program execution (`runtimeCode.apply()` at line 356)?
248303
- Does FindBin `$0 = 'can_ok'` come from test harness or incorrect `-e` handling?

src/main/java/org/perlonjava/app/scriptengine/PerlLanguageProvider.java

Lines changed: 27 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -335,7 +335,15 @@ public static RuntimeList executePerlAST(Node ast,
335335
private static RuntimeList executeCode(RuntimeCode runtimeCode, EmitterContext ctx, boolean isMainProgram, int callerContext) throws Exception {
336336
runUnitcheckBlocks(ctx.unitcheckBlocks);
337337
if (isMainProgram) {
338-
runCheckBlocks();
338+
// Push a CallerStack entry so caller() inside CHECK/INIT/END blocks
339+
// sees the main program as their caller, matching Perl 5 behavior
340+
// where these blocks run from the main program scope.
341+
CallerStack.push("main", ctx.compilerOptions.fileName, 0);
342+
try {
343+
runCheckBlocks();
344+
} finally {
345+
CallerStack.pop();
346+
}
339347
}
340348
if (ctx.compilerOptions.compileOnly) {
341349
RuntimeIO.closeAllHandles();
@@ -345,7 +353,12 @@ private static RuntimeList executeCode(RuntimeCode runtimeCode, EmitterContext c
345353
RuntimeList result;
346354
try {
347355
if (isMainProgram) {
348-
runInitBlocks();
356+
CallerStack.push("main", ctx.compilerOptions.fileName, 0);
357+
try {
358+
runInitBlocks();
359+
} finally {
360+
CallerStack.pop();
361+
}
349362
}
350363

351364
// Use the caller's context if specified, otherwise use default behavior
@@ -357,7 +370,12 @@ private static RuntimeList executeCode(RuntimeCode runtimeCode, EmitterContext c
357370

358371
try {
359372
if (isMainProgram) {
360-
runEndBlocks();
373+
CallerStack.push("main", ctx.compilerOptions.fileName, 0);
374+
try {
375+
runEndBlocks();
376+
} finally {
377+
CallerStack.pop();
378+
}
361379
}
362380
} catch (Throwable endException) {
363381
RuntimeIO.closeAllHandles();
@@ -371,7 +389,12 @@ private static RuntimeList executeCode(RuntimeCode runtimeCode, EmitterContext c
371389
throw e;
372390
} catch (Throwable t) {
373391
if (isMainProgram) {
374-
runEndBlocks(false); // Don't reset $? on exception path
392+
CallerStack.push("main", ctx.compilerOptions.fileName, 0);
393+
try {
394+
runEndBlocks(false); // Don't reset $? on exception path
395+
} finally {
396+
CallerStack.pop();
397+
}
375398
RuntimeIO.closeAllHandles();
376399
}
377400
if (t instanceof RuntimeException runtimeException) {

src/main/java/org/perlonjava/core/Configuration.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ public final class Configuration {
3333
* Automatically populated by Gradle/Maven during build.
3434
* DO NOT EDIT MANUALLY - this value is replaced at build time.
3535
*/
36-
public static final String gitCommitId = "933de6e29";
36+
public static final String gitCommitId = "f97aa6c1c";
3737

3838
/**
3939
* Git commit date of the build (ISO format: YYYY-MM-DD).

0 commit comments

Comments
 (0)