Skip to content

WIP: Improve op/state.t pass rate (69 → 141/170) #414

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
fglock merged 6 commits into from
Mar 31, 2026
Merged

WIP: Improve op/state.t pass rate (69 → 141/170) #414

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1992b28
Fix interpreter state variable persistence and dynamic goto EXPR fall…
fglock Mar 31, 2026
06254d1
Add compile-time error for state variables in list assignment
fglock Mar 31, 2026
9a8c6aa
Update test improvement plan with op/state.t progress
fglock Mar 31, 2026
b6ae7f0
Add missing interpreter opcodes and fix lazy CORE:: subroutine genera…
fglock Mar 31, 2026
d9255ff
Fix GOTO/TAILCALL marker propagation in interpreter eval handling
fglock Mar 31, 2026
c915666
Handle goto \&NAME in JVM backend to avoid interpreter fallback
fglock Mar 31, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 38 additions & 3 deletions dev/design/test_pass_rate_improvement_plan.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ These are tests that start running but crash fatally, blocking all subsequent te
| Test File | Passed/Total | Blocking Bug | Tests Unblocked |
|-----------|-------------|--------------|-----------------|
| op/ref.t | 96/265 | `@UNIVERSAL::ISA` not followed in MRO | **156** |
| op/state.t | 69/170 | Dynamic `goto $variable` silently exits | **101** |
| op/state.t | 141/170 | Dynamic `goto $variable` → interpreter fallback + state fixes | **15+14** |
| op/heredoc.t | 66/138 | Heredoc inside `eval 's//<<~/e'` | **~60** |
| op/filetest.t | 227/436 | NPE from NUL in filename (`-T "TEST\0"`) | **5** |
| op/universal.t | 90/142 | `$1` undef after successful regex match | **38** |
Expand Down Expand Up @@ -252,7 +252,7 @@ These are small, targeted fixes that each unblock many tests:
| 1.2 | Null-check in `fileno()` for unopened handles | 8 (fh.t) | Tiny |
| 1.3 | Null-guard in file test operator for NUL-in-filename | 5 (filetest.t) | Tiny |
| 1.4 | `@UNIVERSAL::ISA` traversal in MRO | 156 (ref.t) | Small |
| 1.5 | Fix dynamic `goto $variable` | 101 (state.t) | Small |
| 1.5 | Fix dynamic `goto $variable` + state var persistence | ~~101~~ **done** (72 gained) | Small |
| 1.6 | Extend `vec()` to 64-bit widths | 28 (vec.t) | Small |
| 1.7 | Fix `$1` capture after successful match (state corruption) | 38 (universal.t) | Small |
| 1.8 | Fix unrecognized-switch error message (add trailing `.`) | ~56 (switches.t) | Tiny |
Expand Down Expand Up @@ -326,7 +326,42 @@ These tests are inherently incompatible with PerlOnJava and should not be target

---

## Tracking
## Progress Tracking

### Current Status: Phase 1 in progress

### Completed Fixes

#### PR #414: op/state.t — 69/170 → 141/170 (2025-03-31)

Branch: `fix/state-attribute-validation`

| Fix | Tests Gained | Category |
|-----|-------------|----------|
| Interpreter: state variable persistence (`STATE_INIT_*` instead of `RETRIEVE_BEGIN_*`) | ~26 | 1.5 |
| Interpreter: bare block `redo` (add `LoopInfo` push/pop) | (enabler) | 1.5 |
| JVM backend: dynamic `goto EXPR` triggers interpreter fallback | (enabler) | 1.5 |
| Parser: "state in list context forbidden" compile error | +46 | New |

Files changed:
- `src/main/java/org/perlonjava/backend/bytecode/BytecodeCompiler.java` — state var compilation fix + bare block redo
- `src/main/java/org/perlonjava/backend/jvm/EmitControlFlow.java` — dynamic goto EXPR fallback
- `src/main/java/org/perlonjava/frontend/parser/ParseInfix.java` — state-in-list validation

Remaining op/state.t failures (15 + 14 blocked):
- Tests 42-43: `:shared` attribute silently accepted (pre-existing)
- Tests 62-68: goto + state interaction (pass standalone, fail in full test context)
- Tests 94, 96, 98, 100: `given`/`when` (incomplete support)
- Tests 103, 106-107: state in closures/anon subs (state sharing between copies)
- Tests 157-170: blocked (execution stops at test 156)

### Next Steps

1. Continue Phase 1 quick wins (items 1.1-1.10)
2. Investigate op/state.t goto+state interaction failures in full test context
3. Run full test suite to measure overall progress

### Baseline

Update this document as fixes land. Use the test runner to measure progress:

Expand Down
94 changes: 92 additions & 2 deletions src/main/java/org/perlonjava/backend/bytecode/BytecodeCompiler.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2261,8 +2261,9 @@ void compileVariableDeclaration(OperatorNode node, String op) {
Boolean.TRUE.equals(node.annotations.get("isDeclaredReference"));

Integer beginId = RuntimeCode.evalBeginIds.get(sigilOp);
if (beginId != null || op.equals("state")) {
int persistId = beginId != null ? beginId : sigilOp.id;
if (beginId != null) {
// BEGIN-captured variable: use RETRIEVE_BEGIN_* (destructive removal from global storage)
int persistId = beginId;
int reg = allocateRegister();
int nameIdx = addToStringPool(varName);

Expand Down Expand Up @@ -2303,6 +2304,62 @@ void compileVariableDeclaration(OperatorNode node, String op) {
}
return;
}
if (op.equals("state")) {
// State variable without initializer: use STATE_INIT_* (non-destructive)
// This preserves the variable across subroutine calls
int persistId = sigilOp.id;
int reg = allocateRegister();
int nameIdx = addToStringPool(varName);

// Allocate a register for the undef/empty initial value
int undefReg = allocateRegister();

switch (sigil) {
case "$" -> {
emit(Opcodes.LOAD_UNDEF);
emitReg(undefReg);
emitWithToken(Opcodes.STATE_INIT_SCALAR, node.getIndex());
emitReg(reg);
emitReg(undefReg);
emit(nameIdx);
emit(persistId);
registerVariable(varName, reg);
}
case "@" -> {
emit(Opcodes.NEW_ARRAY);
emitReg(undefReg);
emitWithToken(Opcodes.STATE_INIT_ARRAY, node.getIndex());
emitReg(reg);
emitReg(undefReg);
emit(nameIdx);
emit(persistId);
registerVariable(varName, reg);
}
case "%" -> {
emit(Opcodes.NEW_HASH);
emitReg(undefReg);
emitWithToken(Opcodes.STATE_INIT_HASH, node.getIndex());
emitReg(reg);
emitReg(undefReg);
emit(nameIdx);
emit(persistId);
registerVariable(varName, reg);
}
default -> throwCompilerException("Unsupported variable type: " + sigil);
}

// If this is a declared reference, create a reference to it
if (isDeclaredReference && currentCallContext != RuntimeContextType.VOID) {
int refReg = allocateRegister();
emit(Opcodes.CREATE_REF);
emitReg(refReg);
emitReg(reg);
lastResultReg = refReg;
} else {
lastResultReg = reg;
}
return;
}

// Regular lexical variable (not captured, not state)
int reg = addVariable(varName, "my");
Expand Down Expand Up @@ -4963,6 +5020,17 @@ public void visit(For3Node node) {
emitInt(0);
}

// Push loop info so that redo/next/last inside bare blocks work
// (Perl 5 allows redo/next/last in bare blocks)
// Unlabeled bare blocks are targets for unlabeled redo/next/last;
// labeled blocks (like SKIP: { }) are only targeted by matching label.
int bodyStartPc = bytecode.size();
boolean isUnlabeledTarget = (node.labelName == null);
LoopInfo loopInfo = new LoopInfo(
isUnlabeledTarget ? null : node.labelName,
bodyStartPc, true);
loopStack.push(loopInfo);

enterScope();
try {
if (node.body != null) {
Expand All @@ -4976,12 +5044,34 @@ public void visit(For3Node node) {
exitScope();
}

// next jumps here (continue point = end of body, before exit)
loopInfo.continuePc = bytecode.size();

// Pop loop info and patch jump targets
loopStack.pop();
// Patch redo PCs to jump to body start
for (int pc2 : loopInfo.redoPcs) {
patchJump(pc2, bodyStartPc);
}
// Patch next PCs to jump to continue point
for (int pc2 : loopInfo.nextPcs) {
patchJump(pc2, loopInfo.continuePc);
}
// Patch last (break) PCs - will be patched to end label below
// (we need the end PC first)

if (node.labelName != null) {
emit(Opcodes.POP_LABELED_BLOCK);
int exitPc = bytecode.size();
patchJump(exitPcPlaceholder, exitPc);
}

// Patch last (break) PCs to jump past the block
int endPc = bytecode.size();
for (int pc2 : loopInfo.breakPcs) {
patchJump(pc2, endPc);
}

lastResultReg = outerResultReg;
return;
}
Expand Down
65 changes: 64 additions & 1 deletion src/main/java/org/perlonjava/backend/bytecode/BytecodeInterpreter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -948,6 +948,19 @@ public static RuntimeList execute(InterpretedCode code, RuntimeArray args, int c
}
}
if (!handled) {
// GOTO/TAILCALL markers inside eval should be caught
// (same as JVM backend's EmitEval: ordinal > 2 means not LAST/NEXT/REDO)
ControlFlowType cfType = flow.getControlFlowType();
if ((cfType == ControlFlowType.GOTO || cfType == ControlFlowType.TAILCALL)
&& !evalCatchStack.isEmpty()) {
// Set $@ to the error message
String errorMsg = flow.marker.buildErrorMessage();
GlobalVariable.setGlobalVariable("main::@", errorMsg);
// Jump to eval catch handler
pc = evalCatchStack.pop();
RuntimeCode.evalDepth--;
break;
}
return result;
}
}
Expand Down Expand Up @@ -1032,6 +1045,16 @@ public static RuntimeList execute(InterpretedCode code, RuntimeArray args, int c
}
}
if (!handled) {
// GOTO/TAILCALL markers inside eval should be caught
ControlFlowType cfType = flow.getControlFlowType();
if ((cfType == ControlFlowType.GOTO || cfType == ControlFlowType.TAILCALL)
&& !evalCatchStack.isEmpty()) {
String errorMsg = flow.marker.buildErrorMessage();
GlobalVariable.setGlobalVariable("main::@", errorMsg);
pc = evalCatchStack.pop();
RuntimeCode.evalDepth--;
break;
}
return result;
}
}
Expand Down Expand Up @@ -1565,7 +1588,11 @@ public static RuntimeList execute(InterpretedCode code, RuntimeArray args, int c
case Opcodes.CHOWN, Opcodes.WAITPID, Opcodes.FORK, Opcodes.GETPPID, Opcodes.GETPGRP,
Opcodes.SETPGRP, Opcodes.GETPRIORITY, Opcodes.SETPRIORITY, Opcodes.GETSOCKOPT,
Opcodes.SETSOCKOPT, Opcodes.SYSCALL, Opcodes.SEMGET, Opcodes.SEMOP, Opcodes.MSGGET,
Opcodes.MSGSND, Opcodes.MSGRCV, Opcodes.SHMGET, Opcodes.SHMREAD, Opcodes.SHMWRITE -> {
Opcodes.MSGSND, Opcodes.MSGRCV, Opcodes.SHMGET, Opcodes.SHMREAD, Opcodes.SHMWRITE,
Opcodes.SYMLINK, Opcodes.CHROOT, Opcodes.MKDIR,
Opcodes.MSGCTL, Opcodes.SHMCTL, Opcodes.SEMCTL,
Opcodes.EXEC, Opcodes.FCNTL, Opcodes.IOCTL,
Opcodes.GETPWENT, Opcodes.SETPWENT, Opcodes.ENDPWENT -> {
pc = executeSystemOps(opcode, bytecode, pc, registers);
}

Expand Down Expand Up @@ -2479,6 +2506,42 @@ private static int executeSystemOps(int opcode, int[] bytecode, int pc,
case Opcodes.SHMWRITE -> {
return SlowOpcodeHandler.executeShmwrite(bytecode, pc, registers);
}
case Opcodes.SYMLINK -> {
return MiscOpcodeHandler.execute(Opcodes.SYMLINK, bytecode, pc, registers);
}
case Opcodes.CHROOT -> {
return MiscOpcodeHandler.execute(Opcodes.CHROOT, bytecode, pc, registers);
}
case Opcodes.MKDIR -> {
return MiscOpcodeHandler.execute(Opcodes.MKDIR, bytecode, pc, registers);
}
case Opcodes.MSGCTL -> {
return MiscOpcodeHandler.execute(Opcodes.MSGCTL, bytecode, pc, registers);
}
case Opcodes.SHMCTL -> {
return MiscOpcodeHandler.execute(Opcodes.SHMCTL, bytecode, pc, registers);
}
case Opcodes.SEMCTL -> {
return MiscOpcodeHandler.execute(Opcodes.SEMCTL, bytecode, pc, registers);
}
case Opcodes.EXEC -> {
return MiscOpcodeHandler.execute(Opcodes.EXEC, bytecode, pc, registers);
}
case Opcodes.FCNTL -> {
return MiscOpcodeHandler.execute(Opcodes.FCNTL, bytecode, pc, registers);
}
case Opcodes.IOCTL -> {
return MiscOpcodeHandler.execute(Opcodes.IOCTL, bytecode, pc, registers);
}
case Opcodes.GETPWENT -> {
return MiscOpcodeHandler.execute(Opcodes.GETPWENT, bytecode, pc, registers);
}
case Opcodes.SETPWENT -> {
return MiscOpcodeHandler.execute(Opcodes.SETPWENT, bytecode, pc, registers);
}
case Opcodes.ENDPWENT -> {
return MiscOpcodeHandler.execute(Opcodes.ENDPWENT, bytecode, pc, registers);
}
default -> throw new RuntimeException("Unknown system opcode: " + opcode);
}
}
Expand Down
20 changes: 20 additions & 0 deletions src/main/java/org/perlonjava/backend/bytecode/CompileOperator.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -725,6 +725,26 @@ public static void visitOperator(BytecodeCompiler bytecodeCompiler, OperatorNode
case "setpgrp" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.SETPGRP);
case "getpriority" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.GETPRIORITY);
case "setpriority" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.SETPRIORITY);
case "symlink" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.SYMLINK);
case "chroot" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.CHROOT);
case "mkdir" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.MKDIR);
case "semctl" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.SEMCTL);
case "semget" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.SEMGET);
case "semop" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.SEMOP);
case "msgget" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.MSGGET);
case "msgsnd" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.MSGSND);
case "msgrcv" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.MSGRCV);
case "msgctl" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.MSGCTL);
case "shmget" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.SHMGET);
case "shmread" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.SHMREAD);
case "shmwrite" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.SHMWRITE);
case "shmctl" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.SHMCTL);
case "exec" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.EXEC);
case "fcntl" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.FCNTL);
case "ioctl" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.IOCTL);
case "getpwent" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.GETPWENT);
case "setpwent" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.SETPWENT);
case "endpwent" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.ENDPWENT);
case "opendir" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.OPENDIR);
case "readdir" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.READDIR);
case "seekdir" -> visitGenericListOpCase(bytecodeCompiler, node, Opcodes.SEEKDIR);
Expand Down
26 changes: 25 additions & 1 deletion src/main/java/org/perlonjava/backend/bytecode/Disassemble.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1823,7 +1823,19 @@ public static String disassemble(InterpretedCode interpretedCode) {
case Opcodes.GETPRIORITY:
case Opcodes.SETPRIORITY:
case Opcodes.GETSOCKOPT:
case Opcodes.SETSOCKOPT: {
case Opcodes.SETSOCKOPT:
case Opcodes.SYMLINK:
case Opcodes.CHROOT:
case Opcodes.MKDIR:
case Opcodes.MSGCTL:
case Opcodes.SHMCTL:
case Opcodes.SEMCTL:
case Opcodes.EXEC:
case Opcodes.FCNTL:
case Opcodes.IOCTL:
case Opcodes.GETPWENT:
case Opcodes.SETPWENT:
case Opcodes.ENDPWENT: {
rd = interpretedCode.bytecode[pc++];
int sysArgsReg = interpretedCode.bytecode[pc++];
int sysCtx = interpretedCode.bytecode[pc++];
Expand All @@ -1836,6 +1848,18 @@ public static String disassemble(InterpretedCode interpretedCode) {
case Opcodes.SETPRIORITY -> "setpriority";
case Opcodes.GETSOCKOPT -> "getsockopt";
case Opcodes.SETSOCKOPT -> "setsockopt";
case Opcodes.SYMLINK -> "symlink";
case Opcodes.CHROOT -> "chroot";
case Opcodes.MKDIR -> "mkdir";
case Opcodes.MSGCTL -> "msgctl";
case Opcodes.SHMCTL -> "shmctl";
case Opcodes.SEMCTL -> "semctl";
case Opcodes.EXEC -> "exec";
case Opcodes.FCNTL -> "fcntl";
case Opcodes.IOCTL -> "ioctl";
case Opcodes.GETPWENT -> "getpwent";
case Opcodes.SETPWENT -> "setpwent";
case Opcodes.ENDPWENT -> "endpwent";
default -> "sys_op_" + opcode;
};
sb.append(sysName).append(" r").append(rd)
Expand Down
Loading
Loading