Skip to content

Implement eval STRING variable capture in interpreter mode#193

Merged
fglock merged 6 commits intomasterfrom
feature/interpreter-eval-string-variable-capture
Feb 13, 2026
Merged

Implement eval STRING variable capture in interpreter mode#193
fglock merged 6 commits intomasterfrom
feature/interpreter-eval-string-variable-capture

Conversation

@fglock
Copy link
Copy Markdown
Owner

@fglock fglock commented Feb 13, 2026

Summary

Implements complete variable capture support for eval STRING in interpreter mode, achieving Perl 5 parity for dynamic eval workloads.

Key Features

1. Variable Capture for eval STRING (Commits 1-2)

  • InterpretedCode: Added variableRegistry field to track variable name → register mappings
  • BytecodeCompiler: Added constructor accepting parentRegistry for eval STRING context
    • Populates capturedVarIndices to mark parent scope variables as captured
    • Uses SET_SCALAR opcode for assignments to captured variables (preserves aliasing)
    • Disables ADD_ASSIGN optimization for captured variables
  • EvalStringHandler: Builds adjusted registry and captures variables from parent scope
  • BytecodeInterpreter: Preserves variableRegistry when creating closures

Fixes:

  • my $x = 1; for (1..10) { eval "\$x++" }; print $x now prints 11 ✓
  • my $x = 1; my $y = 2; eval "\$x = \$x + \$y" now updates $x to 3 ✓
  • Nested eval STRING with variable capture works correctly ✓

2. Numeric Literal Parsing with Underscores (Commit 3)

  • Strip underscores before parsing (Perl allows them as digit separators: 10_000_000)
  • Use ScalarUtils.isInteger() for consistent number validation with compiler
  • Handle large integers (>32-bit) by storing as strings (Perl 32-bit emulation)
  • Use LOAD_INT for regular integers to create mutable scalars (needed for ++/--)

Fixes:

  • my $x = 10_000_000; print $x now works ✓
  • for (1..100_000) { $x++ } now works ✓

3. Global Variable Increment/Decrement (Commit 5)

  • Implement ++ and -- for global variables in interpreter
  • Load global → apply increment/decrement → store back
  • Essential for dynamic eval patterns with computed variable names

Fixes:

  • $vartest++; print $vartest now prints 1 ✓
  • eval "\$vartest++"; print $vartest now prints 1 ✓
  • for my $x (1..N) { eval " \$var$x++" } now works ✓

Performance Benchmarks (Commits 4, 6)

Test 1: Cached eval STRING (Static - 10M iterations)

Implementation Time Speedup
Compiler 3.50s 1.0x (baseline) ✓
Perl 5 9.47s 2.7x slower
Interpreter 12.89s 3.7x slower

Winner: Compiler - Cached closure enables JIT optimization

Test 2: Dynamic eval STRING (Unique strings - 1M iterations)

Implementation Time Speedup
Perl 5 1.62s 1.0x (baseline)
Interpreter 1.64s 1% slower 🎯
Compiler 76.12s 4600% slower ✗

Winner: Interpreter - Achieves Perl 5 parity!

Key Results

  • 46x faster than compiler for dynamic eval (1.64s vs 76.12s)
  • Matches Perl 5 performance (1% slowdown)
  • Interpreter is the RIGHT TOOL for dynamic eval, not just "good enough"

Use Cases

The interpreter excels at:

  • Code generation (meta-programming patterns)
  • Templating (dynamic variable creation)
  • Dynamic eval (unique eval strings per iteration)
  • One-off code execution (no compilation overhead)
  • Development/debugging (faster iteration)

Technical Implementation

  • Variable Registry: Maps variable names to register indices for eval context
  • Captured Variables: Parent scope variables passed as array to eval'd code
  • Adjusted Registry: Remaps parent registers to eval scope (registers 3+)
  • SET_SCALAR Opcode: Preserves variable aliasing for captured variables
  • Global Variable Support: Full ++/-- support for package variables

Files Modified

  • src/main/java/org/perlonjava/interpreter/BytecodeCompiler.java
  • src/main/java/org/perlonjava/interpreter/BytecodeInterpreter.java
  • src/main/java/org/perlonjava/interpreter/EvalStringHandler.java
  • src/main/java/org/perlonjava/interpreter/InterpretedCode.java
  • src/main/java/org/perlonjava/interpreter/SlowOpcodeHandler.java
  • dev/interpreter/OPTIMIZATION_RESULTS.md

Testing

All unit tests pass. Manual testing confirms:

  • Variable capture in eval STRING works correctly
  • Nested eval STRING works
  • Numeric literals with underscores parse correctly
  • Global variable increment/decrement works
  • Dynamic eval achieves Perl 5 performance parity

🚀 This PR completes the eval STRING implementation, making the interpreter production-ready for dynamic code execution patterns.

fglock and others added 6 commits February 13, 2026 12:36
Fixed crash when eval receives a RuntimeList (from string interpolation) instead
of RuntimeScalar. The executeEvalString handler now properly handles both types by
converting RuntimeList to RuntimeScalar using scalar() method.

Before:
  eval "$x++"  # Crash: ClassCastException

After:
  eval "$x++"  # No crash (but variable capture not yet working)

Known Limitation:
Lexical variable capture in eval STRING is not yet implemented. Variables declared
in the outer interpreted scope are not accessible to the eval'd code. This requires
detecting variable references in the eval string and passing the corresponding
registers as captured variables.

Example that doesn't work yet:
  my $x = 1; eval "$x++"; print $x  # Prints 1 (should print 2)

See EvalStringHandler.java lines 86-94 for TODO.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds support for lexical variable capture in eval STRING, matching compiler
mode behavior. Variables from outer scope are now accessible and modifiable
within eval'd code.

Changes:
- InterpretedCode: Add variableRegistry field to track variable name → register
  index mappings for eval STRING support
- BytecodeCompiler: Add constructor accepting parentRegistry for eval STRING,
  populate variableRegistry in compile(), mark parent variables as captured
  using capturedVarIndices, use SET_SCALAR for assignments to captured
  variables instead of MOVE to preserve aliasing
- EvalStringHandler: Build adjusted registry and captured variables array from
  parent scope, pass to eval'd InterpretedCode
- BytecodeInterpreter: Preserve variableRegistry when creating closures
- Disable ADD_ASSIGN optimization for captured variables (use SET_SCALAR path)

Fixes:
- my $x = 1; for (1..10) { eval "\$x++" }; print $x  # now prints 11
- my $x = 1; my $y = 2; eval "\$x = \$x + \$y"       # now updates $x to 3
- Nested eval STRING with variable capture works correctly

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Perl allows underscores as digit separators in numeric literals (e.g.,
10_000_000). The interpreter was not handling these correctly while the
compiler mode was.

Changes:
- BytecodeCompiler.visit(NumberNode): Strip underscores before parsing,
  use ScalarUtils.isInteger() for consistent number validation, handle
  large integers (>32-bit) by storing as strings, use LOAD_INT for
  regular integers to create mutable scalars (needed for ++/-- operations)
- BytecodeCompiler range operator: Strip underscores when parsing
  constant range bounds

Implementation note:
We use LOAD_INT (creates new mutable RuntimeScalar) instead of cached
scalars because MOVE copies references, and variables need to be mutable
for operations like ++, --, etc. Floats use LOAD_CONST since they're less
commonly modified in-place.

Fixes:
- ./jperl --interpreter -e 'my $x = 10_000_000; print $x'  # now works
- ./jperl --interpreter -e 'for (1..100_000) { $x++ }'      # now works

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Documents real-world performance characteristics showing interpreter
excels at dynamic eval while compiler wins on cached eval.

Benchmarks:
- Cached eval (static string): Compiler 3.7x faster than interpreter
- Dynamic eval (unique strings): Interpreter 12.7x faster than compiler
- Dynamic eval vs Perl 5: Interpreter 4x slower, Compiler 50x slower

Key findings:
- Interpreter avoids compilation overhead for dynamic eval strings
- Compilation cost: 50-90ms per unique string (compiler) vs 15-30ms
  (interpreter) = 3-6x faster
- For 1M unique evals: Compiler 75s vs Interpreter 6s vs Perl 5 1.5s
- Interpreter design validated: excels exactly where it should

Primary use case: Dynamic eval strings for code generation, templating,
meta-programming.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The interpreter was throwing "Increment/decrement of non-lexical variable
not yet supported" when trying to increment/decrement global variables.
This is essential for eval STRING with dynamic variable names.

Changes:
- BytecodeCompiler.visit(OperatorNode): For ++ and -- operators, handle
  global variables by:
  1. Loading the global variable with LOAD_GLOBAL_SCALAR
  2. Applying PRE/POST_AUTOINCREMENT/DECREMENT opcode
  3. Storing back with STORE_GLOBAL_SCALAR
- Applies to both bare identifiers (x++) and sigiled operators ($x++)

Fixes:
- $vartest++; print $vartest  # now prints 1
- eval "\$vartest++"; print $vartest  # now prints 1
- for my $x (1..N) { eval " \$var$x++" }  # now works

This enables dynamic eval STRING patterns like code generation and
templating that create variables with computed names.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
After implementing global variable increment/decrement, the interpreter
achieves Perl 5 parity for dynamic eval workloads.

Updated benchmarks (1M unique eval strings):
- Perl 5: 1.62s (baseline)
- Interpreter: 1.64s (1% slower) ✓ Parity achieved!
- Compiler: 76.12s (4600% slower)

Key findings:
- Interpreter is 46x faster than compiler for dynamic eval
- Interpreter matches Perl 5 performance (1% slowdown vs 4600%)
- For 1M unique evals: 1.6s (interpreter) vs 76s (compiler)

Conclusion:
The interpreter isn't just "good enough" for dynamic eval - it's the
RIGHT tool, achieving native Perl performance where compilation overhead
would dominate.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@fglock fglock merged commit 8f97352 into master Feb 13, 2026
2 checks passed
@fglock fglock deleted the feature/interpreter-eval-string-variable-capture branch February 13, 2026 11:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant