Skip to content

fix: fork-open exec emulation and pipe open %ENV passing#383

Merged
fglock merged 10 commits into
masterfrom
fix/fork-open-indirect-object
Mar 27, 2026
Merged

fix: fork-open exec emulation and pipe open %ENV passing#383
fglock merged 10 commits into
masterfrom
fix/fork-open-indirect-object

Conversation

@fglock
Copy link
Copy Markdown
Owner

@fglock fglock commented Mar 27, 2026

Summary

This PR contains multiple fixes for fork/pipe/exec emulation issues and related bugs discovered while testing Module::Build.

Fix 1: Handle indirect object syntax in fork-open exec emulation

Problem: Module::Build's _backticks() method uses this pattern:

my $pid = open *FH, "-|";
if ($pid) {
    return join "", <FH>;
} else {
    exec { $cmd[0] } @cmd;
}

The fork-open emulation was returning empty output because completeForkOpen() was not handling the indirect object syntax correctly.

Fix: Mirror the logic from regular exec() to extract the correct arguments.

Fix 2: Pipe open not passing %ENV to child processes

Problem: Pipe open (open($fh, "$command|")) was NOT passing the Perl %ENV hash to child processes. This caused:

  • Environment variables set in the parent (like $ENV{PERL5LIB}) to be invisible to children
  • Test::Harness and Module::Build tests to fail because PERL5LIB was not propagating

Fix: Added copyPerlEnvToProcessBuilder() method to both PipeInputChannel.java and PipeOutputChannel.java.

Fix 3: Test/Harness.pm relative path handling

Fixes Test/Harness.pm to convert relative paths to absolute paths so they work correctly in child processes that may run from a different directory.

Fix 4: blib.pm not setting PERL5LIB

Problem: The -Mblib switch only modified @INC in the current process but did not set PERL5LIB. Since PerlOnJava cannot use fork() to share address space, child processes did not inherit the blib paths.

Fix: Modified blib.pm to also set PERL5LIB so child processes can find modules in blib/lib and blib/arch.

Fix 5: File::Spec->canonpath not removing leading ./

Problem: The Java implementation of File::Spec->canonpath() did not strip leading ./ prefixes from paths, unlike the Perl version.

Example: canonpath("./lib/Simple.pm") returned "./lib/Simple.pm" instead of "lib/Simple.pm"

Impact: Caused Module::Build's DistGen->clean() to fail matching files, leading to incorrect file deletion.

Fix: Updated FileSpec.java to remove leading ./ while preserving ./ when it is the entire path.

Fix 6: Regex capture groups for optional groups

Problem: When a regex with optional capture groups (like m{^(.*/)?(.*)}s) matched, groups that did not participate were skipped in the return list instead of returning undef.

Example: my ($a, $b) = ("MANIFEST" =~ m{^(.*/)?(.*)}s) was yielding ($a = "MANIFEST", $b = undef) instead of ($a = undef, $b = "MANIFEST")

Impact: Broke File::Basename::dirname() which relies on this pattern, causing dirname("MANIFEST") to return "MANIFEST" instead of ".". This in turn caused Module::Build to incorrectly delete all files during clean().

Fix: Updated RuntimeRegex.java to include undef values for unmatched groups.

Fix 7: Regex capture groups for branch reset patterns

Problem: Fix 6 caused a regression (-8 tests) in re/pat.t for branch reset patterns ((?|...)). These patterns were returning undef values for groups in non-matching alternatives.

Root cause: Java's regex engine creates separate capture groups for each alternative in a branch reset pattern, while Perl reuses the same group numbers. When alternative 3 of (?|(p)(q)|(x)(y)|(a)(b)) matches "ab":

  • Java groups 1-4: null (from non-matching alternatives)
  • Java groups 5-6: "a" and "b"

With Fix 6, all 6 groups were returned as (undef, undef, undef, undef, "a", "b") instead of ("a", "b").

Fix: Track whether a pattern uses branch reset syntax and:

  • For branch reset patterns: skip null groups (original behavior)
  • For non-branch-reset: include undef for unmatched optional groups

This restores re/pat.t to 1065/1298 while preserving the File::Basename fix.

Fix 8: Document test environment variables

Added documentation to AGENTS.md about environment variables needed when running tests directly (JPERL_UNIMPLEMENTED, JPERL_OPTS, PERL_SKIP_BIG_MEM_TESTS).

Test Plan

  • Unit tests pass (make)
  • Manual test: exec { $cmd[0] } @cmd with fork-open now returns correct output
  • Environment variables now pass through pipe open to child processes
  • -Mblib now properly propagates blib paths to child processes
  • File::Spec->canonpath("./foo") correctly returns "foo"
  • File::Basename::dirname("MANIFEST") correctly returns "."
  • re/pat.t: 1065/1298 tests pass (no regression from Fix 6)
  • Branch reset: "ab" =~ /(?|(p)(q)|(a)(b))/ returns ("a", "b") correctly
  • Module::Build basic.t: Runs all 58 tests (previously stopped at test 20)
  • Module::Build compat.t: 138/165 tests pass

Generated with Devin

@fglock fglock changed the title fix: handle indirect object syntax in fork-open exec emulation fix: fork-open exec emulation and pipe open %ENV passing Mar 27, 2026
fglock and others added 10 commits March 27, 2026 14:33
The fork-open emulation (open FH, "-|") was not correctly handling
the indirect object syntax exec { $cmd[0] } @cmd used by Module::Build.

The completeForkOpen() method was passing raw flattenedArgs to ProcessBuilder,
but with indirect object syntax, flattenedArgs[0] is the program and
flattenedArgs[1] is the duplicate program name (argv[0]).

This fix mirrors the logic from the regular exec() path that correctly
extracts the program and skips the argv[0] argument.

Fixes Module::Build _backticks() which uses this pattern.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
When `open STDOUT, "> file"` redirects STDOUT to a file, the
RuntimeIO.selectedHandle was not updated. This caused `print`
without an explicit filehandle to still write to the original
stdout instead of the file.

The fix updates RuntimeGlob.setIO() to check if the old IO was
the currently selected handle, and if so, updates selectedHandle
to point to the new IO.

This enables proper STDOUT/STDERR redirect patterns used by
Module::Build and other modules.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
When `open $fh, ">&BAREWORD"` is used to duplicate a filehandle by
name, look up the bareword in the caller's package first before
falling back to main::. This fixes the "Unsupported filehandle
duplication: SAVEOUT" error when using patterns like:

    package MBTest;
    local *SAVEOUT;
    open SAVEOUT, ">&" . fileno(STDOUT);
    # ... redirect ...
    open STDOUT, ">&SAVEOUT";  # This now works

Uses RuntimeCode.getCurrentPackage() which leverages caller() to
get the package context at runtime, working for both JVM-compiled
and interpreter code.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The jar:PERL5LIB marker is an internal PerlOnJava convention for
accessing modules bundled in the JAR file. When Test::Harness spawns
child processes, it passes @inc entries as -I switches, but jar: paths
do not exist as filesystem directories.

Filter out jar:* entries from @inc in both _filtered_inc() and
_default_inc() to prevent passing invalid -I switches to child
test processes.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
PipeInputChannel and PipeOutputChannel were not copying Perl's %ENV
hash to the ProcessBuilder environment, causing environment variables
(including PERL5LIB) to be invisible to child processes spawned via
pipe open (e.g., open($fh, "$command|")).

This fix adds copyPerlEnvToProcessBuilder() to both classes, matching
the behavior already present in SystemOperator.java for system() and
exec().

Also fixes Test/Harness.pm to convert relative paths to absolute paths
so they work correctly in child processes that may run from a different
directory.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Since PerlOnJava can't use fork() to share address space, child processes
don't inherit @inc modifications. This caused tests using -Mblib to fail
when they spawn child processes (e.g., Module::Build compatibility tests).

The fix makes blib.pm set PERL5LIB in addition to modifying @inc, so
child processes can find modules in blib/lib and blib/arch.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The Java implementation of canonpath() was missing the logic to strip
leading "./" from paths, unlike the pure-Perl File::Spec::Unix version.

This caused issues in Module::Build where files stored with
"lib/Simple.pm" didn't match canonicalized "./lib/Simple.pm" paths,
leading to files being incorrectly deleted during clean().

Now canonpath("./lib/Simple.pm") returns "lib/Simple.pm" as expected,
while preserving "./" when it's the entire path (e.g., canonpath("./")
returns "./").

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Previously, when a regex with optional capture groups like m{^(.*/)?(.*)}s
matched against a string without a slash (e.g., "MANIFEST"), the optional
group that did not participate in the match was skipped in the return list.

Perl returns undef for groups that do not participate, so:
  my ($a, $b) = ("MANIFEST" =~ m{^(.*/)?(.*)}s);
Should yield: $a = undef, $b = "MANIFEST"

But PerlOnJava was yielding: $a = "MANIFEST", $b = undef

This broke File::Basename::dirname() which uses this pattern, causing
dirname("MANIFEST") to return "MANIFEST" instead of "." as expected.

The fix includes undef values in the return list for unmatched groups,
matching Perl behavior.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The previous fix to include undef for unmatched optional groups
(commit 4c3a0b4) caused a regression in branch reset patterns
(?|...). These patterns were returning undef values for groups
in non-matching alternatives.

The issue is that Java regex engine creates separate capture groups
for each alternative in a branch reset pattern, while Perl reuses
the same group numbers. When alternative 3 matches, Java returns:
- Groups 1-4: null (from non-matching alternatives 1-2)
- Groups 5-6: the matched values (from alternative 3)

With the undef fix, all 6 groups were being returned, breaking
assignment to fewer variables.

The fix tracks whether a pattern uses branch reset syntax and:
- For branch reset patterns: skip null groups (original behavior)
- For non-branch-reset: include undef for unmatched optional groups

This restores the re/pat.t test count to 1065/1298 while preserving
the File::Basename::dirname fix for patterns like m{^(.*/)?(.*)}s.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Added documentation about JPERL_UNIMPLEMENTED, JPERL_OPTS, and
PERL_SKIP_BIG_MEM_TESTS environment variables that perl_test_runner.pl
sets automatically but need to be set manually when running tests
directly with ./jperl.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock fglock force-pushed the fix/fork-open-indirect-object branch from 4ed242d to 02f44e6 Compare March 27, 2026 13:34
@fglock fglock merged commit 36a15d5 into master Mar 27, 2026
2 checks passed
@fglock fglock deleted the fix/fork-open-indirect-object branch March 27, 2026 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant