Skip to content

fix(parser): do not consume keywords across whitespace in qualified names#613

Merged
fglock merged 1 commit intomasterfrom
fix/pkg-stash-keyword-parse
Apr 29, 2026
Merged

fix(parser): do not consume keywords across whitespace in qualified names#613
fglock merged 1 commit intomasterfrom
fix/pkg-stash-keyword-parse

Conversation

@fglock
Copy link
Copy Markdown
Owner

@fglock fglock commented Apr 29, 2026

Summary

Fixes a parser bug where %Foo:: and 2 (and similar) was rejected with a syntax error.

In real perl, qualified identifiers cannot contain whitespace around the :: separator. So:

  • %Foo:: and 2 → stash hash %Foo:: followed by the low-precedence operator and
  • %Foo::and → hash named "and" in package Foo (no space, single identifier)

PerlOnJava's IdentifierParser.parseComplexIdentifierInner was calling Whitespace.skipWhitespace immediately after consuming :: (and the legacy ' separator), which caused it to greedily pull the next keyword across the whitespace into the qualified name. So %Foo:: and 2 was parsed as %Foo::and 2 and rejected.

Why this matters

The bundled src/main/perl/lib/Dumpvalue.pm line 110 contains:

and %overload:: and defined &{'overload::StrVal'};

so any code path that loads Dumpvalue died at compile time. CPAN.pm lazily requires Dumpvalue from its error/dump handler, so the user-visible symptom was that running e.g.

$ ./jcpan -t JSON::Literal

died with syntax error at jar:PERL5LIB/Dumpvalue.pm line 110, near "&" instead of the actual CPAN diagnostic. Reduced repro:

$ ./jperl -e '%foo:: and 2 ; print "ok\n"'   # syntax error  (before)
$ perl     -e '%foo:: and 2 ; print "ok\n"'  # ok

Fix

Stop skipping whitespace after :: and ' in parseComplexIdentifierInner. The next token must be flush against the separator to be treated as a continuation of the qualified name. Keywords (and, or, etc.) are still allowed inside qualified names when there's no whitespace, e.g. $Foo::and, &UNIVERSAL::isa.

Test plan

  • New unit test src/test/resources/unit/stash_var_keyword_op.t covers:
    • %Foo:: and 1, %Foo:: or ..., %Foo:: xor 0, 1 and %Foo:: and 2
    • not %Foo::, 1 and %Foo:: (trailing stash with no operator)
    • %Foo:: && 1 (high-precedence form, regression check — was already OK)
    • keys %Foo::, scalar(keys %Foo::) cmp 0
    • %Foo::and (no space) still resolves to the hash named "and" in Foo
    • $Foo::or (no space) still resolves to the scalar named "or" in Foo
    • The exact Dumpvalue.pm line 110 pattern (and %overload:: and defined ...) compiles
  • Test fails on master, passes on this branch
  • make (full unit test suite) passes
  • ./jperl -e 'use Dumpvalue; print "ok\n"' now succeeds

Generated with Devin

…ames

In real perl, qualified identifiers like %Foo:: cannot contain whitespace
around the :: separator. So "%Foo:: and 2" tokenizes as the stash hash
%Foo:: followed by the low-precedence operator `and`, while "%Foo::and" (no
space) is the hash named "and" in package Foo.

PerlOnJava's IdentifierParser was skipping whitespace immediately after ::
(and after the legacy ' separator), which caused it to greedily pull the
next keyword into the qualified name. As a result, "%Foo:: and 2" was
parsed as "%Foo::and 2" and rejected with a syntax error.

This broke the bundled Dumpvalue.pm at line 110

    and %overload:: and defined &{'overload::StrVal'};

which in turn broke any code path that lazily required Dumpvalue. The
user-visible symptom was that CPAN.pm's error reporter (require'd from
CPAN/Shell.pm) failed to compile, so e.g. `jcpan -t JSON::Literal` died
with a confusing parse error instead of the actual CPAN error.

Fix: stop calling Whitespace.skipWhitespace after consuming :: or ' in
parseComplexIdentifierInner. The next token must be flush against the
separator to be treated as a continuation of the qualified name.

Adds a regression test (src/test/resources/unit/stash_var_keyword_op.t)
covering the keyword operators (and / or / xor / not / cmp), the
high-precedence forms (&&, ||) which were already correct, and the
exact Dumpvalue.pm pattern that triggered the original report.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock fglock merged commit 5402f21 into master Apr 29, 2026
2 checks passed
@fglock fglock deleted the fix/pkg-stash-keyword-parse branch April 29, 2026 09:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant