fix(parser): process pending heredocs before use/no BEGIN-time evaluation#554
Merged
fix(parser): process pending heredocs before use/no BEGIN-time evaluation#554
Conversation
…aluation
`use Module LIST;` compiles and executes LIST immediately via
runSpecialBlock("BEGIN", ...) inside parseUseDeclaration. If LIST
contained a heredoc operator (e.g. `use constant FOO => <<'EOT';`),
the heredoc body was still un-resolved at emit time because heredoc
bodies are normally filled in only when whitespace processing consumes
the following NEWLINE — which had not happened yet. The emitter then
threw "HEREDOC marker <id> not found".
Mirror the existing fix for explicit BEGIN { ... } blocks in
SpecialBlockParser: before runSpecialBlock, if heredocs are pending,
seek to the next NEWLINE, call ParseHeredoc.parseHeredocAfterNewline
to fill bodies, record heredocSkipToIndex/heredocNewlineIndex so the
outer whitespace handler does not re-process them, then restore
tokenIndex.
Observed via `jcpan -t Text::Table` where t/10_Table.t failed all
166 subtests at a `use constant T_SINGLE => <<'EOT2';` declaration.
After the fix all 175 Text::Table tests pass across 7 files.
Generated with [Devin](https://cli.devin.ai/docs)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes a parser bug where heredocs used as arguments to
use/nodeclarations (e.g.use constant FOO => <<'EOT';) were not resolved before the import list was compiled and executed at BEGIN time, causing:Root cause
StatementParser.parseUseDeclarationevaluates the import list immediately viarunSpecialBlock(parser, "BEGIN", list, ...). Heredoc bodies are normally filled in by the whitespace handler when it consumes the NEWLINE after the statement, but that hasn't happened yet at the pointrunSpecialBlockruns — so the HEREDOC placeholder OperatorNode is still unresolved andEmitOperatorNode.handleMissingHeredocthrows.SpecialBlockParseralready had the same fix for explicitBEGIN { ... }blocks; theuse/nopath just wasn't mirroring it.Fix
Before
runSpecialBlock, if heredocs are pending, seek forward to the next NEWLINE, callParseHeredoc.parseHeredocAfterNewlineto fill the bodies, recordheredocSkipToIndex/heredocNewlineIndexso the outer whitespace handler does not re-process them, then restoreparser.tokenIndex.Discovered via
jcpan -t Text::Table→t/10_Table.tpreviously failed all 166 subtests at:Test plan
use constant NAME => <<'EOT',use constant { NAME => <<'EOT' }, anduse constant A => "x", B => <<'EOT'all compile and run correctly.jcpan -t Text::Table—Result: PASS, 175/175 tests across 7 files (was 0/166 int/10_Table.t).make(full unit test suite) —BUILD SUCCESSFUL, no regressions.Generated with Devin