fix: detect __DATA__/__END__ and no Module; terminators in source filters#389
Merged
Conversation
…ters When applying source filters, detect when the filtered output ends with a terminator pattern (__DATA__, __END__, or "no Module;") and stop filtering at that point. The remaining source after the terminator is appended unchanged. This is important for Filter::Simple which uses these terminators to mark where the filter should stop processing. Without this fix: - Content after __DATA__/__END__ would be lost or filtered incorrectly - Code after "no Module;" would be filtered when it should pass through Fixes: - Filter::Simple t/data.t - DATA section now preserved correctly - Filter::Simple t/filter.t - "no Module;" terminator now handled Note: t/filter_only.t still fails due to a separate POD parsing bug (content between =end and =cut not treated as POD). Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Per perlpod documentation, =end formatname only ends a =begin block, but does not exit the POD section. Content between =end and =cut should still be treated as POD documentation, not code. This fixes Filter::Simple filter_only.t which uses =begin/=end blocks. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
fe6a95b to
9153c9b
Compare
…odel In native Perl, source filters process remaining source incrementally during parsing, so each filter completes before the next filter module is loaded. In PerlOnJava, we tokenize upfront then apply filters, so multiple filter modules may be loaded before filters run. This caused @Transforms (a package variable in Filter::Simple) to accumulate transforms from different modules. When the second filter ran, its $multitransform closure would include the first module transforms, causing incorrect behavior. Fix: Make @Transforms lexical in FILTER_ONLY so each call has its own transform list. Mark Filter::Simple as protected in config.yaml. Also adds Filter::Simple to bundled modules (with tests). Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Improves source filter handling to detect terminator patterns and stop filtering at the appropriate point.
Problem
When applying source filters, Filter::Simple uses terminators like
__DATA__,__END__, andno Module;to mark where filtering should stop. Without proper terminator detection:__DATA__/__END__was being lost or filtered incorrectlyno Module;was being filtered when it should pass through unchangedSolution
After each filter chunk, check if the output ends with a terminator pattern:
__DATA__or__END__at end of lineno ModuleName;at start of line (with optional comment)When a terminator is detected, append the remaining source unchanged instead of continuing to filter.
Test Results
Filter::Simple tests:
Note: t/filter_only.t fails due to a separate PerlOnJava bug where content between
=end formatnameand=cutis not treated as POD.Test Plan
make- all unit tests passGenerated with Devin