Skip to content

Improve lexing of unclosed escaped strings#2164

Closed
rubinovitz wants to merge 2 commits into
pallets:mainfrom
rubinovitz:codex/fix-unclosed-string-lexer-performance
Closed

Improve lexing of unclosed escaped strings#2164
rubinovitz wants to merge 2 commits into
pallets:mainfrom
rubinovitz:codex/fix-unclosed-string-lexer-performance

Conversation

@rubinovitz

Copy link
Copy Markdown

Fixes #2109.

This avoids running the general string regular expression when the lexer is already positioned at a quoted string inside a block, variable, or line statement. A small scanner finds the closing quote while respecting escaped quotes, or raises the same syntax error path immediately when the string is unterminated.

This keeps the existing regex for compatibility, but skips its backtracking-heavy failure case for unclosed strings with many escape sequences.

Validation:

@rubinovitz rubinovitz marked this pull request as ready for review May 4, 2026 06:06
@davidism davidism closed this May 4, 2026
@github-actions github-actions Bot locked as resolved and limited conversation to collaborators May 19, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

poor performance when parsing unclosed string with many escape characters

2 participants