Skip to content

Use byte offsets for internal ranges#128

Merged
msujew merged 3 commits into
mainfrom
msujew/byte-range
Jul 23, 2026
Merged

Use byte offsets for internal ranges#128
msujew merged 3 commits into
mainfrom
msujew/byte-range

Conversation

@msujew

@msujew msujew commented Jul 10, 2026

Copy link
Copy Markdown
Member

To reduce on memory usage, this change removes the line/column information for all internal text ranges and only leaves the byte-offset ranges in place. On one hand, this simplifies some code, while making some other more complicated (especially LSP response handling).

Reduces the memory usage of tokens by 20% (16 bytes). Also removes 16 bytes from all AST nodes. Since we no longer have to track utf16 line/column infos, this speeds up the lexer by roughly 10%.

@msujew
msujew force-pushed the msujew/byte-range branch from 6ec43a3 to dbe2165 Compare July 10, 2026 21:46

@sailingKieler sailingKieler left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks pretty good to me, and makes sense.
In practice, I assume only the workspace symbol and document symbol providers might notably be affected performance-wise. In critical cases we always can modify the concrete ...Impl struct generation to mitigate potential runtime performance issues - if such ever show up, right?

I have just some API-related hints/questions.

Comment thread text.go Outdated
Comment thread ast.go Outdated
Comment thread internal/generator/parser_generator.go Outdated
@msujew
msujew force-pushed the msujew/byte-range branch from dbe2165 to 01e9344 Compare July 21, 2026 13:01
@msujew
msujew force-pushed the msujew/byte-range branch from 01e9344 to 385d8a8 Compare July 21, 2026 13:03

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.50.

Benchmark suite Current: b69a4a6 Previous: d803417 Ratio
BenchmarkWorkspaceCycle (typefox.dev/fastbelt/examples/statemachine) - MB/s 12.98 MB/s 5.21 MB/s 2.49

This comment was automatically generated by workflow using github-action-benchmark.

@sailingKieler sailingKieler left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really nice, many source portions feel much more streamlined.

I have just two very minor points, see below. 🚀

Comment thread test/doc_fixture.go Outdated
Comment thread text_range.go
@msujew
msujew merged commit 78ef32f into main Jul 23, 2026
5 checks passed
@msujew
msujew deleted the msujew/byte-range branch July 23, 2026 09:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants