Skip to content

feat: use compiler tracers for project indexing#705

Draft
doorgan wants to merge 1 commit into
mainfrom
doorgan/compiler-tracers
Draft

feat: use compiler tracers for project indexing#705
doorgan wants to merge 1 commit into
mainfrom
doorgan/compiler-tracers

Conversation

@doorgan
Copy link
Copy Markdown
Contributor

@doorgan doorgan commented Jun 1, 2026

Very early WIP

The main idea of this PR is that we can use Compiler Tracers for all compiled files like NextLS does. This allows us to index definitions and references as the project compiles rather than as a separate step. As a consequence, the indexer no longer goes over .ex files, as those are compilable, and indexes all .beam files instead if a compiler tracer missed them.

Compiler tracers seem to provide all the information we need: aliases, alias references, remote/imported functions/macros, local functions/macros, struct expansion, and definitions(via the module's compiled bytecode), which in principle should allow us to skip indexing from source code, or in the worst case scenario reduce the source analysis to range refinement, which is faster than full AST analysis.

For non-compiled files like .exs scripts and tests, we still run the source code based indexer so we don't lose support for those.

Roughly speaking this works as follows:

  • File is saved, compilation is scheduled
  • The compiler emits tracer events, we pick them up and update both the index and the indexer manifest
  • The indexer runs, skips the .beams from files already analyzed via compiler tracer events, indexes the rest of the stale files

Later on for 0.4 we will want to investigate using the compiler locks to share the expert build with the user builds and save repeated work. That would be partially incompatible with compiler tracers because a user triggered build will not send the compiler tracer events to expert. In that case we want to also have the ability to index what's missing after compilation, and this PR should set the foundations for that.

There are a lot of rough edges I'm still figuring out, but the bare minimum is working.

@doorgan doorgan force-pushed the doorgan/compiler-tracers branch from 2b073da to da72261 Compare June 3, 2026 00:20
@doorgan doorgan force-pushed the doorgan/compiler-tracers branch from da72261 to a83aeef Compare June 3, 2026 00:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant