Skip to content

Refactor source transformation pipeline to a NodeVisitor-only architecture with loader-level format-preserving rewrites#539

Closed
Copilot wants to merge 19 commits into
masterfrom
copilot/feature-direct-ast-transformation
Closed

Refactor source transformation pipeline to a NodeVisitor-only architecture with loader-level format-preserving rewrites#539
Copilot wants to merge 19 commits into
masterfrom
copilot/feature-direct-ast-transformation

Conversation

Copilot AI commented Apr 17, 2026

Copy link
Copy Markdown
Contributor

The framework now moves source transformation orchestration into SourceTransformingLoader with a single AST traversal + format-preserving print pass, while keeping StreamMetaData as a DTO of original parse artifacts (stream metadata, original AST, original tokens).

  • DTO-only StreamMetaData

    • StreamMetaData stores original metadata/AST/tokens only.
    • Source mutation/reload APIs were removed.
  • Centralized AST traversal and printing in loader

    • SourceTransformingLoader::transformCode() returns transformed AST (Node[]).
    • One central NodeTraverser with registered visitors is used.
    • Format-preserving pretty-printing (printFormatPreserving) is performed in loader using old AST/tokens + new AST.
  • NodeVisitor-only transformation pipeline

    • Legacy SourceTransformer runtime flow was removed.
    • AspectKernel::registerTransformers() block was removed.
    • SourceTransformingLoader::$transformers and addTransformer() were removed.
    • SourceTransformer interface was removed.
  • Typed visitor result reporting

    • Added NodeTransformerResultReporter with getNodeTransformerResult(): TransformerResultEnum.
    • Visitors report transformation state via this interface.
    • Loader aggregates visitor results without method_exists(...) checks.
  • Weaving converted to NodeVisitor

    • WeavingTransformer now participates in traversal as a node visitor and reports TransformerResultEnum.
    • Kernel now registers weaving in the node-visitor list.
  • Namespace-level context injection

    • Added FileNameInjectorNodeVisitor.
    • Context attributes are injected only on top-level Namespace_ nodes (including StreamMetaData DTO and original file name), not on every node.
    • MagicConstantTransformer reads filename context from namespace attributes and keeps local namespace-scope state.
  • Caching flow in stream filtering

    • Cache freshness check and cached-source fast-path run in SourceTransformingLoader.
    • On stale cache, loader transforms source, pretty-prints, and writes refreshed cache.
  • Docs / BC break

    • Added 4.0.0 changelog entry documenting removal of the legacy transformer approach as a BC break.
  • Tests updated for visitor pipeline

    • Transformer tests validate visitor traversal + format-preserving reconstruction.
    • Weaving tests now execute weaving through traversal (injector + weaving visitor) rather than direct transform() calls.
    • Constructor expectations remain in original no-parentheses/property-access form where required.
$newStmts = SourceTransformingLoader::transformCode($metadata);

$printer = new Standard();
$source = $printer->printFormatPreserving(
    $newStmts,
    $metadata->syntaxTree,
    $metadata->tokenStream
);

Copilot AI changed the title [WIP] Add direct AST-based format-preserving source-code transformation Refactor source transformation pipeline to AST format-preserving rewrites Apr 17, 2026
Copilot AI requested a review from lisachenko April 17, 2026 21:26
Comment thread src/Instrument/ClassLoading/SourceTransformingLoader.php Outdated
Comment thread src/Instrument/ClassLoading/SourceTransformingLoader.php Outdated
Comment thread src/Instrument/Transformer/CachingTransformer.php Outdated
Comment thread src/Instrument/Transformer/CachingTransformer.php Outdated
Comment thread src/Instrument/Transformer/CachingTransformer.php Outdated
Comment thread tests/Go/Instrument/Transformer/ConstructorExecutionTransformerTest.php Outdated
Comment thread tests/Go/Instrument/Transformer/ConstructorExecutionTransformerTest.php Outdated
Comment thread tests/Go/Instrument/Transformer/ConstructorExecutionTransformerTest.php Outdated
Comment thread tests/Go/Instrument/Transformer/FilterInjectorTransformerTest.php Outdated
Comment thread tests/Go/Instrument/Transformer/MagicConstantTransformerTest.php Outdated
@lisachenko

Copy link
Copy Markdown
Member

Also, tests are failing, need to fix them

Copilot AI changed the title Refactor source transformation pipeline to AST format-preserving rewrites Refactor source transformation pipeline to centralized AST visitors and loader-level format-preserving rewrites Apr 17, 2026
Copilot AI requested a review from lisachenko April 17, 2026 22:36
Comment thread src/Instrument/ClassLoading/SourceTransformingLoader.php Outdated
Comment thread src/Instrument/ClassLoading/SourceTransformingLoader.php Outdated
Comment thread src/Instrument/ClassLoading/SourceTransformingLoader.php Outdated
Comment thread src/Instrument/ClassLoading/SourceTransformingLoader.php Outdated
Comment thread src/Instrument/ClassLoading/SourceTransformingLoader.php Outdated
Comment thread src/Instrument/Transformer/ConstructorExecutionTransformer.php Outdated
Comment thread src/Instrument/Transformer/FilterInjectorTransformer.php Outdated
Comment thread src/Instrument/Transformer/MagicConstantTransformer.php
Comment thread src/Instrument/Transformer/MagicConstantTransformer.php Outdated
Comment thread src/Core/AspectKernel.php Outdated

Copilot AI commented Apr 17, 2026

Copy link
Copy Markdown
Contributor Author

Also, tests are failing, need to fix them

Fixed in 933b34b/ad660bb by removing CloningVisitor usage in these transformer tests and aligning visitor-based traversal/context handling. UI screenshot: N/A (no UI changes).

Copilot AI requested a review from lisachenko April 17, 2026 23:04
Comment thread src/Core/AspectKernel.php Outdated
Comment thread src/Instrument/ClassLoading/SourceTransformingLoader.php Outdated
Comment thread src/Instrument/ClassLoading/SourceTransformingLoader.php Outdated
Comment thread src/Instrument/ClassLoading/SourceTransformingLoader.php
Comment thread src/Instrument/Transformer/MagicConstantTransformer.php
Copilot AI requested a review from lisachenko April 17, 2026 23:27
@lisachenko

Copy link
Copy Markdown
Member

@copilot unit tests are failing

Copilot AI commented Apr 18, 2026

Copy link
Copy Markdown
Contributor Author

@copilot unit tests are failing

Fixed in df748e1 by updating the failing transformer unit tests to use CloningVisitor before applying visitors, so printFormatPreserving() receives distinct old/new AST trees and no longer crashes on lowest-dependency runs. UI screenshot: N/A (no UI changes).

@lisachenko

Copy link
Copy Markdown
Member

@copilot still build is failing.

Could you please also add GH token for composer to phpstan + phpunit to fix download error and to make phpstan/phpunit working

- name: Setup PHP
  uses: shivammathur/setup-php@v2
  with:
    ...
  env:
    COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Agent-Logs-Url: https://github.com/goaop/framework/sessions/7222fd61-2aa4-449b-a482-83a9be6df6f2

Co-authored-by: lisachenko <640114+lisachenko@users.noreply.github.com>

Copilot AI commented Apr 18, 2026

Copy link
Copy Markdown
Contributor Author

@copilot still build is failing.

Could you please also add GH token for composer to phpstan + phpunit to fix download error and to make phpstan/phpunit working
...

Implemented in a7d28cd: added COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }} to the Install PHP (shivammathur/setup-php@v2) steps in both .github/workflows/phpunit.yml and .github/workflows/phpstan.yml so Composer can use authenticated GitHub downloads. UI screenshot: N/A (no UI changes).

@lisachenko

Copy link
Copy Markdown
Member

@copilot phpstan and phpunit builds are failing, please fix them

Copilot AI and others added 3 commits April 18, 2026 08:35
Agent-Logs-Url: https://github.com/goaop/framework/sessions/20cf0406-3022-4120-9d3b-37dadd7d08c0

Co-authored-by: lisachenko <640114+lisachenko@users.noreply.github.com>
Agent-Logs-Url: https://github.com/goaop/framework/sessions/20cf0406-3022-4120-9d3b-37dadd7d08c0

Co-authored-by: lisachenko <640114+lisachenko@users.noreply.github.com>

Copilot AI commented Apr 18, 2026

Copy link
Copy Markdown
Contributor Author

@copilot phpstan and phpunit builds are failing, please fix them

Implemented in 56bbdaf (plus c96302d/44108f6): fixed the remaining PHPStan/PHPUnit failures by normalizing parser tokens to PhpParser\Token in StreamMetaData, tightening URI typing in SourceTransformingLoader, skipping anonymous-class new rewrites in ConstructorExecutionTransformer, and guarding filename attribute typing in MagicConstantTransformer. UI screenshot: N/A (no UI changes).

@lisachenko

Copy link
Copy Markdown
Member

@copilot update your own instruction in the repository to use PHP8.4 version for running the copilot workflow. Configure and install necessary version for code-review, ensure that composer install is completed successfully, otherwise stop.

Once composer will work - check build failure again and fix it

Copilot AI commented Apr 18, 2026

Copy link
Copy Markdown
Contributor Author

@copilot update your own instruction in the repository to use PHP8.4 version for running the copilot workflow. Configure and install necessary version for code-review, ensure that composer install is ...

Implemented in f9f9296 (plus 94ab8ab): added .github/workflows/copilot-setup-steps.yml to force PHP 8.4 for Copilot cloud-agent and run composer install as a required setup step, added matching repo instruction in CLAUDE.md, then rechecked build failures and fixed the reported issues in SourceTransformingLoader (cache freshness/URI guard) and StreamMetaData (stable tokenization for format-preserving printing). UI screenshot: N/A (no UI changes).

@lisachenko

Copy link
Copy Markdown
Member

@copilot please recreate this PR from scratch - keep all GIT changes, but make another branch and PR to start from clean context

Copilot AI commented Apr 18, 2026

Copy link
Copy Markdown
Contributor Author

@lisachenko I've opened a new pull request, #540, to work on those changes. Once the pull request is ready, I'll request review from you.

@lisachenko lisachenko closed this Apr 18, 2026
@lisachenko lisachenko deleted the copilot/feature-direct-ast-transformation branch April 18, 2026 09:20
@lisachenko

Copy link
Copy Markdown
Member

Still, not working ( AST pretty-printing often fails to keep new lines, breaking the line number mapping, therefore not possible at the moment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Direct AST-based format-preserving source-code transformation

2 participants