Skip to content
This repository was archived by the owner on Apr 23, 2026. It is now read-only.

Add automated regression testing system#24

Merged
max-leuthaeuser merged 1 commit into
masterfrom
add-regression-testing
Apr 13, 2026
Merged

Add automated regression testing system#24
max-leuthaeuser merged 1 commit into
masterfrom
add-regression-testing

Conversation

@max-leuthaeuser

@max-leuthaeuser max-leuthaeuser commented Apr 13, 2026

Copy link
Copy Markdown
Collaborator

Automated regression testing framework that compares AST outputs across SwiftAstGen versions.

Features:

  • CI workflow running on PRs against master
  • Tests 3 Swift corpora (Alamofire, Vapor, RxSwift)
  • Comparative metrics: file counts, sizes, timing, diffs
  • Local runner for development (scripts/regression-local.py)
  • Reports posted as PR comments with artifact links

Files:

  • .github/workflows/regression.yml - CI workflow
  • scripts/regression.py - Main harness
  • scripts/regression-local.py - Local test runner

@max-leuthaeuser max-leuthaeuser force-pushed the add-regression-testing branch 5 times, most recently from 13b6a2c to 5cd56a1 Compare April 13, 2026 08:25
Automated regression testing that compares AST outputs across SwiftAstGen versions.

- CI workflow running on PRs against master
- Tests 3 Swift corpora: Alamofire 5.9.1, Vapor 4.99.3, RxSwift 6.7.1
- Generates comparative reports with metrics and AST diffs
- Local runner for development (scripts/regression-local.py)
- Reports posted as PR comments with workflow artifacts
@max-leuthaeuser max-leuthaeuser force-pushed the add-regression-testing branch from 5cd56a1 to 4e09aa7 Compare April 13, 2026 08:29
@github-actions

Copy link
Copy Markdown

SwiftAstGen Regression Report

PR: #24
PR ref: add-regression-testing @ de8dd19
Base ref: master @ fbae6a4

alamofire (Alamofire/Alamofire@5.9.1)

Metric base (main) PR #24 Delta
JSON files generated 42 42 0
Total JSON size 28.3 MB 28.3 MB +0.0%
Wall-clock time 3.3 s 3.3 s +2.1%
Files with AST diffs 0

vapor (vapor/vapor@4.99.3)

Metric base (main) PR #24 Delta
JSON files generated 208 208 0
Total JSON size 47.4 MB 47.4 MB +0.0%
Wall-clock time 5.7 s 5.0 s -13.4%
Files with AST diffs 0

rxswift (ReactiveX/RxSwift@6.7.1)

Metric base (main) PR #24 Delta
JSON files generated 153 153 0
Total JSON size 38.2 MB 38.2 MB +0.0%
Wall-clock time 4.1 s 3.9 s -3.3%
Files with AST diffs 0

📎 Full diff files available as workflow artifacts.

@max-leuthaeuser max-leuthaeuser merged commit e02c186 into master Apr 13, 2026
4 checks passed
@max-leuthaeuser max-leuthaeuser deleted the add-regression-testing branch April 13, 2026 08:45
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant